- Ethereum 2.0’s Medalla testnet is back online following a synchronization error that brought down the network.
- Testnet node participation rates are back to roughly 70% of pre-crash levels.
- Developers are designing a more detailed strategy in case the network must respond to future issues.
Share this article
Ethereum 2.0’s testnet is back online following a multi-day outage, according to current network statistics.
Medalla is saved!
INFO blockchain: Synced new block block=0x68a46008… epoch=3428 finalizedEpoch=3424 finalizedRoot=0xf0e6b2fe… slot=109700
— prestonvanloon.eth (@preston_vanloon) August 19, 2020
Outage Comes to an End
On Aug. 14, Ethereum 2.0’s Medalla testnet crashed due to a Cloudflare outage that affected node synchronization. This issue prevented nodes from validating blocks and processing transactions.
One of the crucial issues was the dominance of the network clients called Prysm and Lighthouse. A diversification between clients is necessary to keep the network intact if some client breaks.
The issue wasn’t addressed, which led to almost complete loss of the network’s participants during August 15-16. The nodes running the popular Prysm clients jumped four hours forward and started to make blocks for the chain that didn’t exist.
It turned out Prysm had a design flaw. The client relied on the third party, Cloudflare, to fetch the time. Meanwhile, Cloudflare’s time provision service errored and messed up clocks for a large part of Medalla’s nodes.
The problem was compounded when developers introduced an emergency fix that conflicted with Cloudflare re-enabling its time service. The testnet “could have recovered on its own if we had not acted at all,” Ethereum developer Raul Jordan explains.
This outage caused participation to drop from roughly 80% to 5% over the course of a few days.
As of Aug. 20, the testnet is nearly back to full participation according to 24-hour stats, with…