A misconfigured node caused the Solana network to stop processing transactions and go offline on Friday, the fourth major outage for the popular blockchain since January, when it had a string of partial outages for most of that month, according to data from Solana.
This outage comes one year after a nearly 18-hour outage last September. Meanwhile, the crushing crypto winter has sent SOL, the No. 9 coin by market cap, down 81% in 2022.
At 7:01 pm EST on Friday night, the Solana Status site, operated by the Solana Foundation, posted that the network was “experiencing degraded performance” and that Solana developers were working on diagnosing the issue. Shortly after, Solana posted that the network is “experiencing an outage and not processing transactions.”
The Solana network is experiencing an outage and not processing transactions. Developers across the ecosystem are working on diagnosing the issue and to restart the network. More information will be provided as it becomes available.
— Solana Status (@SolanaStatus) October 1, 2022
What caused the outage?
“It appears a misconfigured node caused an unrecoverable partition in the network,” the Solana validator behind Stakewiz.com tweeted about the incident.
They translated further for Decrypt over Twitter DM: “A validator was running a duplicate validator instance. Meaning when it was their turn to produce a block, they produced one from each instance, for the same slot, so some validators saw the one block, some the others, then couldn’t agree which one was correct.”
Stakewiz said the Solana codebase should have handled the issue, but for some unknown reason, it caused an unrecoverable partition or fork. They believe the misconfiguration was an accident, “probably a failed node failover setup.” Developers restarted the Solana mainnet network at the last confirmed slot, 153139220. At the time of publishing, the reboot was 49% completed, and Solana had been down for 2 hours and 45 minutes.
Solana Mainnet restart is underway, we need 80% of stake to restart.
Track it here: https://t.co/2tIgKBCAHO
— Laine ❤️ stakewiz.com (@laine_sa_) October 1, 2022
What has Solana said about its outages?
Solana Labs did not immediately respond to Decrypt’s request for comment on this outage.
While Solana execs have quibbled in the past with varying tallies in crypto media of the total number of outages, founder Anatoly Yakovenko told Real Vision earlier this month that outages have been “our curse,” and pointed out that the Bitcoin blockchain has at times in its history had hours between two blocks.
One of a slew of smart-contract blockchains hyped as “Ethereum killers,” Solana is a popular blockchain for minting non-fungible tokens, NFTs, and decentralized applications. After Ethereum transitioned to proof of stake on September 15, Solana is now the number three proof-of-stake blockchain by market cap behind Ethereum and Cardano.
But if it’s going to continue to grow, and rival Ethereum, the network will need to stay up.