Lightning Took Down Amazon, Microsoft Clouds

A lightning strike in Dublin on Sunday caused a power outage in data centers belonging to Amazon and Microsoft, causing the companies’ cloud services to go offline.

Lightning hit a transformer, sparking an explosion and fire which caused the power outage at 10:41 AM PDT, according to preliminary information, Amazon wrote on its Service Health Dashboard. Under normal circumstances, backup generators would seamlessly kick in, but the explosion also managed to disable some of those generators.

By 1:56 PM PDT, power to the majority of network devices had been restored, allowing Amazon to focus on bringing EC2 (Elastic Compute Cloud) instances and EBS (Elastic Block Storage) volumes back online. But progress was slower than expected, Amazon said a couple of hours later.

“We know many of you are anxiously waiting for your instances and volumes to become available, and we want to give you more detail on why the recovery of the remaining instances and volumes is taking so long,” the company wrote at 11:04 PM PDT. “Due to the scale of the power disruption, a large number of EBS servers lost power and require manual operations before volumes can be restored … While many volumes will be restored over the next several hours, we anticipate that it will take 24-48 hours until the process is completed.”

Read More…