VMware Causes Second Outage While Recovering From First 215
jbrodkin writes "VMware's new Cloud Foundry service was online for just two weeks when it suffered its first outage, caused by a power failure. Things got really interesting the next day, when a VMware employee accidentally caused a second, more serious outage while a VMware team was writing up a plan of action to recover from future power loss incidents. An inadvertent press of a key on a keyboard led to 'a full outage of the network infrastructure [that] took out all load balancers, routers, and firewalls... and resulted in a complete external loss of connectivity to Cloud Foundry.' Clearly, human error is still a major factor in cloud networks."
Re:'An inadvertent press of a key on a keyboard' (Score:5, Funny)
This pretty much describes my entire career.
Don't let it happen again (Score:5, Funny)
Re:Slashdot summary non sensationalist (Score:5, Funny)
Don't go knocking my typewriter
It's Electric, and has wonderful BNC connector
for network access. IBM, you did good.
Re:Game Over (Score:2, Funny)
Has anyone mentioned Skynet yet?
Re:This is very bad design (Score:3, Funny)
Sounds like they could benefit from a virtual environment to test things out in.
Re:UR DOING IT WRONG! (Score:5, Funny)
I would like more elaboration on what "touched the keyboard" means.
It was an extreme case of static discharge. The engineer is lucky to be alive -- when doing cloud computing, thunderstorms are a huge hazard.
Re:A cloud in need, is a cloud indeed (Score:4, Funny)
No, no, it is indeed a cloud: Thin, wispy and ephemeral.
Not to mention The Cloud is dangerous!
One time, "The Cloud" corrupted a few files on my server, toasted my dev machine's hard drive (couldn't even re-install!) made several monitors explode, and split the tree outside my home-office completely in two; Flying chunks of bark shattered my windows... to say nothing of the horror that became of the decorative landscape lighting that foolishly linked the outside to my main electrical system, may it rest in pieces.
The ironic thing is that I had a lightning rod installed; I thought I was safe from The Cloud, but The Cloud decided that my, now deceased, 200ft pine tree was a better target of opportunity.
The Cloud is a scary concept -- Super charged flying electrical batteries, always looming overhead, unpredictably destroying their targets with tremendous power, and surgical precision. Hell, the terror of witnessing such an event has permanently emotionally scarred my dog -- She has a prescription for Valium now because she hyperventilates and continuously shakes for hours at the mere sound of distant thunder...
My psyche is not unscathed either: I have to take a tranquilizer whenever I hear the words: "To The Cloud"