Forgot your password?
typodupeerror
Google Bug Communications Software IT

How Google Broke Itself and Fixed Itself, Automatically 125

Posted by timothy
from the arise-phoenix-arise dept.
lemur3 writes "On January 24th Google had some problems with a few of its services. Gmail users and people who used various other Google services were impacted just as the Google Reliability Team was to take part in an Ask Me Anything on Reddit. Everything seemed to be resolved and back up within an hour. The Official Google Blog had a short note about what happened from Ben Treynor, a VP of Engineering. According to the blog post it appears that the outage was caused by a bug that caused a system that creates configurations to send a bad one to various 'live services.' An internal monitoring system noticed the problem a short time later and caused a new configuration to be spread around the services. Ben had this to say of it on the Google Blog, 'Engineers were still debugging 12 minutes later when the same system, having automatically cleared the original error, generated a new correct configuration at 11:14 a.m. and began sending it; errors subsided rapidly starting at this time. By 11:30 a.m. the correct configuration was live everywhere and almost all users' service was restored.'"
This discussion has been archived. No new comments can be posted.

How Google Broke Itself and Fixed Itself, Automatically

Comments Filter:

"A mind is a terrible thing to have leaking out your ears." -- The League of Sadistic Telepaths

Working...