Blackout Worse For Internet Than Previously Thought?

An anonymous reader writes "Renesys (the people who previously brought you cool animated graphs of the US/Canada power outage has a new report out. It challenges the widely held belief that the Internet was largely unaffected by the power outage. Lots of important networks lost connectivity, including banks, hospitals, government organizations and investment funds. There's a cool appendix on the huge Italian power outage in September as well. They conclude that the Internet is not ready to be critical infrastructure."

worked just fine here during blackout

[bbn]

I live in denmark and recently we had a blackout that lasted maybe 10 hours.

While I was unable to make any phone calls, I could get on the internet with GPRS and surf to our server with my laptop for as long as the laptop batteries lasted.

The server is hosted in a colo datacenter which was also in the middle of the affected area. We run a mud on the server, and most of the players are from USA. They never discovered the blackout as the datacenter went on emergency diesel backup and apparently knew to make business with backbone providers that also knew their stuff.

So to the people saying that internet can only route around blackout areas but not _through_ them, this is not true. Seems at least here in denmark all the infrastructure on the backbones got backup power and just keeps working when everyone else is busy lighting candles.

Re: Power Outage

[tibike77]

Well, look at it this way... they say "an UPS is good enough, if power goes out it will go out a few seconds or maybe a half hour", and don't plan for a "worse-case" scenario, in which you have a few hours of "power outage"... so instead of saving everything, commiting caches and so on, they just keep on hoping "in a few seconds power will be back on"... I just hope they DID learn their lesson now, and cut back on cutbacks (lol).

Re:Critical Infrastructure?

[tmu]

This is certainly a topical comment, but it misses the point a little (I think).

A large number of organizations that were multi-homed, using BGP to announce routes out multiple upstream providers lost connectivity. This speaks to the situation that people who have spent a bunch of money on network infrastructure may not have spent enough on power (or may not have carefully evaluated their upstream providers).

One of the organizations located in the study had nine (9!) upstream providers and still went out. This is not a case of people on the far end of a DSL link; this is the case of people not being able to put together reliable network connectivity, even in the face of multi-homing.

Re:That's fine

[kwerle]

Nor most modern phones - which need electrical. Nor traffic lights. Lucky I don't depend on that kinda stuff being up all the time!

Re:Obvious?

[tmu]

The ability to observe the outage (sharply) through routing activity is definitely the part that we thought was coolest.

People are saying two different things here: 1) well, duh, if power is out lots of people can't connect to the web; 2) if the core of the internet routes around that who cares. These are both interesting points. Here are some thoughts:

1) We agree. That's what I though. But read the keynote press releases. Or just google on 'blackout Internet' and you'll find glowing stories about how 'the Internet' didn't even blip under the blackout. We prove pretty conclusively that this is incorrect.

2) The core of the Internet did, indeed, route around the outage. This is good. What is less good is that thousands of networks within the outage area lost connectivity, either due to lost power themselves, or upstreams that lost power (or telcos who lost battery backup on csu/dsu units, or whatever). These are *not* DSL customers (or that grade, anyway). All of these are BGP-speaking networks with their own Autonomous Systems and their own prefixes.

The fact that so many networks went down is significant, given that many organizations are coming to rely on the Internet as a critical communications infrastructure.

my T-Mobile mobile worked great

[Anonymous Coward]

It worked the whole duration of the 9 hour outage. The only problem I had was reaching local numbers; all the trunks for reaching local numbers were filled up. FirstEnergy's outage reporting number gave a busy signal right after I hit send or dialed it on a landline after about 10 minutes (I tried to call it a few times because I was pissed off; I was missing a good episode of Jerry :(

Then my landline died like 3 hours later. Completely. No voltage what so ever. but my mobile worked for the whole duration of the outage. Couldn't make any local calls for the first 30 minutes or so, but, oh well.

Re:Infrastructure

[addaon]

While I know that was meant as a joke, it's important to point out that the power grid /isn't/ used for critical infrastructure. No hospital, or air traffic control station, or powerplant (oh, the irony) would be caught dead without a backup power system.

Re:And the power system is?

[Rick.C]

Pointing out that areas without power didn't have internet connectivity seems rather redundant to me.

For home users and small businesses, you are quite right. What about large businesses that invested in generators so they could stay online 24/7? They were prepared to remain online to conduct their business. They depended on the Internet and it failed them.

I work for a large bank. We were not hit by the power outage, but we were scrambling to find routes around the areas that were.

Re:And the power system is?

[orangesquid]

I think the implied problem was the connectivity that was provided by ISPs and backbone segments running off the affected sections of the power grid.

If the Internet were more redundant and ad-hoc (less backbone-centric), it would recover from problems better. That's how it was originally envisioned; unfortunately, the commercialization of NSFNet has largely destroyed this approach, for better or worse.

We have a more organized network, but it's very dependent on critical points because of it's multiplexing organization strategy, so when that fails...

I was at an internet center

[Servo]

I live and work in the NYC metro area, and was at work when the blackout started. I didn't notice there was a blackout until I walked outside and saw our generators on. For the record, I work for a company that provides services to large internet datacenters. Any datacenter worth its monthly fee wasn't affected by the power outage. Yes, individual institutions including banks etc etc who weren't prepared did lose connectivity, but backbone providers and large carrier centers in the area didn't skip a beat.

Re:I was at an internet center

[LostCluster]

Just wondering... even though you had normal capacity through the blackout, did your site maintain normal usage? Having the datacenter up is nice, but datacenters only exist to store information generated in the "real world".

If a datacenter's up, but nobody's online to use it, do the servers still hum?

I am moderatly impressed

[axelbaker]

The internet impresses me every day. Its the ridiculous expectations of people that blow my mind.

When the world trade center went down, I worked at a major ISP. Verizon is right next door to the WTC, not surprisingly all the main trunks were destroyed. Connectivity for much of the Atlantic including Europe was disrupted. Many carriers had cell towers on top of the building as well. Even from California I didn't need to be told the internet was going to be f#@*ed up on the east. Yet some how people in NYC who had to travel to NJ to find a working phone would call me and ask why their DSL was down.

The fact that the whole north east had no power, and the majority of the internet worked shows the internet has done a very good job of doing just what its designed for. It could do a better job. So lets work on that instead of just talking about it.