Become a fan of Slashdot on Facebook

 



Forgot your password?
typodupeerror
×
The Internet Businesses Networking

Blackout Shows Net's Fragility 287

It doesn't come easy wrote to mention a ZDNet article discussing a recent outage between Level 3 Communications and Cogent Communication. A business feud inadvertently highlighted the fragility of the Internet's skeleton. From the article: "In theory, this kind of blackout is precisely the kind of problem the Internet was designed to withstand. The complicated, interlocking nature of networks means that data traffic is supposed to be able to find an alternate route to its destination, even if a critical link is broken. In practice, obscure contract disputes between the big network companies can make all these redundancies moot. At issue is a type of network connection called 'peering.' Most of the biggest network companies, such as AT&T, Sprint and MCI, as well as companies including Cogent and Level 3, strike "peering agreements" in which they agree to establish direct connections between their networks. "
This discussion has been archived. No new comments can be posted.

Blackout Shows Net's Fragility

Comments Filter:
  • by hkmwbz ( 531650 ) on Friday October 07, 2005 @10:27AM (#13739256) Journal
    As I understand it, these were about the same size and had an agreement, or didn't bother to bill each other. Then suddenly one of them figured out that "hey, we are bigger, so they should pay us!"... And the smaller one cut off the connection because they didn't want to pay since they considered themselves to be as big as their rival.

    What I don't get is why one of them would suddenly want the other to pay up. What's changed now, and why does the smaller company have to pay the big one's bills?

    Am I missing something here?

  • by SolusSD ( 680489 ) on Friday October 07, 2005 @10:29AM (#13739272) Homepage
    yesterday? I swear the entire net came to a crawl yesterday afternoon.
  • by dpilot ( 134227 ) on Friday October 07, 2005 @10:30AM (#13739290) Homepage Journal
    This statement popped up in some of my security readings. It's most "efficient" to have one path between two places, and it's most "efficient" to set up peering agreements to route packets. But these efficient measures can introduce single points of failure.

    On a similar note, that's why there are 13 root DNS servers, and why most of us aren't supposed to use them. The DNS example though, is one where efficiency and robustness agree. It's more efficient, at least in terms of net bandwidth, to use a DNS server closer than the root servers.
  • by digitaldc ( 879047 ) on Friday October 07, 2005 @10:31AM (#13739292)
    http://www.gamergod.com/article_display.cfm?articl e_id=329 [gamergod.com]
    Good article on this situation here

    This situation has adversely affected various users of both companies' services. The inability of Level 3 to handle this situation in a fair and equitable manner to the consumers has alienated many customers and will continue to do so until the current situation is remedied. At what point is it good customer service to discontinue services due to no fault of said consumer base? Market history shows us that the single worse thing a company can do is to arbitrarily allow influences beyond the control of consumers to negatively impact services, determined by consumers to be status quo, without any warning or notification. If left unresolved and unaddressed, the current situation could set dangerous precedents for internet users across the country by allowing service providers to instantly discontinue provided services at the moment they feel that the services they provide are not being adequately compensated for from outside companies.

    On a side note, I was listening to Howard Stern (oh no!) this morning and he said that his Time Warner internet connection at home didn't work. Howard then called a tech guy to come and fix the problem, only for him to call a help desk to figure out what happened. The help desk didn't even know what was wrong. It sounds like Level 3 just pulled the plug and didn't notify ANYONE. Or maybe it was Cogent, the point is nobody outside of that dispute KNEW what was going on.
    This sounds like a good way to alienate your customers and/or ruin your business model. But that is just my opinion.
  • A New Approach (Score:3, Interesting)

    by mysqlrocks ( 783488 ) on Friday October 07, 2005 @10:31AM (#13739295) Homepage Journal
    So, it appears a big part of the Internet traffic is controlled by large companies like Cogent or Level 3. No big surprise. I think this highlights the need for a new approach to connecting people together. I know there's been talk of wireless mesh networks where everybody is both an end point and a router. This would work in populated areas but I'm not sure how well it would work for "long haul" connections which is what the issue is here. Can anybody think of (or know of) any alternatives that gives control and power of the Internet back to the people who use it?
  • by boldtbanan ( 905468 ) on Friday October 07, 2005 @10:37AM (#13739354)
    As I understood the problem, redundancy wasn't an issue. Level 3 was actively filtering out request to Cogent, however they came in. The redundancy was working, but Level 3 was playing NetNanny and blacklisting all Cogent IPs.
  • Governmental Role? (Score:1, Interesting)

    by slipnslidemaster ( 516759 ) on Friday October 07, 2005 @10:38AM (#13739368)
    I don't believe in large government but this strikes me as one of those things that government is good for.

    Why wouldn't it be a good thing for some governmental agency to regulate the physical location of various installations? It seems to me that many providers use the same colocations to house their equipment. It would seem smart that there would be some regulation to prevent all the Internet eggs in one basket.

    Create several more physical IXP's that are located in geographically diverse areas with redundant connections. Then regulate that only a limited number of companies could colocate together within a certain number of IXP's.

    This could prevent one companies "disagreement" with another from effecting the traffic being routed to an alternative link.

    Does this make sense?
  • Re:Ask Slashdot (Score:4, Interesting)

    by AlexTheBeast ( 809587 ) on Friday October 07, 2005 @10:39AM (#13739374)
    The problem with web services is that they need for the internet to be completely secure and completely reliable. The internet of today is neither.

    Physicians trying to use the internet to take care of critically ill patients are already experiencing this. Radiologists sitting home reading films are seeing this as well.

    Is 100% on neccessary? Hell, VoIP is making money like crazy over this unstable network of ours.

    My suggestion is to test with people that will understand the limitations of your service. Then get a little VC money to spread your servers out.
  • ah peering (Score:4, Interesting)

    by bigpat ( 158134 ) on Friday October 07, 2005 @10:41AM (#13739401)
    The only time peering should involve an ongoing exchange of money for bandwidth should be when a network is primarily serving as an intermediary between other networks, such as long haul or backbone networks.

    But if most of the traffic from other networks is going to customers that are connected and already paying for your network's service then it makes no sense and is simply wrong for a network to start charging other network providers. It breaks the end to end communication model and is providing your customers with less than the service they are paying for. People pay for internet connectivity so they can transfer data between other users on the internet, not just the ones on your company's network.

    If money exchanging hands is at all appropriate in this case it might be for the actual installation of routing equipment which establishes the physical connection between networks.

  • not a blackout (Score:2, Interesting)

    by bradk500 ( 921156 ) on Friday October 07, 2005 @10:48AM (#13739444)
    All this crap about it showing weakness in the internet is uninformed bs. They didn't just stop peering, but they are actively blocking traffic from cogent. If Level3 had just stopped peering the traffic would reroute around the problem. The only time you will see problems is if your a cogent customer trying to get a single homed computer on level3's network. We are a cogent customer and an internap customer, and to get around the problem I just reouted traffic destined for level 3 networks over one of our internap t's. This solved the issue for us.
  • Cogent Sucks (Score:2, Interesting)

    by Lamont ( 3347 ) on Friday October 07, 2005 @10:53AM (#13739504)
    As a customer who has had Cogent inflicted on us (when Verio sold all their domestic internet lines to Cogent), we've had nothing but pain and bumbling inefficiency from them for the last six months.

    I contacted Cogent's "premium" help desk last night when I found that I was suddenly no longer able to get to our networks in Australia. The tech had no idea that his own company was in the middle of a huge peering battle with L3. I had to tell them!
  • This was predictable (Score:5, Interesting)

    by PhilipPeake ( 711883 ) on Friday October 07, 2005 @10:58AM (#13739557)
    The Internet was designed to be resiliant to malfunctions and automatically take appropriate action to ensure connectivity.

    Unfortunately, that is not the Internet that we have today. In the original Internet, every router knew about every network connected to the Internet. Most networks had connectivity to many other networks. Discovery protocols allowed alternative routes to be discovered if one failed.

    Today, we don't have a (mostly) fully connected net, we have ISPs who don't know anything about networks which they don't "own", only that certain IP prefixes need to be passed to ISP x, y or z.

    This makes the infrastructure much more fragile than it was originally intended to be. We ended up with this for a few reasons. First, the wimpy routers in use at the time had limited memory available to hold the network maps. The answer chosen was to no longer attempt to hold a full world view, but to divide the world into regions, certain IP prefixes would "belong" to those regions, and all any router would need to know about was networks in its region, plus how to route traffic to other regions, who would take care of routing within the region. This led to "backbone" connections - high capacity links needed because all traffic between regions now didn't "diffuse" through the network, but was channeled into specific connections. It also set the scene to allow the net to be commercialised, those regional centers were obvious "choke points" that an enterprising company could own and pretty much dictate the pricing to lower level enterprises who would do the dirty work of dealing with end-users.

    Slowly but sureley the Internet evolved into a system dependent upon a few companies with high-speed links between them - prime candidates BTW, as locations for government control to be imposed. The self-healing nature of the original Internet was lost because all traffic HAS to pass via the top level companies infrastructure and over their interconnect backbone connections.

    The "self healing" Internet is long gone.

  • Monitor it yourself (Score:5, Interesting)

    by dereference ( 875531 ) on Friday October 07, 2005 @11:11AM (#13739676)
    I found this site while trying to research the problem. I wish I had known of it earlier; it provides a very nice (near) real-time snapshot of all the Tier 1 peering:

    http://www.internetpulse.net/ [internetpulse.net]

    I'm not affiliated with them in any way, and I'm sure there are other similar sites, but I thought it was worth mentioning.

  • Internet cannot route when your providers do not want you to communicate. Nothing can protect you in this case.
    I agree with the first part, not the second part. What protects you if your providers don't want you to communicate is:
    1. contracts that state that your provider is required to allow you to communicate
    2. competition from other providers
  • by tchuladdiass ( 174342 ) on Friday October 07, 2005 @11:59AM (#13740130) Homepage
    The thing I don't under stand is, say you have two network providers, A and B. If A's customers are sending more data to B's customers, then should A pay B for the route, or should B pay A because it is their customers that are requesting the packets?
  • by Red Flayer ( 890720 ) on Friday October 07, 2005 @12:09PM (#13740232) Journal
    Slowly but sureley the Internet evolved into a system dependent upon a few companies with high-speed links between them - prime candidates BTW, as locations for government control to be imposed. The self-healing nature of the original Internet was lost because all traffic HAS to pass via the top level companies infrastructure and over their interconnect backbone connections.

    This is what happens when you have an industry based upon a high cost of entry (physical infrastructure, here) and a low marginal cost of supply. We need fat pipelines because we demand fast speeds and high volumes for our traffic. If we didn't have regions, but instead had the "original self-healing internet," how long do you think it would take to download big files if the source didn't happen to be just 2 or 3 routers away? Say goodbye to streaming video, etc.

    Net cost of transmission would be far higher for packets that are many routers away in a truly web-based system, since not all apths are equal.

    The problem is, how do we balance cheap efficiency (fatline "superhighways") with expensive redundancy to optimize the system for all participants?
  • by fafaforza ( 248976 ) on Friday October 07, 2005 @12:19PM (#13740323)
    Level3's stock price is tanking, they are fighting for survival and the jobs of all their employees, from engineers to secretaries, while Cogent is undercutting the price of bandwidth by a factor of 3 while taking advantage of their peering with not many being able to compete on price, and you call Level3 greedy? C'mon.
  • by LurkerXXX ( 667952 ) on Friday October 07, 2005 @01:15PM (#13740808)
    That's exactly what it looks like. And yes, the routers are set to find another path. The problem is when it finds a new path through some 3rd or 4th ISP to the Level3 network, as soon as the Level3 router sees the packet originated from a Cogent IP address, it null routes it. That's not a problem with fragility of the net, it's Level3 behaving badly. (Note: Cogent should have ponied up money for traffic to a larger provider to avoid this mess in the first place. There are no good guys involved in this.)
  • Userfriendly? (Score:2, Interesting)

    by wembley fraggle ( 78346 ) on Friday October 07, 2005 @01:22PM (#13740868) Homepage
    Is this why I can't read userfriendly [userfriendly.org] or Something Positive [somethingpositive.net] this morning? Or is it just some weird coincidental webcomic blackout?
  • by WuphonsReach ( 684551 ) on Friday October 07, 2005 @01:27PM (#13740928)
    Ah yes, the joys of thinnet. OTOH, it was very easy to debug if you knew how the thinnet was routed from cubicle to cubicle. When you had a broken segment, you went halfway down the line and terminated it off. If the segment started working, your problem was farther away from the bridge (repeater?). Otherwise, you would head back upstream towards the head of the segment and try again. Where you typically ran into trouble were users who constantly moved equipment (test labs, laptop users). User training fixed most of those issues due to the informal posse of coworkers who would hunt down the frequent offender.

    The previous topology in that office had been thicknet (where you had to manually tap the cable). Thinnet was seen as better. Or at least easier to build a network out of in a cubicle environment.

    Token Ring wasn't all that bad. Unlike thinnet, the physical wiring was more of a topology like today's ethernet where you had a dedicated cable running from the patch panel to the workstation's network jack. At least, it was wired that way in the buildings where I've seen it. So it was easy enough to plug/unplug stations from the network in a central location. The topology was also designed to deal with a single break (the stations before/after the break would loopback).

    The usual problems we had with TR were the fragile connectors (problematic for test environments / laptop users with frequent plug/unplug). Plus the issue that you only had 4Mbps (later 16Mbps) and a 4Mbps card wouldn't work on a 16Mbps network. Ethernet hubs/switches did a much better job of handling the upgrade path automatically where one port might be 10Mbps another 100Mbps and a third port running at 1Gpbs without redoing your entire network topology.

It is easier to write an incorrect program than understand a correct one.

Working...