Behind the Cogent-Sprint Depeering

An anonymous reader brings an update to Sprint's depeering with Cogent, which we discussed a few days back — namely, Sprint's side of the story. According to them, no free peering contract had ever existed, Cogent refused to pay the bills to exchange traffic, and after a year Sprint gave Cogent 30 days notice of their intent to disconnect. During this 30-day period, when one or two connections (out of ten) per week were shut down, Cogent made no alternate arrangements to alleviate the impact on their customers — but they had a press release ready when Sprint snipped the final wire. It will be interesting to see how Cogent responds.

No comment from Cogent

They're unreachable due to no internet connection.

Re:They're back?

Are they connected again? The traffic is flowing.

The Spacing Guild must have complained.

The spice must flow!

Holy Shit

Cogent press release: "Sprint [severed its Internet connection to Cogent] in violation of a contractual obligation to exchange traffic with Cogent on a settlement free peering basis."

FACT: At no time did Sprint and Cogent enter into a contract for settlement free peering. In 2006, Sprint and Cogent formed a commercial trial agreement that ended in September 2007. Cogent was unable to satisfy the agreed-upon traffic exchange criteria within the trial agreement, yet refused to pay Sprint or disconnect from Sprint's network.

If what Sprint says is true, Cogent has just dug itself a hole and not just in the court room.

Either there was a settlement free peering contract in place, or there wasn't.
Cogent can spin all it wants, but they aren't actually supposed to lie in a press release.

Cogent's press release would definitely constitute a material statement, which means they could be hearing from the SEC for lying to the public and investors.

Re:Holy Shit

Feel free to correct me, but I seem to remember Cogent and Level-3 having a similar disagreement a few years back, with Cogent being just as intransigent then. Peculiar that a major peer like Cogent would be unable to resolve peering contracts twice in such a short period of time - were I a Cogent enterprise customer I'd be pretty concerned with the pattern that's developing here.

Re:Holy Shit

If Cogent was your sole provider at this point, you'd either be an idiot or completely out of touch. This isn't the first time this has happened with Cogent. It isn't the second either. And it surely won't be the last.

Sprint doesn't get off so easy though. They pulled what amounts to a "sure, try it risk free for 3 months. Cancel if you don't like it, but if you fail to inform us in triplicate you owe us hundreds of thousands of dollars." That's a crock.

Nobody who initiates a settlement-free peering agreement intends to pay for a transit connection if the trial period doesn't work out. If there was a serious chance that Cogent could be convinced to buy service from Sprint, Sprint would never have agreed to peering since the two types of connection are mutually incompatible. And Sprint damn well knows it.

Then too, Sprint probably decided to block Cogent's routes from the rest of their peers. Otherwise I expect Cogent would simply have routed around the downed connections. That was a step too far, one which makes Sprint a dangerous selection as your sole carrier as well.

Re:Holy Shit

They pulled what amounts to a "sure, try it risk free for 3 months. Cancel if you don't like it, but if you fail to inform us in triplicate you owe us hundreds of thousands of dollars."

That's not what Sprint said - "Following a three-month commercial trial agreement during June - September 2007, the peering trial data indicated that Cogent did not meet the minimum traffic exchange criteria agreed to by both parties. As a result, settlement-free peering was not established and Cogent was notified in writing of these results." Given Cogent's history, it seems that Sprint was just covering its bases and trying to find a way to work with Cogent instead of just telling them to go fark themselves when they first approached Sprint about a settlement-free agreement.

Re:Dishonorable

You've officially missed the point of the settlement-free peering trial. As mentioned in TFA, it is only beneficial to both parties in a settlement-free peering agreement if the amount of traffic passed between the two networks is almost equal.

After 3 months, the traffic from Cogent wasn't as much as Cogent claimed it would be, and Sprint opted not to continue with the creation of a real settlement-free peering contract.

Nobody pulled out of anything, Cogent claimed their network passed more traffic than it really did and Sprint had no motivation to let them leech for free.

Re:Holy Shit

You usually end up as the red-headed step child when your competitors charge $50-100/Mb for connectivity and you charge $4.

Re:Holy Shit

Is anyone actually surprised that Cogent is being dicks again? I'm going to believe Sprint's side of the story because: 1) they don't get into peering fights every other month, 2) they don't release nasty press releases about their former peer, and 3) they don't route traffic for their former peer into a black hole and blame someone else. If you or I didn't pay our bills, we'd be nuked *way* faster than Sprint is saying they gave Cogent. The length of time is probably because Sprint knows that Cogent would be assholes about it and that it would break traffic in a bad way.

So many people from the previous thread put all the blame on Sprint because of that horrible press release Cogent had ready the second the last circuit was turned off. Did they all forget that this is Cogent the king of depeering bitch fights that we're talking about? They always cry foul and scream about it every time they get depeered.

Re:Holy Shit

granted, if you own a larger network, you can extort smaller networks (and all of their customers) for money. but that makes Sprint the asshole, not Cogent. claiming that Cogent is in the wrong just because they've been de-peered in the past without actually examining the details of the conflict to see whether Sprint's claims make any sense is rather naive. this isn't like high school where one's merits are based on their popularity. getting picked on often doesn't automatically make you wrong.

If you have a larger network then you're probably handling more traffic on behalf of a smaller network than vice versa. By your rationale, any company or person that sets up a small network (e.g. 3 computers) should be able to get free connectivity from sprint or another backbone provider.

The way the peering thing usually works is when companies A and B decide to peer, they track the traffic that they pass to each other. If the traffic is roughly even then they don't charge each other for the peering. On the other hand, if things are lopsided such as 80% of the traffic being company A handling requests for company B's users, then company B usually is required to pay for the excess bandwidth that it's using.

Yep

It may help people to think of it on a personal level: So suppose you and I are neighbors. I have a nice business class cable connection, you have a nice business class DSL connection. Now turns out we do a lot of traffic between each other, which doesn't go particularly fast since it goes over the net but also since it turns out our connections are routed very differently and have a lot if hops to get to each other. So being geeks, we decide to fix this by peering our networks. We connect up a Cat-5 connection between our houses and set up routers to handle things. What's more, we share each other's net. So traffic goes out the connection based on who's got the shortest route. Also if one connection goes out, we use the other one exclusively till it comes back up.

Now we don't charge each other for this service. We both pay our own costs. I pay for my line, you pay for yours and so on. Hence we are peers. The reason we do this is we both benefit equally form the relationship.

Ok so then another neighbor finds out about this. He's on dialup and would like something better. We say sure you can join our network, but you have to pay. Why? Well he isn't providing us any value. He's just going to cancel his dialup and use our network. That's great, but he's got to help with the costs. Also, he doesn't have anything we want, we aren't going to be accessing his files, so there is no peer situation. We sell him access.

In the case of Cogent it would be like a 4th neighbor. He asks to peer. He says he's got a wireless connection to a great provider, plus lots of stuff we'd want. So we decide to let him on as a peer. However, it turns out to be false. His link is slow and high latency, so we end up just using ours. Further he just uses our connections, since they are better for everything. Finally, we find little data from him we want. So we tell him "Know what? You can pay us to stay on our network, but we aren't letting you peer because we don't get anything out of it."

The idea of peering is just that: You connect to your peers, you equals. Those networks that have data you want, and you have data they want. Since it is an equal agreement, both sides bear their own costs. In unequal agreements, like you purchasing a connection from your ISP, then you have to pay.

Customers jumping ship?

If I were a Cogent customer and unable to connect to the other half of the internet... I would bail and connect with another company that pays their bills. Contract or not - Cogent wouldn't get a dime from me after this B.S.

Where are the angered masses?

Sprint has Restored Cogent's Connection

The summary misses a key point: Sprint has restored its connection [datacenterknowledge.com] to Cogent, meaning the two companies can pursue their lawsuit and grievances without using customers as bargaining chips.

Not so fast.

Cogent is a Tier 1 network service provider (weather or not Sprint and L3 want it to be).

Cogent offers great service at an unbeatable price (4-5 USD per Mbps as opposed to the 15-20 or so Sprint and competitors are charging).

How does Cogent do this? They focused early on metro ethernet services and wave division, instead of wasting money in legacy technologies. They kept their vision clear, and their staff small (under 500 employees).

Cogent is the type of NSP we want as a Tier 1. A very strong backer of Net Neutrality, and no intention of trying to get into the entertainment business unlike Verizon, AT&T, etc. Cogent has a goal of offering the best service at the lowest price (the end result being realistically moving the US forward in terms of available bandwidth).

If you take a look at the CAIDA rankings [http://as-rank.caida.org/], you'll see that Cogent has surpassed Verizon Business (was UUNET) and Global Crossing, and is now right behind Sprint.

Cogent is growing, and if Sprint doesn't do something they're going to loose their no. 3 spot to them. So their strategy is to make a power play and force Cogent into a Tier 2 spot and create uncertainty in the eyes of current and potential customers.

As much as Sprint would like to position itself as a provider for Cogent, it's not. Sprint is a peer for Cogent with Cogent being an equivalent size of the current Sprint network, and larger than many of Sprints other peers.

The idea that Sprint doesn't get as much out of peering with Cogent as Cogent does peering with Sprint is absurd and PR propaganda to try and look like this was anything other than a power-play to keep a competitor at bay.

It will be interesting to see how this goes in court. If I were a Sprint customer I would seriously consider moving to Cogent.

On a side note, Sprint is one of the major opponents against Net Neutrality. Combine that with the fact that Cogent is offering the same level of service for a third the cost, and it's not hard to see why Sprint is trying to take Cogent out of the picture.

Re:Err, no.

Actually unless a third party agrees to transit traffic between Cogent and Sprint the Internet IS broken, there are hundreds of AS's unable to communicate. It's not a technology problem, there is plenty of interconnection, it is a political and financial problem (Layer 8, often the most troublesome).

Re:Err, no.

Multihoming doesn't fix the problem. It does double your costs.

For example, I know one site that was multi-homed. They had Sprint and a regional provider. The regional provider was de-peered by Cogent about a year ago, and the regional provider only buys transit from Sprint (they peer with many other networks).

Guess what, he couldn't reach the University he just executed a major contract with -- they are single-homed through Cogent.

And what do end users do? Multihome?

Re:Folk-Lore.

My lay understanding of the situatin is that, once routing tables are changed to reflect the new topography, most everything goes back to normal

My just as lay understanding is that that would be the case were Sprint not actively filtering [wikipedia.org] routes from Cogent.

In other words, assuming I understand the situation right, Sprint are taking active measures to make sure that routes that originate in the Cogent network never reach Sprint's customers.

Re:Folk-Lore.

I think you mean the other way around; Cogent filtered AS1239 paths, blamed Sprint, and offered Sprint customers free circuits. Cogent customers were still visible via XO through Sprint during this mess, but Cogent filtered the traffic.

Right, I think I'm starting to understand. Correct me if I'm wrong.

Over one year ago, cogent got free peering from Sprint, and subsequently filtered out all alternative routes to Sprint. I read this as Cogent blackmailing Sprint: if you disconnect us, there will be no connectivity between your customers and ours.

Last week, Sprint decided not to go with Cogent's blackmail, and disconnected Cogent with no advance notice to their (Sprint's) customers.

Re:Small ISPs are the most vulnerable

Cogent could have, at any time during this depeering, allowed the sprint bound traffic to route through one of their other peering points. This would have allowed their customers, including yourself, to continue to reach the entire internet even though it may have been slightly slower. It's the beauty of the internet, they could have easily routed the traffic elsewhere but they CHOSE to route sprint to a blackhole route.

Re:Small ISPs are the most vulnerable

This is not accurate.

When you peer with another network, it goes like this:

1) you only exchange routes for your customers.
2) Your routes should not be visible to the peer through any other transit or peering connections.

So if Sprint and Cogent were just exchanging routes and the peer session was removed, Cogent and Sprint no longer see each other. For them to see each other again to happen, one of them would have to pay someone else for the transit.

Sprint is not going to pay for transit. Cogent doesn't want to, or apparently, they don't even want to to do settlement based peering.

Regulation might be ok if it opened the tier 1 peering to more networks. Forcing large networks to peer with much smaller network is shifting the cost of transporting that traffic long distances to the larger network.

Equal size networks setting up connections in multiple locations should have the same benefit and cost to both networks.

Re:Small ISPs are the most vulnerable

You don't seem to understand how peering works. Pushing the Sprint conntent to any of there other peers would have violated there peering agreements and they would not have seen the routes from those peers anyway. The internet only works because all the teir 1's have statement free peering with all the other teir 1's (teir one being the engineering definition of not having any transit links not because the sales guy said so) Transit is when you pay somebody to take your traffic often you cant switch from transit to peering with the same company (if you want ot become a teir 1 you will probably have to pay a 3rd party for transit during the transition). On a peering session you normally only get routes from your peers network and people that are paying them for transit, this means you never get the full 250k or so routes from any one peer.

The tHing you have to realize is every other tier 1 hates cogent, they are one ones that figured out that bandwidth really does not have a high cost if you build out your network smartly. Reliability has it's costs but every carrier has there bad days some more than others. Cogent really only provides service where they can do it cheaply mostly major metro areas especially big shared office buildings. It's dirt cheap bandwidth if you can get it (think $400-1000 for a 100mb connection with no caps etc) and many business are willing to trade some reliability issues with cogent for paying less than there T1 for a pipe 66 times bigger. That being said your silly if cogent or sprint is your only provider.

As a side note this is one of the reasons to avoid tier 1 carriers they are fine if you have at least 2 of them but if you can only have one connection get a tier 2 thats paying 3 or more tier ones for bandwidth and has the capacity to loose any one carrier at any time. As a hint most tier one's have AS numbers below 3k.

Re:Small ISPs are the most vulnerable

As others have noted Sprint was routing Cogent traffic through XO, but Cogent was blackholing the return traffic. Had they updated their BGP the return traffic would have flowed and the end customers would have been unaffected.

Small ISPs need multihoming anyway

If your ISP is only getting its connectivity from Cogent, and isn't homed to multiple upstream ISPs, then they're at risk from any technical problems their Cogent link has as well as from any business problems Cogent has. If they need any regulatory help from the FCC, it's a requirement for Sprint to give them free Clues, not for Sprint to give Cogent free connectivity.

The Internet's a lot more stable than when I got involved with it 25ish years ago, or when small ISPs were a dime a dozen a decade and a bit ago, but it's still not 100% perfect. Back in the mid-90s, small ISPs provided dialup and email service, and they usually bought their first upstream T1 line from the cheapest provider available, but if they stayed up and running for a few months and started to fill it up, they almost universally bought their next upstream T1 from a different provider, because Internet routing flapped all the time, and if you had two providers you were not only less susceptible to your connection failing, you were much less likely to lose connection to half the world whenever a butterfly flapped its wings near MAE-WEST. In fact most ISPs these days can give you a reasonable service level agreement and also a reasonable level of service, but your ISP needs some sort of redundant connection.

Of course, if you think this is a mess, just look at the shape the IPv6 world is in - randomly-connected archipelagos of random little islands, tunnelled together by a maze of twisty little passages.

(Disclaimer: I work for an ISP that's not part of this dispute, but this is entirely my own opinion, not theirs.)

Re:Small ISPs are the most vulnerable

Uh, because Sprint gave them a free trial then they refused to pay for over a year after they were notified they did not meet free peering requirements? How long can you not pay your bills before you get cut off?

Sprint did not cut off their customers; I am a Sprint customer who gets a full BGP table. I could still see Cogent and their customers through XO, but Cogent was dropping return traffic into a black hole.

Re:Do we need regulation?

Two countries sharing a massive border. One has a highly regulated banking sector, one preaches deregulation.

One has a catastrophic banking failure leading to necessary "socialization" of corporate losses. The other has the most stable banking system in the first world.

Screaming "regulation is bad" without considering the situation is duckspeak, and no more intelligent than saying "anything socialist is bad" or "windows is always bad" just because it appears to be a popular viewpoint.