ISPs & P2P, Getting Along Without Getting Cozy

penguin-geek writes "Researchers at Northwestern University have discovered a way to ease the tension between ISPs and P2P users. As we all know, there's been a growing tension between Internet Service Providers (ISPs) and their customers' P2P file-sharing services, and this has driven service providers to forcefully reduce P2P traffic at the expense of unhappy subscribers and the risk of government investigations. Recently, some ISPs have tried to fix the problem through partnerships with certain P2P applications. The Ono project represents an alternative solution: a software service that allows P2P clients to efficiently identify nearby peers, without requiring any kind of cozy relationship between ISPs and P2P users. Using results collected from over 150,000 users, they have found that their system locates peers along paths that have two orders of magnitude lower latency and 30% lower loss rates than those picked at random by BitTorrent, and that these high-quality paths can lead to significant improvements in transfer rates. In challenged settings where peers are overloaded in terms of available bandwidth, Ono provides a 31% average download-rate improvement; in environments with large available bandwidth, Ono increases download rates by 207% on average (and improves median rates by 883%). Ono is available as a plugin for the Azureus BitTorrent client, an open tracker and an standalone service you can integrate into any P2P system."

Standard

Should be made standard into the apps if it does all that it claims.

The problem is that it is stupid.

They are looking at the PHYSICAL location of the machines.

As far as I am aware, most bittorrent clients already search for the machines with the fewest hops and lowest latency. Translation: machines on the same NETWORK as them.

Because if I am on Comcast at home and you have DSL through ATT at home and our homes are within 500' of each other ... that means NOTHING with regard to hops and latency between us.

Re:The problem is that it is stupid.

While your argument makes some sense in theory, it doesn't change the fact that this project is apparently reporting some very straightforward numbers which seem to indicate that in practice your point doesn't hold much water. I understand what you're getting at, but a 207% average speed increase is a 207% average speed increase. If you've investigated and gotten different results, please feel free to share. How directly that translates into a savings in bandwidth for the provider, I don't know, but I don't think that's what the GP was getting at.

Re:The problem is that it is stupid.

I have been using Ono for about 6 months now. When I installed it, it made very little difference at all. I usually get pretty good speeds though, with or without. I am still using the plugin now (with azureus) and am using it more because i'm too lazy to uninstall it, then for the speed increase (if any).

It sounded cool, but didn't work for me. I am curious if anyone else noticed similar findings, or if I am all alone.

Re:

Yeah I'm not seeing how this is going to be too useful in most cases. If you have enough seeds that you can afford to pick and choose which ones you download from, you're going to be getting high speeds anyway. If you have low speeds, you're not going t

Re:The problem is that it is stupid.

Except they aren't only looking at the physical location of the machines. They are basically merging both network and physical location to come up with a hybrid location mapping that provides the lowest latency route.

From the FAQ:
Does this really work? In a paper pending publication, we show that our lightweight approach significantly reduces cross-ISP traffic and over 33% of the time it selects peers along paths that are within a single autonomous system (AS). Further, we find that our system locates peers along paths that have two orders of magnitude lower latency and 30% lower loss rates than those picked at random, and that these high-quality paths can lead to significant improvements in transfer rates.

And that is their flaw.

If I may ...

... Further, we find that our system locates peers along paths that have two orders of magnitude lower latency and 30% lower loss rates than those picked at random ...

And THAT is the problem with this work.

The current torrent clients do not RANDOMLY pick an address. They check latency and hops.

Sure, it's easy to get HUGE IMPROVEMENTS when you choose to compare yourself against something that no one does anyway.

I'll wait to see what their app does when compared to the current methodology of the clients. I'd guess that it would be WORSE than simply measuring the latency and hops. Which is already done and done rather more efficiently than their method of querying 3rd party servers.

Re:And that is their flaw.

Great, except that latency and hops means very little in terms of throughput. As an example, being in Vancouver on Shaw, I'm likely to get better speeds from a node in Toronto on Shaw (quite a few hops away, and relatively latent) than from a Telus user here in Vancouver.

The reason? Shaw owns a national fibre network that crosses the country, and you can traverse that distance without leaving their (impressive) network. In comparision, going to Telus, which is not that far away in terms of hops and latency, requires crossing border routers which, at peak periods, are very likely saturated.

One thing I wish my torrent clients would do is stop accepting uploads from peers with worthless transfer rates. When I have three seeds sending data to me at 120 KB/s on average, and forty sending data at 0.5 KB/s on average (and not downloading at all), those connections are accomplishing pretty much nothing. I'd rather disconnect from them, and try to find other peers with whom I can exchange data faster (in both directions).

Especially on private trackers, where the 'maximum number of peers' I connect to are all downloading from me at 1 kb/s each; this actively harms my ratio, because I have to seed the torrent for weeks to hit 1:1; I'd rather connect to someone else and ship them 100 KB/s so I can get the data out there faster, and not suffer because of people with shitty routes.

That, more than anything, is what I hope for this technology.

internet gps

nice idea...but looks like its piggybacking on Akamai's database for geo/ip mappings. I wonder if Akamai's TOS is friendly to this sort of stuff. In any case, this sort of feature could be built into the BT protocol itself to achieve the same end if necessary.

Re:internet gps using Akamai's servers

Akamai's content distribution system works by putting large numbers of small caching servers around the internet, on ISP networks, and using algorithms to connect clients to the closest server while doing some level of load-balancing. (There are other CDNs that work by putting small numbers of large servers at peering points.) So if two clients get connected to the same Akamai server when they're retrieving some Akamai customer's content, they're probably nearby in a network sense. That doesn't require getting lots of detail from Akamai's network - though it might be more accurate if it did.

It's an interesting approach - you can also do things like identifying IP addresses by BGP Autonomous System Number, which will tell you what sites are in the same ISP, but you might get better P2P performance by connecting to a peer on another ISP in your same city than a peer who's on your ISP but across the country. (Most ISPs seem to assign ASNs on roughly a continent or country level.) So sometimes you'll get better P2P performance by picking close ping times, but as the article says, pinging lots of potential peers can take a long time.

Double Edged Sword

Despite all the legitimate uses for P2P and the associated technologies, there appears to be a rather pervasive view (spin, rather) that all possible uses are nefarious.

As such, this will likely get spun as making the process of copyright infringement more efficient. Will that lead to this being blocked or otherwise pushed back against?

So Hold the handle, not the sharp edges

There are two groups of people who don't like P2P - the RIAA who want to spin it as content thievery (which, ok, it often is), and the ISPs, who don't like getting their networks swamped and having to pay more for transit with upstream ISPs or increasing the size of their peering with peers and their internal distribution links. Right now, both of those forces are pointed in the same direction.

Making P2P more efficient by aligning peer selection with ISP structure makes the ISP side less grouchy about it. This is good. The more precisely you can do that, the more you reduce the impact on the ISP's performance and costs, as well as getting better performance for the P2P system. So they're generally going to like it, though it's obviously a balancing act, because better alignment means you can also find the bottlenecks in your ISP and fill them.

So no, as long as you're not bothering Akamai too much, and as long as this works reasonably well with your ISPs, it's not going to get pushback.

Back when Napster was still around, it did some work with some universities to set up peering student-student rather than student-outsider, because that way most of the bandwidth stayed on the fat cheap university LANs rather than the thinner and rapidly-overloaded links to the Internet. Some of this happened naturally (students would show up as having fast connections, so students would generally upload from other students, but outsiders would also try to upload from students.) Napster could do this fairly easily, because they had a centralized database. Bittorrent and most other P2P systems today are designed to avoid having a centralized database, because it was a target.

Re:Double Edged Sword

Despite all the legitimate uses for handguns|hemp|abortions|porn|foreigners and the associated technologies, there appears to be a rather pervasive view (spin, rather) that all possible uses are nefarious.

As such, this will likely get spun as making the process of violent crime|drug abuse|premarital sex|rape|taking our jobs more efficient. Will that lead to this being blocked or otherwise pushed back against?

Well, that took long enough

That's been the trouble with these "peer to peer" protocols. The routing algorithms have been horribly inefficient. It's quite possible to have the same data flowing in both directions on the same pipe. Multiple copies, even.

It might be cheaper for the telecom industry (which is big) to buy out the music industry (which is tiny) and just cache the RIAA's entire output on local servers. Just cacheing the top 100 releases or so might cut traffic in half.

(This won't scale to movies, though. Movies are bigger and more expensive to make.)

Re:Well, that took long enough

That's been the trouble with these "peer to peer" protocols. The routing algorithms have been horribly inefficient. It's quite possible to have the same data flowing in both directions on the same pipe. Multiple copies, even.

Seems to me that is an artifact of a protocol being designed to operate on a hostile network.

Distribution could be wildly efficient if the users and the network operators were on the "same team." If they wanted to, they could design a bit-torrent variant where chunks are cached by intermediary servers, so that they can always be delivered quickly from a local node. Further, servers could maintain accurate models of network topology, and clients could then use this data to pick the best path. Chunks from popular files would almost always be available from a nearby server cache or a nearby peer.

The problem is that the network is either indifferent to user activities, or actively trying to prevent user activities (throttling, etc.). The end result is that the protocol is tweaked not for efficiency, but for circumvention (e.g. encryption).

I like the idea presented in the summary, since it is in principle a net benefit to both the users and the network operators. However even if it works, it may not last. For instance, ISPs may use even more aggressive tricks (maybe even exploiting this proposed variant), forcing the protocol to become even more inefficient (e.g. switching to a multi-hop TOR-like protocol).

Remote Location Prejudice?

I'm no expert in this field, but this sounds to me like computers in isolated areas would suddenly get the shaft. Am I missing something?

Re:Remote Location Prejudice?

Don't computers in remote locations already get the shaft? If there're no peers within your ttl, then you're sol.

Good Morning Internet

AFAIK there are often ISPs in BFE that can give you a decent ttl. It's just a PITA getting them to honor their TOS so your packets don't go MIA.

"Nearby peer" mechanisms are anticompetitive

One thing that many people do not think about at first (but realize when it's pointed out to them) is that mechanisms which try to identify peers on the same ISP's network are anticompetitive. (That's why only the biggest carriers, like AT&T, support them.) Here's why. The cable and telephone monopolies have so many customers that the odds are there will be someone else on the same provider's network with the requested files. Small ISPs, on the other hand, will rarely if ever have someone with that file and so will still experience a great impact from the cost shifting and congestion caused by P2P. Hence, you can see why the big guys are cautiously embracing schemes like "P4P" as an anticompetitive weapon to block new entrants -- particularly wireless ones.

Re:"Nearby peer" mechanisms are anticompetitive

OK... but the blame lies not on the "big telcos", but reality itself. Network effects exist; better to harness them than kvetch about them. What are the big networks supposed to do, pretend they don't exist and screw their customers in the process?

Two seperate issues between ISPs and P2Ps

There are two seperate issues between the ISPs and the P2Ps. The details of the two issues tend to get mixed according to the perspective of the person making the argument.

The first issue is the amount of data (the bandwidth issue) that the P2P downloader is using relative to the amount of bandwidth that the other ISP users are consuming. The other issue is the ability of the so-called owners the downloaded information to legally extort money from P2P users.

The P2P users are the best customers of the ISPs. In time, the technology improves to handle the growing needs of the P2P community, and the P2P'ers are willing to pay (within reason) for faster access and greater bandwidth. P2P'ers will pay $30-$50 more a month to the ISPs than the dial-up'ers who are mostly checking e-mail, reading specialized websites, and doing eBay trading. This makes the P2P'ers a significant revenue source to the ISPs.

"Significant revenue source", in case you didn't know, is the most important three word phrase in the English language. "You're Under Arrest" is the second-most significant phrase in English. And, of course, the more 'sig rev source' that you have, the less you have to concern yourself with hearing "You're U A!" But, nevertheless, it can still happen. Especially in the current times of great change such as the present when one former source of sig revenue (the music industry) is evaporating and others like the P2P community are rising.

Generally the law follows the money. The golden rule states that he who hath the gold maketh the rule. But, in the real world, money and law tend to be 90 degrees out of phase. Situations arise where a disappearing revenue source has, for a certain period of time, the ability to envoke the legal system to extort money from people in greater proportion than its social usefullness would have it deserve. The music industry, and its extortion arm - the RIAA, is in that position. This industry is entering its 'zombie' phase, in that it is already dead but doesn't seem to know it. Death for a business is a different concept than it is in biology. Zombie businesses are basically unsustainable in the long run because their economic model has been broken, but their structures are still functioning. Basically the RIAA is just the music industry running around like a chicken with its head cut off. It can't last, but you don't want to be in its way before it just falls over.

Since the RIAA uses the ISPs to identify the P2P'ers that it has selected for random extortion, the P2P'ers don't trust the ISPs to come up with a working technical solution to the bandwidth problem. So we have the current situation that is bad for everyone. Personally I work around this by not downloading industry product: I get it in disc format from the local library and copy it from the disc onto my home PC. Then I return the disc to the library for the next person to use.

The music industry insists that this is illegal in their parallel universe. And, there was a time when it appeared that the RIAA was going to take on the US Library Association. But the librarians have been dealing with assholes like this for 300 years and have their arguments in order. It always come down to this point: yes, library users copy the most popular music recordings. Which does cut sales to a minor degree. But the 50,000 libraries buy (at full retail cost) one copy each of thousands of titles that wouldn't be selling 50,000 copies if the libraries weren't buying it. Basically, the library makes available music for people to copy. But the libraries pay off the music industry to ignore it. Everybody is happy.

The P2P'ers need to adopt this model for distribution. They should find out who they are in their local areas, like a university, and then trade physical copies of the materials that they are interested in. Like having ALL the recent music of particular genre or favorite films on a single USB 500Gi

Re:Two seperate issues between ISPs and P2Ps

The P2P users are the best customers of the ISPs. In time, the technology improves to handle the growing needs of the P2P community, and the P2P'ers are willing to pay (within reason) for faster access and greater bandwidth. P2P'ers will pay $30-$50 more a month to the ISPs than the dial-up'ers who are mostly checking e-mail, reading specialized websites, and doing eBay trading. This makes the P2P'ers a significant revenue source to the ISPs.

All this is wrong.
The best customers of the ISPs are "dial-up'ers who are mostly checking e-mail, reading specialized websites, and doing eBay trading" AND "pay $30-$50 more a month to the ISPs".

ISPs hate the traditional bandwidth hog and now they're starting to hate their traditional customers too, because those "dial-up'ers" on broadband are also moving towards bandwidth heavy internet habits.

Re:Paranoia

Sorry Mr AC, but in the US downloading MP3s is legal. Distributin copyrighted works without the copyright holder's permission isn't legal, but downloading anything except child pornography is legal.

The FBI may or may not come after you for uploading, but they will NOT come after you for downloading.

Re:

But since this was about torrents you will indeed upload parts of or the whole work yourself aswell.

Re:Hot Potato for ISPs

ISPs don't actually care about copyright infringement, except possibly the cable modem companies which are also selling television and might have their advertising revenues impacted. Back when Napster and @Home were still around, @Home had two positions on Napster - officially, they'd say "Evil Copyright Infringers are Bad! And people generating upstream bandwidth from home are Bad!". Unofficially, the people who worked there mostly said "Well, duh! The reason people are buying broadband at home is to download music - Napster's really great for us!"

ISPs care about money - buying more upstream costs money, and upgrading peering links or internal distribution networks costs money. They also care about customer perceived performance, and if P2P uses their networks inefficiently, and swamps a neighborhood's upstream in ways that interfere with TCP performance, that's bad. For the most part, this technology will reduce their costs by reducing exterior bandwidth, and that's good, as long as it doesn't do it in ways that the improved P2P performance finds other bottlenecks in their system to step on. The better the P2P paths can match the structure of the ISP, the lower the impact on their network will be.

This approach doesn't actually require the ISP to install anything, or to do anything, or expose them to participating-in-P2P-themselves infringement conflicts; there are other approaches that do, such as putting P2P caching servers in their network. So it's pretty much all gravy for them, especially since they know that some large fraction of the bits they're carrying are P2P. (The Akamai caching servers here aren't being used to cache the P2P - they're web caches used by traditional content providers, and what this tool is doing is using their location to identify some of the structure of the ISP network to do better P2P peer matching.)