Evolving Swarms with Swarmstreaming

Orasis writes "Applications like Bittorrent have broadly validated swarming technology in the real-world. Now, the inventor of swarming has released a new technology called swarmstreaming that allows smooth progressive playback of content, skipping ahead, and random access without downloading the entire file. It's an HTTP proxy, so browsers, podcasting, and RSS apps should be able to use it transparently. "

Something like this

[Anonymous Coward]

is going to bring the internet to a grinding halt someday.

Re:How does this work?

[ravenspear]

"Skipping ahead" Will skip to a part of the clip that you may not have. This=lag

The technology to eliminate lag already exists and has been implemented. I have used it myself.

What's more, usually you cannot download one second of movie in one second of time, unless you have a crazy tricked out connection.

What nonsense. Have you ever downloaded a trailer from here [apple.com]? If the trailer starts to play immediately when you start downloading (i.e. the gray progress bar proceeds faster than the location marker), then you are downloading 1 second of movie in a time faster than 1 second. I can assure you that millions of people have a connection fast enough to do this.

This means that if you skip to a part you haven't seen yet, you will have to wait even longer for buffering.

Again, not necessarily. Buffering is when the streaming software requires that you download x amount of content ahead of the time you actually view it to account for inconsistencies in the stream or packet loss. If those can be eliminated, and connections made fast enough, there is no empirical reason why buffering must continue to be utilized.

Re:No more slashdotting?

[Pxtl]

No. Heaven forbid anyone ever try and improve HTTP - that's blasphemy, even though it seems to be the source of most of the weakness of the internet (can't home serve due to lack of NAT traversal, can't serving large files is suicide because of lack of swarming, no way to differentially update content or inform the user of updates so you have users mashing "refresh" over and over again and redownloading the same goddamn html).

No, http is perfect, that's why nobody ever ever bothers to change it.

But I'm not bitter.

Re:Lets not forget the price of entry. A BARGAIN!

[Sai Babu]

If you have the use for it.

Consider internet radio or TeeVee

Streaming the same packets to each IP wanting them gets to be a real mess real fast. The beauty of this system is that so long as a recipients have adequate upload bandwidth to accomodate the stream bandwidth plus some delta (bigger delta will mean lower latency as parallism will increase with fewer stes away from the source) than the 'broadcaster' only needs enough bandwidth to get the stream out to a few people in order to each millions and millions. Don't forget, radio and TeeVee delays of a few seconds or minutes are easily tolerable when the alternative is no program at all.

Imagine a 'fee free' version of this. Anyone could reach as many people as clear channel radio for the expenst of a megabit or two of outbound bandwidth!

If I had programming to deliver and felt it would interest a few hundred thousand people, the on etime $25K would be a drop in the bucket considering what I'd have to pay to reach these same people by traditional radio, TV, or buying enough bandwisth for 100k streams.

Think about it in radio terms. If I'm running a 128kbps MP3 stream and 100k people want it, I need 12.8gbps and the hardware to stuff it. Hell, a 45mpbs and $25K, one time, is a BARGAIN!!!

I predict we will see some serious challanges to big media corps from this and it won't take long. Just watch how fast the PORN guys snap this stuff up!.

Also, imagine running live feeds from public events. A laptop, this application, a WiFi connection, and there's no limit to the number of people who can join in to 'attend' the event.

Re:more linkage

[Anonymous Coward]

I seriously doubt that he is the very first one to have thought of swarming. Swarming has been around since before 1999 (when he claims he invented it). He *may* be the first one to have applied it to p2p/networking however.

Not even that. BitTorrent is based on the old Mojo Nation system (another early mutli-source downloading system) and I know that the MN guys gave a demo to the bay area cypherpunks back in late 1999 or early 2000. Since they probably did not re-write the system overnight to create a "swarming" app I am guessing that it has been around the p2p world for a bit longer than this guy claims...

Re:I dunno...

[Orasis]

Its not just for media. Imagine a 10GB CAD file that you want to grab a 10MB object out of. You could grab just the bytes you need and skip the rest.

Re:not free

[dbacher]

It's targetted at a different market than BitTorrent.

Imagine for example that you are a company distributing a maintenence release of a 40m application.

You seed this on a web server on your US east server, and you have the "swarm" running on US west, EU, Asia Pacific, etc.

Users connect to the proxy, but the proxy can use bandwidth from all of those sites. Assuming most users upgrade during the day, you're probably paying for a lot of bandwidth you're not using, that you could use to distribute the content.

That, I believe, is the target market from reading this. Think about Microsoft, with hundreds of network centers, most of which are empty at any given time. They would need a lot less combined bandwidth if they could distribute a service pack this way.

The issue with Torrent is that Torrent requires client software. This system runs in a proxy on the web server itself, so there is no client side isntallation required.

Prevention of Downloading Bogus Files!

[dos4who]

This looks like it could be the next big thing in preventing the download of large bogus files.

Currently, in p2p programs (ala Kazaa, etc), you'd have to download the entire 600 MB file "Lord of the Rings.avi" (or "Busty Nurses. avi".. depending on your cinematic preferences), only to realize that someone has posted a bogus video in it's place.

Swarming the file (ie: "Lord of the Rings.avi"), would allow you to preview various portions of the file to ensure it's integrity... (personal integrity aside) before downloading the entire file to your local PC.

This is going to really pi$$ off the MPAA

Re:In other news...

[talaphid]

There should be an active movement to get the next revolutionary protocol (or this one, if possible) renamed zerg, so it's "file zerging". Blizzard should be on board with this, because they've taken from the Bittorrent thoughtshare, so returning a word as thanks shouldn't be a big deal... and then, years from now, on the news, we'll actually hear, "Today two teenagers were arrested at Minnesota Heights High school for file zerging..." and then my dream of a real life Kent Brockmanism will come true... nouning a verbed noun, and using the word zerg.

Re:iceswarm?

[Orasis]

This is exactly what Brandon Wiley's Alluvium [sourceforge.net] is doing, except that it uses Oggs instead of MP3. Its basically an implementation of the whole "Judo Radio" concept where you download and cache the files ahead of time and just receive a playlist that tells you the order the files should be streamed.

Re:Something like this

[ScrewMaster]

Well ... we had a national telephone system for almost a century: it was called "Ma Bell" and was composed of AT&T and 13-odd Regional Bell Operating Companies. It was an artificial monopoly instituted by the Federal Government for the express purpose of providing reasonably-priced phone service to all regardless of location. And it worked. But about twenty five years ago it was deemed that such a monopoly was no longer in the public interest, in that the public was not being given access to the latest technology had to offer. Monopolies are more interested in the status-quo than in offering the best service. So, old AT&T was broken up and the entire phone system was privatized. Whether that was ultimately a good thing (or not) is not an easy question to answer.

Ironically, the reason that a private sector organization like AT&T was created to implement and maintain the phone system was because the Federal Government recognized that it was ill-equipped to perform the same service in an efficient manner. That hasn't changed, I'm afraid, and I'd hate to see what a government owned-and-operated Internet would be like. It's easy to say "owned by everyone" but the reality is that the government is hideously inefficient in everything it does, and the very last thing we want is to have a nationalized communications infrastructure. Better to have a heavily regulated communications industry (as AT&T once was) with high quality-of-service standards (like AT&T used to have.)