Become a fan of Slashdot on Facebook

 


Forgot your password?
Close
typodupeerror
×
The Internet Technology

Evolving Swarms with Swarmstreaming 246

Posted by CmdrTaco from the the-next-logical-step dept.
Orasis writes "Applications like Bittorrent have broadly validated swarming technology in the real-world. Now, the inventor of swarming has released a new technology called swarmstreaming that allows smooth progressive playback of content, skipping ahead, and random access without downloading the entire file. It's an HTTP proxy, so browsers, podcasting, and RSS apps should be able to use it transparently. "
This discussion has been archived. No new comments can be posted.

Evolving Swarms with Swarmstreaming More

Evolving Swarms with Swarmstreaming

Comments Filter:

  • In other news... (Score:5, Funny)

    by k4_pacific ( 736911 ) <k4_pacific.yahoo@com> on Monday December 13, 2004 @05:39PM (#11076031) Homepage Journal
    In other news, the inventor of swarming was attacked by killer bees.

  • Well, the linked site isn't using it... (Score:5, Funny)

    by datastalker ( 775227 ) on Monday December 13, 2004 @05:40PM (#11076044) Homepage
    ...since it's Slashdotted after one comment. :(

    Google Cache [216.239.57.104]

  • No more slashdotting? (Score:2, Funny)

    by X0563511 ( 793323 ) on Monday December 13, 2004 @05:40PM (#11076045) Homepage Journal
    So, if this works 'transparently' to browsers, ect, does this mean slashdotting a site will be much harder?

    • Re:No more slashdotting? (Score:2, Interesting)

      by Pxtl ( 151020 ) on Monday December 13, 2004 @06:14PM (#11076385) Homepage
      No. Heaven forbid anyone ever try and improve HTTP - that's blasphemy, even though it seems to be the source of most of the weakness of the internet (can't home serve due to lack of NAT traversal, can't serving large files is suicide because of lack of swarming, no way to differentially update content or inform the user of updates so you have users mashing "refresh" over and over again and redownloading the same goddamn html).

      No, http is perfect, that's why nobody ever ever bothers to change it.

      But I'm not bitter.

      • Re:No more slashdotting? (Score:2)

        by ColaMan ( 37550 ) on Monday December 13, 2004 @07:35PM (#11077090) Journal
        What the?

        can't home serve due to lack of NAT traversal
        Isn't that a transport layer issue?

        can't serving large files is suicide because of lack of swarming
        Isn't that a Phat Pipe issue?

        no way to differentially update content or inform the user of updates
        Heard of HTTP code 304 - Not Modified? Servers and browsers already know how to use it, and they do it transparently to the user. And maybe you should consider putting a "last modified" timestamp on your web page, to help stop clueless/hopeful people from overriding their browser and downloading something they arleady have.

        HTTP has it's faults, but none of the problems you mention can be seriously blamed on HTTP.

  • Inventor? (Score:4, Funny)

    by Anonymous Coward on Monday December 13, 2004 @05:40PM (#11076048)
    Now, the inventor of swarming has released a new technology

    Uh, so the killer bees are inventing technology now, and nobody is alarmed? I, for one, welcome our new technology-wielding killer bee overlords.

  • Uhoh (Score:3, Funny)

    by Anonymous Coward on Monday December 13, 2004 @05:41PM (#11076051)
    This [unl.edu] is what his server looks like...

  • It's been done before. (Score:2)

    by sakusha ( 441986 ) on Monday December 13, 2004 @05:41PM (#11076052)
    I can't quite tell what this app is all about, this is surely a record, ZERO comments and the site's already slashdotted.

    But all those features mentioned in the ./ summary are already available in QuickTime Streaming Server.

  • How is this new? (Score:2, Informative)

    by ravenspear ( 756059 ) on Monday December 13, 2004 @05:41PM (#11076056)
    smooth progressive playback of content, skipping ahead, and random access without downloading the entire file

    Quicktime has had all that for several years. Apple called it "Instant On". I think both Real and Microsoft already use something similar.

    • Re:How is this new? (Score:2)

      by kakos ( 610660 ) on Monday December 13, 2004 @05:59PM (#11076247)
      Jesus, man. Okay... so you didn't read the article because it is /.ed, but you can at least read the summary! It's streaming using swarming and it provides smooth progressive playback of content. Streaming itself is very old. But streaming while swarming is news.

    • Re:How is this new? (Score:2, Informative)

      by Agret ( 752467 ) <alias DOT zero2097 AT gmail DOT com> on Monday December 13, 2004 @06:01PM (#11076278) Homepage Journal
      Quicktimes "Instant On" doesn't let you skip to anywhere you want in the file until it has actually got to that part. It downloads it progressively.

      • Re:How is this new? (Score:5, Informative)

        by ravenspear ( 756059 ) on Monday December 13, 2004 @06:08PM (#11076333)
        That is incorrect. Instant On has to do with streaming, not progressive downloading. The two are different.

        Progressive downloading is where you download something like http://www.whatever.com/movie.mov in a web browser and it starts to play as soon as part of it is downloaded. You can then skip to wherever you want once you have downloaded that part (because at that point all you are doing is scrubbing through a movie file stored on your local machine.)

        Streaming is where you load something like rtsp://stream.whatever.com/something.mov into a video player and it streams it to you. At no time during that process is anything stored on your local mahcine aside from what you are currently viewing and whatever the client has buffered ahead of that. Instant On instructs the server to skip the stream to another section.

  • So... (Score:4, Funny)

    by Mindwarp ( 15738 ) on Monday December 13, 2004 @05:43PM (#11076086) Homepage Journal
    ...will this become SwarmPorning or PornStreaming do you think?

  • How does this work? (Score:3, Insightful)

    by koreaman ( 835838 ) <uman@umanwizard.com> on Monday December 13, 2004 @05:43PM (#11076088)
    No amount of swarming will ever get around the fact that a piece of something has to be in your local system before you can view it. "Skipping ahead" Will skip to a part of the clip that you may not have. This=lag. What's more, usually you cannot download one second of movie in one second of time, unless you have a crazy tricked out connection. This means that if you skip to a part you haven't seen yet, you will have to wait even longer for buffering. This is hardly worth it.

    • Re:How does this work? (Score:2)

      by OblongPlatypus ( 233746 ) on Monday December 13, 2004 @05:53PM (#11076201)
      What exactly is your definition of "crazy tricked out"? You can easily stream movies in real-time on a common megabit ADSL connection.

      (Most movie releases these days are Xvid-encoded to fit on a single CD. Assume 93 minutes of movie and 700 megabyte file size, and it works out to just about exactly 1Mbps.)

      • Re:How does this work? (Score:2)

        by Pulzar ( 81031 ) on Monday December 13, 2004 @06:08PM (#11076325)
        Most movie releases these days are Xvid-encoded to fit on a single CD. Assume 93 minutes of movie and 700 megabyte file size, and it works out to just about exactly 1Mbps.

        Do you often achieve the maximum download rate, and then sustain it for an hour and a half? If you do, you have a much better connection than most average ADSL users.

        Also, if we're talking about swarming, it'll be even tougher to sustain the max download rate while upload clients are dropping in and out of the swarm.

        • Re:How does this work? (Score:2)

          by OblongPlatypus ( 233746 ) on Monday December 13, 2004 @06:38PM (#11076627)
          Yes, I regularly achieve the practical max of my connection, and sustaining it is never a problem. I do realize there are wildly varying levels of quality in the ADSL offerings out there though.

          As for swarming making it tougher, have you ever actually used BitTorrent? I have never seen any sort of evidence of that. Although of course that would depend entirely on the quality and number of seeders in your particular torrent, so I guess once again there are people less lucky than myself.

          But what I'm trying to say is that "swarmstreaming" movies in real time is perfectly technically possible for me, and I'm pretty average with no "crazy tricked out" connection. The only thing that's stopping me from doing it right now is that the defacto standard is BitTorrent, which requires you to wait for the entire download to finish before watching.

          • Re:How does this work? (Score:2)

            by Pulzar ( 81031 ) on Monday December 13, 2004 @06:58PM (#11076792)
            As for swarming making it tougher, have you ever actually used BitTorrent? I have never seen any sort of evidence of that.

            I pretty much have something going through torrent at all times. I see wildly varying download rates during a download of a same item. It can go anywhere from 100KB/s to 30KB/s and back up in one hour. And, this is with very active torrents with 100+ clients.

            The inactive torrents are actually a lot more stable, because there are only a few seeds and they are not being leetched to death.

            I find this to be the norm in the circles that I go in, but I understand that the experiences vary with regions and ISPs. I can't say what the overall "world" average is.

      • Re:How does this work? (Score:2)

        by EvilAlien ( 133134 ) on Monday December 13, 2004 @08:05PM (#11077298) Journal
        "crazy tricked out" = modem with giant spoiler, large R decal, and neon lights on the bottom. Spinner RJ45 plugs are optional for crazy tricked out bling bling connections.

    • Re:How does this work? (Score:5, Interesting)

      by ravenspear ( 756059 ) on Monday December 13, 2004 @05:54PM (#11076210)
      "Skipping ahead" Will skip to a part of the clip that you may not have. This=lag

      The technology to eliminate lag already exists and has been implemented. I have used it myself.

      What's more, usually you cannot download one second of movie in one second of time, unless you have a crazy tricked out connection.

      What nonsense. Have you ever downloaded a trailer from here [apple.com]? If the trailer starts to play immediately when you start downloading (i.e. the gray progress bar proceeds faster than the location marker), then you are downloading 1 second of movie in a time faster than 1 second. I can assure you that millions of people have a connection fast enough to do this.

      This means that if you skip to a part you haven't seen yet, you will have to wait even longer for buffering.

      Again, not necessarily. Buffering is when the streaming software requires that you download x amount of content ahead of the time you actually view it to account for inconsistencies in the stream or packet loss. If those can be eliminated, and connections made fast enough, there is no empirical reason why buffering must continue to be utilized.

      • Re:How does this work? (Score:3, Insightful)

        by Neophytus ( 642863 ) * on Monday December 13, 2004 @06:03PM (#11076292)
        But millions of people don't have large enough upload capacity to support millions of other people streaming at that speed. With many domestic broadbands the ratio can be as bad as 12:1.

        And that's before you even deal with people needing to set up port forwarding.

      • Re:How does this work? (Score:2)

        by Pulzar ( 81031 ) on Monday December 13, 2004 @06:04PM (#11076297)
        The technology to eliminate lag already exists and has been implemented. I have used it myself.

        Can you provide any details? I don't understand how you can see something without a lag if you can't download it in real time?

        What nonsense. Have you ever downloaded a trailer from here?

        Those trailers are reduced to very small resolutions, so that you can watch them right away. They are hardly good enough to replace regular TV or way off from DVD or HD content.

        • Re:How does this work? (Score:2)

          by ravenspear ( 756059 ) on Monday December 13, 2004 @06:14PM (#11076387)
          I don't understand how you can see something without a lag if you can't download it in real time?

          You can. That was the point I was trying to make. There are plenty of connections out there fast enough to download video in real time, provided the data rate is reasonable.

          They are hardly good enough to replace regular TV or way off from DVD or HD content.

          Well that's obvious, but I wasn't referring to DVD or HD content. Streams at those data rates are basically nonexistent on the net today because very few connections can utilize them. I was referring to the more standard video formats that are used for streaming today, some of which can yield very good quality at a data rate far below what DVD or HD uses.

        • Re:How does this work? (Score:3, Informative)

          by ctr2sprt ( 574731 ) on Monday December 13, 2004 @11:01PM (#11078496)
          There are other codecs than MPEG2. XVID, for example, can do DVD quality at about half the bitrate. If you knock down the resolution a notch, like from 720xwhatever to 640xwhatever, you can cut the bitrate even further. 1.5Mb/s XVID (including audio) is very, very good on a computer monitor at that resolution. Since I get 3Mb/s on my cable, we could even stand to bump up the resolution to, say, 800x600 (which is what most HDTVs actually are).

          You'd still probably want your nice expensive HDTV for stuff where quality really matters, but as far as delivering high-quality video over the Internet the capacity is definitely there. Well, it's there on the user side: if a million people suddenly started downloading 2.6GB files all at once I can imagine a few of the server's routers running shrieking from the data center while on fire.

      • Re:How does this work? (Score:2)

        by coldmist ( 154493 ) on Monday December 13, 2004 @06:17PM (#11076413) Homepage
        The current (before today) tech could do it from one single fat source, ie one of apples server for example, with a scaled down video source.

        This can be getting the bits from 100 different sources, achieving the same effect, for potentially larger (height/width, such as HD content) video clips.

        Now, how does this "already exist"?

        • Re:How does this work? (Score:2)

          by ravenspear ( 756059 ) on Monday December 13, 2004 @06:38PM (#11076636)
          Now, how does this "already exist"?

          The capability to eliminate lag exists. I didn't say it can currently be applied to all forms of serving video or all data rates. Swarming might help there.

          The grandparent post seemed to be taking the position that buffering would be a perpetual necessity, i.e. that as long as there was streaming, there would be lag. This is not necessarily the case. It is true that currently there is no easy way to avoid it for very large video sizes and data rates. However that is just a limitation of connection speeds and processing power.

      • Re:How does this work? (Score:2)

        by chris_mahan ( 256577 ) <chris.mahan@gmail.com> on Monday December 13, 2004 @06:26PM (#11076508) Homepage
        from apple I can do the small and medium like you describe. Large view area doesn't download fast enough. I think full-screen is out of the question. Time Warner residential cable ISP in west san-fernando valley here.

        Is there a divx player for firefox as a plugin? (please say yes)

    • Re:How does this work? (Score:2)

      by joshv ( 13017 ) on Monday December 13, 2004 @07:00PM (#11076807)
      What's more, usually you cannot download one second of movie in one second of time, unless you have a crazy tricked out connection.

      1.5 Mbps DSL here. Downloading compressed movies from Starz online movie service I regular have download times that are 2/3rds to 1/2 the playing time of the movie. Granted, these are heavily compressed, but they are better than VHS quality, and I would not call my connection "tricked out"

      -josh

    • buffering (Score:2)

      by ad0gg ( 594412 ) on Monday December 13, 2004 @07:29PM (#11077040)
      Stop using Real player.

  • I dunno... (Score:2)

    by rackhamh ( 217889 ) on Monday December 13, 2004 @05:44PM (#11076097)
    It seems to me that the types of media that swarming is commonly used for won't benefit much from being able to skip forward.

    I mean, if you're downloading a feature film or TV show, do you really want to watch the middle before the beginning?

    • Re:I dunno... (Score:3, Insightful)

      by dJOEK ( 66178 ) on Monday December 13, 2004 @05:45PM (#11076119)
      no, you wanna start seeing it, then people bother you, then later when you have more free time you can watch the rest

    • Re:I dunno... (Score:3, Interesting)

      by Orasis ( 23315 ) on Monday December 13, 2004 @06:50PM (#11076735)
      Its not just for media. Imagine a 10GB CAD file that you want to grab a 10MB object out of. You could grab just the bytes you need and skip the rest.

    • But BT doesn't even allow you to watch first (Score:3, Informative)

      by cryptochrome ( 303529 ) on Monday December 13, 2004 @07:25PM (#11077013) Journal
      No, you want to watch from the beginning. However, BitTorrent is not designed for this - it specifically targets the worst represented portions of the file to help make the swarm as diverse as possible, so that as seeds disappear major bottlenecks do not arise. There are implementations like Azureus that can favor the first chunks, but the result is sketchy at best. Specifically delivering a desired chunk has merit anyway - but only for streamable content, and only if it can deliver fast enough for playback.

      Also, BT's speed is often bottlenecked by ISPs which cap uploads, therefore penalizing you on the downloading side.

      It's right there on the website though - due to the nature of the technology you have to update your playback/reader/displayer app to take advantage of this. BT just works as is. If the BT developers really wanted to they could update the protocol to allow all this, or even more interesting features - like parchive file repair and recovery [slashdot.org], which would effectively eliminate the problem of swarms with lost seeds.

    • Re:I dunno... (Score:3, Insightful)

      by AtomicBomb ( 173897 ) on Tuesday December 14, 2004 @12:03AM (#11078752) Homepage
      Not really. Skip forward makes sense in many cases.

      I am a regular listener of the RTHK radio archive (a Hong Kong government funded radio station). The subscription is free. The audio clip contains the news, government ad (like don't throw rubblish blahblah) and the radio show itself... Not one bothers to cut the junk out. In a typical 2-hr session, 20-25 mins are news and other junk... It is just odd and a waste of time sit still and listen to "news" several months or even years ago...

  • How to get it (Score:5, Funny)

    by Kipsaysso ( 828105 ) on Monday December 13, 2004 @05:44PM (#11076105) Homepage Journal
    Does anyone have a Torrent link to download it?

  • Were we just beta testers? (Score:5, Funny)

    by DeckardJK ( 555299 ) on Monday December 13, 2004 @05:44PM (#11076106)
    If so... go back to the drawing board Justin...

  • i wonder how long (Score:2, Insightful)

    by bill11082 ( 458037 ) on Monday December 13, 2004 @05:45PM (#11076118)
    wonder how long until the RIAA/MPAA uses the DMCA to declare this technology illegal

  • We have a new record! (Score:2, Funny)

    by xv4n ( 639231 ) on Monday December 13, 2004 @05:46PM (#11076138)
    16 mentions to the word "swarm" and it's derivatives in 4 paragraphs!

  • Deeper Link (Score:2, Informative)

    by ethzer0 ( 603146 ) on Monday December 13, 2004 @05:49PM (#11076168)
    Here's a link to Onion Networks' Swarmstream product. It appears that to use the product, you need to purchase a Swamstream SDK, though. [onionnetworks.com]

  • Server slowing down.. article text: (Score:4, Informative)

    by Folmer ( 827037 ) * on Monday December 13, 2004 @05:50PM (#11076174)
    Swarmstreaming: Swarming Downloads Evolved
    I'm proud to finally unveil swarmstreaming our third generation of swarming algorithms that are designed for the fastest downloads of web content and multimedia without any special server software or silly .swarm files. This is probably our most exciting advancement since the original invention of swarming.

    The technology improves swarming by ensuring that the bytes that the user wants next are scheduled to be received next. So if they're playing back a video file, the bytes from the front of the file will be received first. If the user (or application) skips forward to the middle of the file, the bytes at the middle of the file will be prioritized. Thus, unlike first generation swarming systems like Swarmcast or Bittorrent, you don't have to wait for the entire file to download to do something useful with it!.

    Under the covers it is almost unimaginably more complicated than this because it also provides Self-Healing Downloads, implements a full-blown, scalable, Web Proxy Cache, and actively works to ensure that the video playback never studders or buffers by constantly monitoring and adapting to changing network conditions. For a raw feature dump, check out the SwarmStream SDK Feature Matrix

    Nowadays, because of its immense popularity, most people have only heard of swarming because of Bittorrent. I have no animosity towards Bittorrent because it has done more than any application to prove the value of swarming to the general public. But if people are impressed by Bittorrent, they're going to be absolutely blow away by swarmstreaming and how far we've taken swarming since its humble beginnings five years ago.

    The best source of information right now on swarmstreaming is Onion Networks SwarmStream SDK, so check it out and let me know what you think.

    He links to http://onionnetworks.com/technology/swarming/#swar mstreaming [onionnetworks.com]

  • Crichton (Score:5, Funny)

    by bwy ( 726112 ) on Monday December 13, 2004 @05:59PM (#11076249)
    Whenever I hear swarm in an IT context, I can't help but think about Crichton's Prey.

  • Lets not forget the price of entry. (Score:5, Informative)

    by Bruha ( 412869 ) on Monday December 13, 2004 @06:03PM (#11076294) Homepage Journal
    SwarmStream Development Suite Features

    * Object code for the entire suite of SwarmStream(TM) APIs, including WebRAID(TM), DirectCache(TM), Throttling, and THEX.

    * Visualization tools to perform live inspections and demonstrations of what SwarmStream is doing during your application run time.

    * One full license for WAN Transport(TM) Server (normally $2950), an HTTP server specifically designed provide advanced SwarmStream features such as self-healing downloads and automatic mirror discovery.

    * One full day of developer training

    * 20 hours of ongoing support

    * One year of free upgrades for all of the above software.

    * Unlimited right to use and implement SwarmStream technology for testing, prototyping, demonstrations, or creation of reference designs or applications. Production deployment requires an additional Deployment License.

    * One-time fee: $25,000

    • Swarmcast is Free (Score:2, Informative)

      by Anonymous Coward on Monday December 13, 2004 @06:26PM (#11076506)
      The application itself is free for everyone to use, even as a proxy. The license above is if you want to develop an application that includes SwarmStream.

    • Re:Lets not forget the price of entry. (Score:2)

      by Lord_Dweomer ( 648696 ) on Monday December 13, 2004 @06:33PM (#11076571) Homepage
      Yeah, and then lets remember the fact that someway, somehow, an open source solution will arise, and in the mean time, pirates will have this a week before its released.

      • Re:Lets not forget the price of entry. (Score:2, Informative)

        by Anonymous Coward on Monday December 13, 2004 @07:13PM (#11076891)
        Yeah, and then lets remember the fact that someway, somehow, an open source solution will arise
        It already has. Dijjer [dijjer.org] is open source, and was developed by the creator of Freenet [freenetproject.org]. It is still alpha, but is developing rapidly.

        These guys must be pretty pissed that someone got slashdotted weeks before they did with some software that is entirely free, and does at least as much as what they claim their non-free software does.

    • Re:Lets not forget the price of entry. A BARGAIN! (Score:2, Interesting)

      by Sai Babu ( 827212 ) on Monday December 13, 2004 @06:36PM (#11076606) Homepage
      If you have the use for it.

      Consider internet radio or TeeVee

      Streaming the same packets to each IP wanting them gets to be a real mess real fast. The beauty of this system is that so long as a recipients have adequate upload bandwidth to accomodate the stream bandwidth plus some delta (bigger delta will mean lower latency as parallism will increase with fewer stes away from the source) than the 'broadcaster' only needs enough bandwidth to get the stream out to a few people in order to each millions and millions. Don't forget, radio and TeeVee delays of a few seconds or minutes are easily tolerable when the alternative is no program at all.

      Imagine a 'fee free' version of this. Anyone could reach as many people as clear channel radio for the expenst of a megabit or two of outbound bandwidth!

      If I had programming to deliver and felt it would interest a few hundred thousand people, the on etime $25K would be a drop in the bucket considering what I'd have to pay to reach these same people by traditional radio, TV, or buying enough bandwisth for 100k streams.

      Think about it in radio terms. If I'm running a 128kbps MP3 stream and 100k people want it, I need 12.8gbps and the hardware to stuff it. Hell, a 45mpbs and $25K, one time, is a BARGAIN!!!

      I predict we will see some serious challanges to big media corps from this and it won't take long. Just watch how fast the PORN guys snap this stuff up!.

      Also, imagine running live feeds from public events. A laptop, this application, a WiFi connection, and there's no limit to the number of people who can join in to 'attend' the event.

  • Orasis? (Score:4, Informative)

    by EVuL_C ( 80873 ) on Monday December 13, 2004 @06:05PM (#11076308)
    Anyone care that Orasis (the story author) = Justin Chapweske?

    http://www.advogato.org/person/orasis/ [advogato.org]

    -c

  • more linkage (Score:2, Informative)

    by Glog ( 303500 ) on Monday December 13, 2004 @06:08PM (#11076335)
    Here is some info on the new technology from the guy's company's website: http://onionnetworks.com/products/swarmstream/ [onionnetworks.com]

    On a sidenote, I seriously doubt that he is the very first one to have thought of swarming. Swarming has been around since before 1999 (when he claims he invented it). He *may* be the first one to have applied it to p2p/networking however.

  • Swarming + streaming (Score:5, Insightful)

    by Anonymous Coward on Monday December 13, 2004 @06:09PM (#11076346)
    When you download something via BitTorrent, it's downloaded in random order, as pieces become available. While this works, it means you've got a huge file on your hard disk, but it's completely useless because random pieces are utter garbage bytes. For example, unlike with a straight download, you can't start watching a video file that's still being saved to disk.

    The only thing swarmstreaming changes is that it tries to download data in order, so you can use it more quickly, like any other conventional stream-oriented protocol (which is basically anything that uses TCP, along with various streaming media protocols). Now, the innovation is putting together streaming media with the power of swarming--imagine being able to feed a live TV feed from a single stream from the "seed". This is basically what multicast promised, but due to infrastructure problems, has yet to deliver.

    Now, the devil is in the details. You're going to have problems with a distributed application that tries to deliver the same data to all nodes in the network at once, since you don't get all those nice properties of randomized distribution of different pieces. Some lossiness would definitely be desirable, meaning you don't really want to use it like a Web proxy. Furthermore, it's physically impossible to deliver data around the planet without many tens or hundreds of milliseconds of latency, so it's not good for interactive applications.

    It might be a big win for TV-on-the-Web, though. Imagine if just anyone with a couple hundred kbps could serve a worldwide audience... all those Internet radio stations that are begging for donations to pay bandwidth costs could slash their total bandwidth needs, while upgrading service as well.

    I'm not sure if this particular product is going to do the trick (swarmstreaming isn't a new idea, after all, and lots of people have been working on it), but anything that gets people thinking about it should help in the long run.

    • Re:Swarming + streaming (Score:3, Informative)

      by Orasis ( 23315 ) on Monday December 13, 2004 @06:45PM (#11076690)
      Yup, this guy hit the nail on the head. We've actually been doing swarming+progressive downloads since 2001. The big change with swarmstreaming is that this stuff scales to huge numbers of files in a single cache and supports out-of-order random access, which turns out to be much harder than progressive playback is. So if you have an application that only needs to read small ranges of bytes in a huge file, you can now use swarmstreaming to do the trick.

    • Re:Swarming + streaming (Score:2)

      by hackstraw ( 262471 ) * on Monday December 13, 2004 @06:58PM (#11076795)
      Torrents (AFAIK) don't download in random order but close enough for a simple explanation. They certainly do not download from beginning to end. One fix might be to dl in dynamic chunks that where the chunks are in order, but within the chunk it may not be in order. By dynamic, I mean that the chunk size could be dynamic according to the speed of the dl.

      A kick ass technology would be to put a swarming like technology into a transparent caching web proxy server. Typically this would be installed by the ISP or at the outside edge of a network where the UL and DL speeds are symetric and FAST.

    • Re:Swarming + streaming (Score:2)

      by snookums ( 48954 ) on Monday December 13, 2004 @07:25PM (#11077012)
      When you download something via BitTorrent, it's downloaded in random order, as pieces become available. While this works, it means you've got a huge file on your hard disk, but it's completely useless because random pieces are utter garbage bytes. For example, unlike with a straight download, you can't start watching a video file that's still being saved to disk.

      I would contest this on three grounds

      1) If you use an operating system and filesystem which support sparse files then the missing blocks will be zeros, not random garbage, and the file will only be as big as the data downloaded.

      2) A media file or archive with random chunks missing is far from useless. So long as the file header is available (see next point), you should be able to preview a media file for content or quality; and view the listing of an indexed archive or extract some of the contained files.

      3) Advanced Bittorrent clients such as Azureus can prioritise specific parts of files -- i.e. the beginnings of files where the metadata usually live.

      That said, I have to agree that having "I want this bit" built in to the protocol would have to be an advantage. Imagine having this "swarmstreaming" built into an archive manager or media editor. Download the header/metadata, then grab just the archived files or sample clip that you want. Awesome!

  • related presentation on javalobby (Score:2, Informative)

    by Anonymous Coward on Monday December 13, 2004 @06:11PM (#11076363)
    There is a link to a presentation about this stuff on javalobby:

    http://www.javalobby.com/eps/swarmstream/

  • Anyone notice the time this story was posted (Score:4, Funny)

    by Anonymous Coward on Monday December 13, 2004 @06:20PM (#11076434)
    24 hour PST - 13:37.

  • similar (Score:2)

    by minus_273 ( 174041 ) <`aaaaa' `at' `SPAM.yahoo.com'> on Monday December 13, 2004 @06:23PM (#11076466) Journal
    I actually wored on somehting similar in college. The difference being that i trtied to pay attention to the netowrk structure and chose hosts close to me. I have noticed more than the avalibility of bandwidth, it is the usage of it that makes the big difference in networks like these. In other word the overlay topology should be such that it tries to find the host that is closest and tries to take advantage of existing connections on common links. Unfortunatly, so far, here is so far nothing that compares to multicast on the router level and most systems do not support it.

    • Re:similar (Score:2)

      by cyngus ( 753668 ) on Monday December 13, 2004 @06:37PM (#11076619)
      I worked on something similar as well. What I always found funny was reading the papers that said they had solved the problem by using hashing techniques to assign IDs and data to nodes. Of course, while it was easy to see that the number of hops was low, the node IDs had little or no relation to "closeness" between nodes in either a latency or bandwidth sense. The great trick that no one has really figured out is how to make the node IDs (a the overlay network) build itself in a way that accounts for resource availability between nodes AND deal with nodes coming and going. Its cool stuff to think about, particularly because it always seems like the idea is just around the corner.

  • the concept is appealing BUT... (Score:3, Insightful)

    by __aailob1448 ( 541069 ) on Monday December 13, 2004 @06:33PM (#11076572) Journal
    First of all, where is the proof of concept??? I've been loking that the website and I see no sample that I can download using this "awesome swarmstreaming technology". What the hell is up with that? I launched the simulation and the java thingie is downloading at 7 k/s.....can you say underwhelmed?

    Second, there is a lot of boasting , marketspeak and references to patents, business and whatnot. We're far, far away from GPL territory here. At least bittorrent is opensource.

  • Suprnova? (Score:3, Insightful)

    by Lord_Dweomer ( 648696 ) on Monday December 13, 2004 @06:37PM (#11076615) Homepage
    I think we might just be witnessing the straw that breaks the camels back in terms of people shifting their viewing from the tv to the computer.

    Imagine if instead of having to wait a few hours downloading torrents off of Suprnova, you could simply browse through their catalogue (which I swear is bigger than Blockbuster's, and has music and tv shows), click something you wanted to watch, and BAM, its on.

    Welcome to the future of Internet TV. I just hope the law doesn't fuck it up.

    • Re:Suprnova? (Score:3, Insightful)

      by advocate_one ( 662832 ) on Tuesday December 14, 2004 @03:35AM (#11079572)
      that only works as long as several people have got the file and have left themselves seeding it... and it's very easy for it to be strangled by isps giving their customers highly assymetric down up ratios... In order to pull the stuff down at the fair speed required for live playback would require a ridiculous number of people to remain seeding several weeks after the initial upload.

    • Re:Suprnova? (Score:3, Insightful)

      by burns210 ( 572621 ) <maburns@gmail.com> on Tuesday December 14, 2004 @04:32AM (#11079743) Homepage Journal
      Would be a killer dedicated app. A bittorrent(well, swarmstream) client, that does a tivo/tv guide menu and presents a nice big video screen... Like a quicktime+tv guide+tivo...

      Would be a KILLER app. Being able to download and save, schedule, find 'if you like this you might also like...' shows. Works with tv and movies.

      Man, that would be a powerful use of a couple megs of harddrive space.
  • Here is a good presentation of this technology:
    Justin's presentation [javalobby.com] (Flash). I just watched it earlier today.

  • Prevention of Downloading Bogus Files! (Score:2, Interesting)

    by dos4who ( 564592 ) <top_mcse@h[ ]ail.com ['otm' in gap]> on Monday December 13, 2004 @07:03PM (#11076829)

    This looks like it could be the next big thing in preventing the download of large bogus files.

    Currently, in p2p programs (ala Kazaa, etc), you'd have to download the entire 600 MB file "Lord of the Rings.avi" (or "Busty Nurses. avi".. depending on your cinematic preferences), only to realize that someone has posted a bogus video in it's place.

    Swarming the file (ie: "Lord of the Rings.avi"), would allow you to preview various portions of the file to ensure it's integrity... (personal integrity aside) before downloading the entire file to your local PC.

    This is going to really pi$$ off the MPAA

Slashdot Top Deals

Money is the root of all evil, and man needs roots.

Close