Stories
Slash Boxes
Comments
Search

News for nerds, stuff that matters

Slashdot Log In

Log In

Create Account  |  Retrieve Password

Purdue Streams a Movie At 7.5Gb/sec

Posted by kdawson on Tue Nov 21, 2006 03:34 PM
from the fat-pipe dept.
Networking IT
the_psilo writes, "My friend just got back from the Supercomputing conference in Tampa, FL where she and the rest of the Purdue Envision Center rocked the High Performance Computing Bandwidth Challenge by streaming a 2-minute-long, 125-GB movie over a 10-Gb link at 7.5 Gb/sec. They used 6 Apple Xserve RAIDs connected to 12 clients projecting onto their tiled wall (that's 12 streams in all). Lots of accolades from the people who set up the challenge. More links to articles and reviews can be found at the Envision Center Bandwidth Challenge FAQ page." The two-minute video is a scientific visualization of a cell structure from a bacterium. The Envision Center site hosts a reduced version of the video.
+ -
story
This discussion has been archived. No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.

Purdue Streams a Movie At 7.5Gb/sec 50 Comments More /

 Full
 Abbreviated
 Hidden
More
Loading... please wait.
  • We all hope that this marvel of IT world will be used for things better than video streaming!
  • This just proves that porn really is the driving factor in Internet growth. Cells are the basic building blocks of people and people are what make (most) porn, porn.

      • Completely off-topic, but I wonder if I'm the only one that laughed hysterically at 'onky'

        Just a funny sounding word.

  • Hmmm, looks a bit blocky to me. I think they need more key-frames, and less compression.

  • Reduced version? (Score:5, Funny)

    by smooth wombat (796938) on Tuesday November 21 2006, @03:41PM (#16938706) Homepage Journal
    We don't need no stinkin reduced version!

    I'm sure our employers wouldn't mind if we took a look at the full version.

    *psst* *psst* *psst* *mumble* *mumble* *mumble*

    The whole thing? Really?

    My boss has told me to take the full version of my personal desk stuff home now.

  • Sooo... (Score:3, Interesting)

    by TubeSteak (669689) on Tuesday November 21 2006, @03:46PM (#16938812) Journal
    2 minutes long & 125-GB
    "A resolution of 4096x3072, with 24-bit color, running at 30 frames per second,"

    What codec did they use?

    • Re: (Score:2, Informative)

      by Anonymous Coward
      if you do the math, none
      4096*3072(res)*3(bytes per pixel)*30(frames per second)*120(seconds) = 126Gb of data :)
      so no compression, just the raw data

    • Re:Sooo... (Score:4, Informative)

      by Anonymous Coward on Tuesday November 21 2006, @03:59PM (#16939060)
      Probably no compression at all:

      4096 * 3072 * 3 bytes per pixel = 36MB per frame (exactly... resolution I'll bet was not chosen randomly )

      36 * 30 frames * 120 seconds = 126.5625 GB

      About time the envision center actually did something. They have been a huge money pit for a long time. They demolished the pool hall in the student union which had been a student favorite for about 20 years for the center and have had very little actual results up until this point.

      Purdue Alum

  • Hmmmm (Score:4, Funny)

    by truthsearch (249536) on Tuesday November 21 2006, @03:47PM (#16938826) Homepage Journal
    A 2-minute-long, 125-GB movie... that must have been one super-high resolution chicken.

  • Not impressive (Score:3, Interesting)

    by MetricT (128876) on Tuesday November 21 2006, @03:49PM (#16938872) Homepage
    That's not that impressive in the scheme of things (ie Supercomputing 2006). We (Vanderbilt) were there, and we were streaming 35 Gb/sec for hours on end over a parallel filesystem we're developing. And even we weren't the fastest there. Going to Supercomputing is like stepping 3-4 years into the future.
      • We did. Unofficially of course, but we would have come in 2nd place in the bandwidth challenge. And we had less than 24 hours to prepare, so we're really looking forward to next year.
  • Two minutes??? But I want it noooooow!

  • And... (Score:4, Funny)

    by jo42 (227475) on Tuesday November 21 2006, @03:52PM (#16938918) Homepage
    ...underneath the table, behind the curtain where 12 BETAMAX machines hooked directly into the screens...


    :-p

  • Why in the world do they [perdue.com] need 7.5Gb/sec video streaming? Are they ramping up for a hi-def knockoff of Burger King's silly idea [subservientchicken.com]?

  • Use for the bandwidth (Score:5, Interesting)

    by Anonymous Coward on Tuesday November 21 2006, @04:05PM (#16939154)
    I have a friend who just got back from the same conference. He works at Ohio State University in the electron optics facility (read: likes to play with awesome equipment). Here's what he sent along to me: "This week I am in Tampa, Fl at the Supercomputing Conference "SC 06" with the guys from OSC (Ohio SuperComputer group). We have connected the Global Link boxes up and I am attempting to run the Quanta from Florida. Traffic is being routed from the conference to Atlanta, the Internet2, then though to the Third Frontier Network and back to OSU." With the bandwidth he had, he was able to observe, interact, and analyze a sample in a scanning electron microscope (the Quanta) in Ohio, in real time. Now that's a use for bandwidth; it's a lot cheaper to have a great internet connection than to buy the latest electron microscopes.
    • You really dont need such datarates, even for the newest, best, ect ones.
      You simple dont get that much information out of electron detectors.
      If you want to push bandwith, you need high sample rates and get "real time" rubbish noisy shit.
      And for good statistics, 1-2 Mbit are more than enough. You arent playing Maxwells Daemon, you know, so there is no atom to catch or something....

      At least that my opinion, as someone who was also dissapointed the first time he noticed that the 3 million $ SEM only outputs XG

  • Why a 10g link would average 7.5g (Score:5, Informative)

    by twigles (756194) on Tuesday November 21 2006, @04:13PM (#16939296)
    When you try and fill a 10g pipe with a single tcp session, the congestion avoidance mechanisms of tcp will prevent you from filling the pipe. Essentially the sender will ramp up the rate of packets very quickly initially until the receiver sends back a congestion notification. The sender will then cut the send rate *in half*, and climb it back up very slowly - 1 extra byte per round-trip if memory serves (don't quote me on that). This works great for 100m, but to climb from 5g to 10g takes about 30 minutes if you have a cross-US round-trip-time (RTT).

    To get around this you can:
    1. Patch your TCP stacks with a few high-performance modifications
    2. Figure out - using the RTT, interface buffer sizes, and bandwidth - what the number of outstanding packets can be before the receiver sends back a "slow down" message. Then configure the sender to have a smaller packet queue.

    Great article on this here:
    http://www.cisco.com/web/about/ac123/ac147/archive d_issues/ipj_9-2/gigabit_tcp.html [cisco.com]

    It's tough to say if that was the problem here (I'm actually assuming it was not) since after a little digging I didn't see any details on their implementation. And no, I'm not interested in truly digging (I have a pesky job thingy to get back to).
    • while i know some will gawck at the "in half" but that makes sence.. if it is continued.. if it ramped up then halved it up and then if no feedback faved it up again and then haved down using the last known good rate as the base then you would find the optimum transmission rate in log n time .. which is damn good.. but climbing back up at 1 byte per round trip is obseen.. i am not sure.. i can't remember how ti does it.. i didn't realize the frame control was that bad.. if it is someone seriously needs to

    • When you try and fill a 10g pipe with a single tcp session, the congestion avoidance mechanisms of tcp will prevent you from filling the pipe. Essentially the sender will ramp up the rate of packets very quickly initially until the receiver sends back a congestion notification. The sender will then cut the send rate *in half*, and climb it back up very slowly - 1 extra byte per round-trip if memory serves (don't quote me on that). This works great for 100m, but to climb from 5g to 10g takes about 30 minut

  • Marge: Does the world really need that much porno?

    Homer: {{drool}} One Million Times faster
  • The two-minute video is a scientific visualization of a cell structure from a bacterium.
    It starts out noble that way.

    Within a year, they will be streaming the most vile pornography allowed by human morals.

  • The MPAA are set to reveal later today their new SLT (Streaming Lawsuit Technology) offering. This should make the extortion process easy and seamless for end-users.

  • I despise linked videos in wmv format (Score:4, Insightful)

    by daniel23 (605413) on Tuesday November 21 2006, @07:07PM (#16942304)

    I've seen this quite a lot recently here, linking videos from the front page and they are in .wmv format. My fireFox doesnt show them, it offers to load a plugin, but fails to do so. I can go "manual" and it guides me to microsoft.com where I can some DRM-infested POS which doesnt like my OS etcetc.

    Please, this is /. and not yahoo or msn, right? can we somehow convince our editors to stop standardising on microsoft orthodx video formats? thankx
  • I smell a disturbance in the Force; as if a 1000 MPAA execs all suddenly crapped their pants at the same time.
  • This is hardly interesting, this isn't an accomplishment of any significance. Not are others claiming this has been done before, its really not very difficult.

    The most difficult challenge, I think would be the matters of bus bandwidth and storage. As expected, the article makes note of the storage concerns, and they 'took the easy way out' and mirrored the data across several arrays.

    What would be more impressive is if they used native infiniband storage arrays, could access them as a single NAS mount (rat
  • "The Envision Center site hosts a reduced version of the video."

    Come on, this is slashdot, at least link to the full thing...

  • Xserve has 1 gbit ethernet (Score:3, Funny)

    by Fefe (6964) on Wednesday November 22 2006, @03:04AM (#16946390) Homepage
    How could they possibly reach 7.5 gbit with 6 Xserves, if each Xserve only has a 1 gbit ethernet connection?

    This looks to me like some desperate attempt to justify the money they wasted on bad (Apple!) hardware, drugs and hookers :-)
    • > Help me out: using 75% of a 10Gb/s link "rocks"? If I read that right, usage was 7.5Gb of 10Gb total. What exactly was the Feat?

      Daring to brave the power of this fully operational Slashdotting station.

      (Then again, I got a 14-megabyte .MPG of the movie in less time than it took to post this. We'll see if they're still alive by the time this hits the board, though.)

    • Re:Help me out: using 75% of a 10Gb/s link "rocks" (Score:5, Informative)

      by Zebra_X (13249) on Tuesday November 21 2006, @03:42PM (#16938716)
      There is significant overhead associated with the use of TCP/IP. A typical 6.0 Mb/s connection will deliver appx. 4.2 Mb/s this is only about 70% of the connections actual bandwidth. So, 75% is looking pretty good.

      What rocks is the ability to reliably deliver 7.5 Gb/s AND do something useful with it.
      • There is significant overhead associated with the use of TCP/IP.


        Not to mention the hardware itself (NICs, routers, bridges, etc.) adds some not-insignificant overhead as well.

        What rocks is the ability to reliably deliver 7.5 Gb/s AND do something useful with it.


        Right. While any idiot can get 10 Gb/s link and get 7.5 Gb/s or more out of it, the real feat here was in doing something useful with it at the same time. Remember that in streaming video applications, there's typically a lot of dropped packets because the client has to actually do something with the video immediately. It may just buffer it, but the application that's doing the buffering is often busy doing other stuff as well, like, say decoding video and audio streams and actually piping all that data through to the I/O bus -- oh, and the NIC is usually sitting on that same I/O bus, so that makes things even worse.
        • Not really. I agree with the part about the 7.5 Gb/s (as long as you are using PCI-e 10 Gb NICs as opposed to PCI-X). You can very easily get about 8.2 Gb/s running from memory to memory single stream...over 9 with multiple streams. What is difficult is getting it from disk that quickly. While it is interesting, it really is a toy experiment when compared to a super computer. Running parallel file transfers on thousands of nodes is where the real challenge lies and parallel file systems just don't work
      • Oh believe me, the Envision Center doesn't really do all that much that's truly "useful". Cool as hell, and even some stuff my work is tangentially related to, but not "useful".

        (Ex: immersive, interactive 3-d environment to teach math skills in American Sign Language. Really neat stuff. But until deaf schools get $10 million projection studios, um. Not "useful".)

      • Re:Help me out: using 75% of a 10Gb/s link "rocks" (Score:4, Informative)

        by PSC (107496) on Tuesday November 21 2006, @04:19PM (#16939416)
        There is significant overhead associated with the use of TCP/IP. A typical 6.0 Mb/s connection will deliver appx. 4.2 Mb/s this is only about 70% of the connections actual bandwidth.

        While there is overhead associated with TCP/IP, it's nowhere near 30%. On a 100 Mbit link in a LAN, you routinely get 11 MB/s (just verified by transferring an Ubuntu image via FTP over the local ethernet with noname switching hardware). With a theoretical maximum of 12.5 MB/s, that's an efficacy of 90%.

        6 Mbit sounds like a DSL connection to me. Quite possibly your provider or the servers you download from are responsible for your effective 4.2 Mbit, because TCP/IP isn't.

        • Re: (Score:3, Insightful)

          by cheater512 (783349)
          Actually it is that high. The faster you get, the higher the overhead percentage.
          If the size of the packets scaled with the bandwidth then the overhead percentage would remain constant.

          10gbps is around 150,000 packets (9k jumbo packets) per second. 100mbps is 1,500 packets per second (also using jumbo packets).
          As you can see the overhead increases with the bandwidth.
      • 6.0 Mb/s connection? 30% overhead? Are you basing your findings off your cable modem performance? 25-30% overhead is terrible. You're doing something wrong. When testing a couple 10Gb cards through a few different switches, the most overhead I saw was maybe 8-10%, tops.

        Also, there are several applications where a 10Gb connection would be used, so I don't think the fact that they used the 7.5 gigs is very impressive. To me it's the fact that they pushed video that played 7.5 Gbps. Now THAT would make f

All trademarks and copyrights on this page are owned by their respective owners. Comments are owned by the Poster. The Rest © 1997-2009 Geeknet, Inc.