Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 



Forgot your password?
typodupeerror
Get HideMyAss! VPN, PC Mag's Top 10 VPNs of 2016 for 55% off for a Limited Time ×
Networking Media The Internet

BBC Optimizing UHD Video Streaming Over IP (bbc.co.uk) 72

johnslater writes: A friend at the BBC has written a short description of his project to deliver UHD video over IP networks. The application bypasses the OS network stack, and constructs network packets directly in a buffer shared with the network hardware, achieving a ten-fold throughput improvement. He writes: "Using this technique, we can send or receive uncompressed UHD 2160p50 video (more than 8 Gbps) using a single CPU core, leaving all the rest of the server's cores free for video processing." This is part of a broader BBC project to develop an end-to-end IP-based studio system.
This discussion has been archived. No new comments can be posted.

BBC Optimizing UHD Video Streaming Over IP

Comments Filter:
  • Interesting (Score:5, Interesting)

    by jd ( 1658 ) <imipak@nOSpaM.yahoo.com> on Saturday October 10, 2015 @03:20AM (#50697855) Homepage Journal

    Kernel bypass plus zero copy are, of course, old-hat. Worked on such stuff at Lightfleet, back when it did this stuff called work. Infiniband and the RDMA Consortium had been working on it for longer yet.

    What sort of performance increase can you achieve?

    Well, Ethernet latencies tend to run into milliseconds for just the stack. Tens, if not hundreds, of milliseconds for anything real. Infiniband can achieve eight microsecond latencies. SPI can get down to two milliseconds.

    So you can certainly achieve the sorts of latency improvements quoted. It's hard work, especially when operating purely in software, but it can actually be done. It's about bloody time, too. This stuff should have been standard in 2005, not 2015! Bloody slowpokes. Back in my day, we had to shovel our own packets! In the snow! Uphill! Both ways!

    • Re: Interesting (Score:4, Informative)

      by tomalpha ( 746163 ) * on Saturday October 10, 2015 @05:06AM (#50698043)

      Yeah, this kind of thing has been around for a while.

      These days the added latency of going through the kernel IP stack is generally measured in micro rather than milliseconds but the difference is still the same order of magnitude. Solarflare, Mellanox and others will happily sell you expensive Ethernet network cards that come bundled with drivers that let you bypass the kernel IP stack. The stack itself isn't especially slow but the system call and extra memcpys still do all add up. I've also seen an in-house user space stack built largely on top of lwIP [wikipedia.org].

      So I'd agree that none of this particularly new, but I reckon it's still interesting that the BBC is using it. Maybe that'll help spur more widespread adoption.

    • by azav ( 469988 )

      Care to explain a little of the overall design?

  • If people would just accept a decent MTU none of this would matter.
    The max is 64 K but we're stuck with 1500 (including overhead) because you can't be sure that every hop will support your MTU.
    Internally you can enable jumbo frames and shit will work, but once you need to go out over the internet all bets are off, so you limit your shit to 1500 and your performance goes to all hell.

    We're basically delivering UHD movies via telegram.

    • by Zarhan ( 415465 )

      The use case here is moving uncompressed video within a studio environment. In here, you have full control over the hardware and Internet does not come into play. I'd think that in such cases they have no problems in going to jumbo frames.

    • by Anonymous Coward

      Jumbo frames play very badly when you have other stuff going over the same link though. Each connection can't send a packet until the previous one has finished sending, and those gaps are much further apart when using jumbo frames. Yes it does improve throughput, but only when you're using it for approximately one thing.

    • If people would just accept a decent MTU none of this would matter.
      The max is 64 K but we're stuck with 1500 (including overhead) because you can't be sure that every hop will support your MTU.
      Internally you can enable jumbo frames and shit will work, but once you need to go out over the internet all bets are off, so you limit your shit to 1500 and your performance goes to all hell.

      We're basically delivering UHD movies via telegram.

      Packet size is a tradeoff - for high throughput you want big packets, for low latency you want small packets. So fine, just tailor the packet size to your application - well no, when you're sharing a network, the packet sizes used by other applications have a significant impact.

      So lets say you're doing something that requires a low latency, such as VoIP. And lets say you've got QoS set up to ensure the small VoIP packets are always inserted in front of any big packets, since that's a sensible thing to do.

      • by Mandrel ( 765308 )

        Packet size is a tradeoff - for high throughput you want big packets, for low latency you want small packets.

        There'd be no such trade-off if routers and computers pipelined packets, starting (or queuing) to forward as soon as the destination IP address is read and an interface route determined, possibly also waiting to check the TCP/IP header checksum.

      • 64 * 1024 * 8 / 2 / 1000 / 1000 = 262 ms worst case, not 328 ms.

        And routers should know the capability of the links and can split up the jumbo frames into multiple packets to let VOIP through ahead without wasting much bandwidth at all. Hell, my shitty D-Link does this - every boot it scans the link to determine connection speed and uses that in its QoS engine.

        Further, the use case in the article is 8 Gbps in a studio environment. They can dedicate the entire link to video. 8 Gbps down a 2 Mbps pipe is

    • by Bengie ( 1121981 )
      A 64KiB packets on a 10Mb/s connection is about 5ms. That's a huge amount of jitter. 64KiB packets may be acceptable for 10Gb connections, but I like to keep my connection below 1ms of jitter. To give an idea of how horrible 5ms of jitter is, I get about 2ms of jitter from Midwest USA to Frankfurt Germany.
      • And for a 10 Mbps connection you drop the max MTU and split the packets. Routers in the middle of a path can do this.

        Video Streaming Service A sends a 64 KB packet to ISP B over a 100 Mbps link, ISP B knows Customer C is on the Shit Tier package and can handle 10 Mbps, and decides to split up the 64 KB packet into 4 KB or whatever packets, Customer C gets their shit.

        4 KB / 10 Mbps 64 KB / 100 Mbps, no additional jitter. Without even inspecting the traffic to see if it's Netflix or Skype or their own VoIP

        • by Bengie ( 1121981 )
          IPv6 does not allow packet fragmentation by the routers. You also have the issue that if a single fragment is dropped, the entire packet must be retransmitted.
  • https://birds-are-nice.me/musi... [birds-are-nice.me]

    I show how the concept of the public domain has been crushed by demonstrating just how little popular music exists in it.

    • Damnit, posted to the wrong story! That was supposed to go to the one about TPP.

    • I show how the concept of the public domain has been crushed by demonstrating just how little popular music exists in it.

      Are you sure it wouldn't suck anyway?

      • Musical styles change. Entire genres of music have been invented in the last seventy years, the current copyright term for music here. There's no justification for a duration so long - in what way does it promote the creation of new music? It doesn't.

  • by SeaFox ( 739806 ) on Saturday October 10, 2015 @04:40AM (#50697997)

    How about the BBC stop requiring Flash for videos. That would be a better place to start.

    • Okey dokey [bbc.co.uk].

    • by AmiMoJo ( 196126 )

      4k is aiming a bit low anyway. NHK, the Japanese equivalent of the BBC, I'd going directly to 8k for the 2020 Olympics. Test broadcasts will begin in 2018, about 2.5 years from now. 4k is going to be short lived.

      • by Anonymous Coward

        Meanwhile, 4k is going to be needed (I use the phrase in the loosest possible way) and 2020 is 5 years away. Solving the problems at 4k are steps towards solving them with 8k and faster processors / faster network infrastructure / better equipment / better compression / better decompression.

  • by Rosco P. Coltrane ( 209368 ) on Saturday October 10, 2015 @06:03AM (#50698139)

    to be able to watch Eastenders in Ultra HD...

    To the Beeb's credit though, the Sky at Night in UHD would definitely be a lot more interesting, surely. But out of thousands of mediocre shows and movies released year after year after year, is it worth buying a new tv to marvel at a dozen really good programs? Somehow this don't seem to be a good value proposition.

    • Re:I can't wait (Score:5, Informative)

      by SuricouRaven ( 1897204 ) on Saturday October 10, 2015 @06:20AM (#50698165)

      This isn't for you to watch UHD. It's for internal use in production, so they can shunt live UHD video around their studios. That way they keep full quality right up until the final stage before distribution, when it gets resized according to the end device. Your TV will get plain old 1080p as always - but they'll have UHD capability ready to go for transmitting to cinemas or sending to big public displays, and they can archive a UHD version for future use so they can zoom in tighter on the action in future highlights.

    • Interestingly, the place to look for UHD content is YouTube (and recently Vimeo, as well). The flexibility of the 'amateur' video producer and that of the internet as a distribution platform really show in this area.

      There is some beautiful and awesome stuff out there:
      https://vimeo.com/115541651 [vimeo.com]
      https://www.youtube.com/watch?... [youtube.com]
      https://www.youtube.com/watch?... [youtube.com]

      Given that the high-end smartphones are outputting UHD movies now as well, there is going to be an onslaught of UHD content.

  • by Anonymous Coward

    In order to pick up UHD I need to connect that round antenna to the back of the TV, right?

  • Congratulations BBC, you managed to re-invent DPDK at the license payers expense.
  • now I can watch reruns of top gear and star trek TNG in UHD

  • Intel's DPDK library is specifically built to bypass the OS and provide high-speed low-latency networking. Sems like a natural fit.

Interchangeable parts won't.

Working...