The Road To Terabit Ethernet

stinkymountain writes "Pre-standard 40 Gigabit and 100 Gigabit Ethernet products — server network interface cards, switch uplinks and switches — are expected to hit the market later this year. Standards-compliant products are expected to ship in the second half of next year, not long after the expected June 2010 ratification of the 802.3ba standard. Despite the global economic slowdown, global revenue for 10G fixed Ethernet switches doubled in 2008, according to Infonetics. There is pent-up demand for 40 Gigabit and 100 Gigabit Ethernet, says John D'Ambrosia, chair of the 802.3ba task force in the IEEE and a senior research scientist at Force10 Networks. 'There are a number of people already who are using link aggregation to try and create pipes of that capacity,' he says. 'It's not the cleanest way to do things...(but) people already need that capacity.' D'Ambrosia says even though 40/100G Ethernet products haven't arrived yet, he's already thinking ahead to terabit Ethernet standards and products by 2015. 'We are going to see a call for a higher speed much sooner than we saw the call for this generation' of 10/40/100G Ethernet, he says."

Physics?

[happy_place]

Does anyone know what are the physical limitations of highspeed ethernet? I mean at some point doesn't it become impossible to move electrons or modulate data any faster?

Re:Physics?

[Penguinoflight]

That's more a problem with copper wiring. Cat5e seems to have problems reaching above 700-800 Mbits/s, I assume cat6 does better but wouldn't expect to see 10Gbit or even close.

At this point we haven't really started to see limitations on how fast a fiber optic connection can be switched, although I wouldn't doubt there being a theoretical limit.

Re:Physics?

[Shakrai]

Umm, I've reached 950 Mbit/s on Cat5e before. Fairly long (70-80 yards) runs of it too. Are you sure you aren't running into a limitation of your hardware? I've seen a lot of PCs with crummy NICs or slow PCI buses that can't reach full gigabit speeds no matter how good the cabling is.

Re:More data forces the need for more bandwidth

[petermgreen]

I don't see gigabit being superseeded for connections to end systems anytime soon. 10GBASE-CX requires expensive cable and has annoying run-length limitations. IIRC 10GBASE-T is a power hog. Fiber is both expensive and a PITA for such applications (I very much doubt fiber patch cords would last very long in a typical desktop environment)

It might be an idea to select gigabit switches with the capability to handle 10 gigabit uplinks though.

Re:Physics?

[empiricistrob]

That's a bit hard to say. But here's a way of thinking about it:

The Shannon-Hartley theorem [wikipedia.org] states that the channel capacity (e.g. the data bandwidth, measured in bits per second) is related to the channel bandwidth (measured in hertz). If we assume a very pessimistic signal to noise ratio of 1:1, the SH theorem says that the cable's bandwidth in hertz will be the same as the cable's bandwidth in bps.

So if we want a cable capable of transmitting information at 1tbps, the cable will need a bandwidth of roughly 1000 GHz. That means that it would be impossible to carry that amount of information using even microwaves. We're talking about at minimum infrared light. Or in other words -- we're talking about fiber optics, not cat5.

Re:Physics?

[setagllib]

We can always keep adding more bandwidth - in the extreme case (as in TFS) by trunking together more of the same links. But latency is not really improving. Ethernet itself is very high-latency compared to e.g. Infiniband. But fundamental limits of latency are impossible to overcome, and the best you can do is get closer and closer, perhaps asymptotically so. Between our planet and another, any latency in hardware is going to be a rounding error compared to the latency in the electromagnetic waves themselves, which propagate at "only" the speed of light.

Please make IEEE-1588 a standard part of 1TbE

[wowbagger]

An open letter to any hardware vendor considering making chips for these higher speed protocols:

Please add the timestamp counters needed to support IEEE-1588 Precise Timing Protocol [nist.gov]. These counters don't add much in the way of complexity when added to the NIC, but they are VERY complex to add after the fact.

Being able to synchronize the clocks of 2 hosts to 5nS or less may seem esoteric right now, but for these sorts of transfer speeds, you are going to have a significant number of users (Test and Measurement folks like me, scientists at places like CERN and FermiLab, grid computing) who will need that kind of time sync.

And it will be integrated directly into the CPU

[egghat]

cause the PCIe bus is way too slow for transporting terabits.

Or am I wrong?

bye egghat