Stories
Slash Boxes
Comments
Search

News for nerds, stuff that matters

Slashdot Log In

Log In

Create Account  |  Retrieve Password

Network Monitoring Appliance Looks Below 1 Microsecond

Posted by ScuttleMonkey on Mon Oct 22, 2007 04:41 PM
from the bofh-to-heroe-in-no-time-flat dept.
Networking IT
eweekhickins writes "Corvil has unveiled a new tool to help network managers cope with increasing pressure to improve performance. This appliance, from the Dublin-based company (with backing from Cisco), passively monitors traffic across networks in segments below 1 microsecond in length and correlates monitoring data with remote appliances and gives a complete picture of latency, jitter, packet loss and other phenomena that affect network and application performance. Corvil CEO Donal Byrne noted that 'If you can drop a millisecond [of latency] off, you're a hero.'"
+ -
story

Related Stories

Submission: Network Monitoring Appliance Makes Heroes by eweekhickins (1167593)
This discussion has been archived. No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.

Network Monitoring Appliance Looks Below 1 Microsecond 50 Comments More /

 Full
 Abbreviated
 Hidden
More
Loading... please wait.

  • Drop a millisecond (Score:1, Interesting)

    by Anonymous Coward
    "If you can drop a millisecond [of latency] off, you're a hero."

    This is the kind of attitude that breeds the Scotty types (you know who you are). If you can cut 2 ms, then only cut 1 ms now and save the other for when you really need it. And when the company is going to spend thousands for analysis, then suddenly cut the last 1 ms.

    • Re: (Score:3, Interesting)

      by BSAtHome (455370)
      However, it might be more effective to make your application more tolerant to latency (and fix your TCP window first).

      • Re:Drop a millisecond (Score:4, Informative)

        by molo (94384) on Monday October 22 2007, @05:10PM (#21078171) Journal
        Some applications are natively sensitive to latency and jitter. Consider VOIP or teleconferencing, or algorithmic stock trading.

        -molo

        • Time for token ring? (Score:3, Insightful)

          by khasim (1285)

          Some applications are natively sensitive to latency and jitter. Consider VOIP or teleconferencing, or algorithmic stock trading.

          I guess that would depend upon where both points are. One has to be on your network. The other ... ?

          Now, with Ethernet, one machine can hog the switch (I'll guess that they aren't using hubs). What use is shaving a millisecond off the app if you're still vulnerable to someone else hogging the network at the moment that you're trying to complete your transaction?

          • Re: (Score:2, Interesting)

            by morgan_greywolf (835522)

            Now, with Ethernet, one machine can hog the switch (I'll guess that they aren't using hubs). What use is shaving a millisecond off the app if you're still vulnerable to someone else hogging the network at the moment that you're trying to complete your transaction?

            That's what proper network segmenting is for. The guy that hogs the bandwidth usually has some business need to do so (but not always ;). Anyway, say the CAD guys do large file transfers multiple times a day. Well, you segment them off. That wa

            • Or perhaps implement QoS and throw in VLANs for good measure?

            • Actually, that is what QOS is for. Segmenting off everyone that wants to do some transfer makes for a fragmented unsummarized network with stretch VLANS all over god and country. Segmenting in little networks is fine...but a disaster in large ones.

              • Re: (Score:2, Interesting)

                by smellotron (1039250)

                Actually, that is what QOS is for. Segmenting off everyone that wants to do some transfer makes for a fragmented unsummarized network with stretch VLANS all over god and country. Segmenting in little networks is fine...but a disaster in large ones.

                You're missing where one of the parents commented about cases where speed matters. If you're doing algorithmic trading and you're using software QoS, and your competitor is using physical hardware segmentation, your competitor wins (all other things being equ

              • Re: (Score:2, Informative)

                by Anonymous Coward
                >Actually, that is what QOS is for. Segmenting off everyone that wants to do some transfer makes for a fragmented unsummarized network with stretch VLANS all over god and country. Segmenting in little networks is fine...but a disaster in large ones.

                All of which just basically proves how shitty collision-based networking is, especially as network size and speeds increase: You have to throw more and more hardware at it, to preserve performance.

                The only thing that switching and VLANs do, is attempt to recti
                • >Because, when you get right down to it - it's still Ethernet, and so, is still basically CSMA/CA, though the switches, VLANs, etc., hide it for the most part.

                  When did wired Ethernet become CSMA/CA, and what decade are you in? Collision-based networking? The CD in CSMA/CD has been irrelevant after almost a decade of full-duplex microsegmentation, effectively rendering the MA point-to-point, rather than "multiple access", and throwing out the CS in favour of "empty your buffers as fast as you can".

                  If you
                    • Broadcast storms can be avoided with even a modicum of proper planning. Of course you can hack it, but you can hack anything if you have the patience and skill. That does by no means constitute any fundamental failures of Ethernet.
                    • That's not what I'm saying at all.

                      You can effectively stop broadcast storms at layer 2 with the right implementation, so what I'm saying is akin to saying "Well, sure, you can hack Windows with a few simple scripts, but that doesn't constitute any fundamental failure in the concept of operating systems."

                      There are of course better things than vanilla Ethernet for regular IP networks, but Ethernet is still a very viable strategy if implemented correctly.

        • Some applications are natively sensitive to latency and jitter. Consider VOIP or teleconferencing, or algorithmic stock trading.

          Most VoIP codecs can work with a maximum of 30ms Jitter. You can't drop below 1ms because of the latency implied by the network equipments (just to go through their hardware takes a few milliseconds - not to mention stateful equipment such as firewalls or load balancers, etc.)

          Also, I wonder how they can passively measure latency or jitter - accurately, that is. Network Testin

          • Re: (Score:3, Funny)

            by amorsen (7485)
            You can't drop below 1ms because of the latency implied by the network equipments (just to go through their hardware takes a few milliseconds - not to mention stateful equipment such as firewalls or load balancers, etc.)

            Here's a nickel, kid, go buy yourself a real firewall.
            • I read my post again and I wasn't clear. I didn't mean that one firewall created implied several millisecond of latency (although this can be true when you reach some critical load). But every device adds some latency, lower than 1ms, and since packets go through a whole lot of these equipments, in the end you can't really drop below a certain amount of latency.
    • "If you can drop a millisecond [of latency] off, you're a Hero."

      Does this mean that a next step in human evolution is being able to measure time with microsecond accuracy?
      • I wouldn't believe it if I hadn't seen it for myself, but the users at my previous company could detect a 0.3 millisecond additional delay on a fibre line less than 100m long (they were whiny son's of bitches anyway). We took out the old Ethernet line out for some reason or another, and when we switched them over to this link there was instant complaints. They didn't even know we'd made the switch but still they cried.

        So I guess we've already evolved that far... the next step would have to be inbuilt dart

        • Re: (Score:2)

          by Mr Z (6791)

          Note: the delay appears to have been the switches at either end not working nicely with the new link medium

          Was there any sort of increased packet loss? Also, was it merely an average increase of 0.3ms? If there were any sort of peaks in the latency, i.e. increased jitter, that could be much more noticeable than an average latency increase might suggest.

          If your signals travel the speed of light, the propagation delay from pt. A to pt. B (100m in your case) should be around .33us (microseconds). Propaga

          • Unfortunately (or thankfully as the case may be), my information is second hand, coming from the system administrator at the time rather than my own checks.

            There wasn't any packet loss that I am aware of, some pretty intensive (proprietary) UDP applications operating across the link (there was no TFTP style checks on this) and dropping a packet would have been noticed. Average latency was, (for example) 0.7ms and increased to 1.0ms... some fairly standard diagnostics such as a continuous ping showed no maj
  • Can anyone explain to me what the advantages of this actually is?
    sorry if I sound stupid. It seems like greak to me. I'm just used wireshark etc
    • I am definitly not a subject matter expert... however using wireshark to trace packets from a specific box to another with intentions of determining and fixing a network issue is much different than activly monitoring and storing all traffic going through your switches. Wireshark is "on-demand" while what they are talking about is "real-time".

      The breakthrough appears to be that it is the fastest of these type of devices available.

    • Re: (Score:3, Funny)

      by evil agent (918566)

      sorry if I sound stupid. It seems like greak to me.

      That just about says it all...

      • Ah, sorry about the spelling in that post everyone!
        I managed to miss out more words, and make more spelling mistakes in that single post than I usually do in a week.
        I guess I need some more Coffee.

    • Re:and again in layman's terms?? (Score:5, Informative)

      by DigitalCH (582593) on Monday October 22 2007, @05:50PM (#21078609)
      The benefit depends on the person using it. Take an investment bank and an algorithmic trading system. Most of your money is made on volume, the faster you reply the more deals you get, the more volume you have, the more money you make. I've seen a lot of presentations at investment banks where every 5 milliseconds they shave off is $50+ million/year more money they make. Keep in mind that most of these companies have gotten to the point where they can do round trip for the whole trade transaction in 5 milliseconds or less. So each millisecond is like a 20% improvement.

    • Re: (Score:3, Informative)

      by thegrassyknowl (762218)
      It seems quite simple. I took the following from the article:

      They timestamp the packet at some point in the network and when it arrives at the other side they timestamp it again to work out the trip time. Not really rocket science, but they seem to have come up with ways of measuring time pretty accurately at two different places and keeping the clocks in sync or working around clock drift in their measurements.

      The other part of their system is some algorithmic work that correlates packets and tries to wo

      • Re: (Score:2, Funny)

        by Virgil Tibbs (999791)
        I did of Course mean Greek.
        And that I was "used to" Wireshark.

        FYI, my Greek is much worse, than my first language, English. It is even worse than French and Russian, two other languages I can speak to varying proficiency.
        However, sir, I am very tired.
        At the end of a long day, it is not unusual to think in straight lines while typing nonsense.
        Moreover, thank you kindly sire, for asking God to help me;
        I needed some guidance to help me past morons like you.
      • If you "Greak" is as bad as your English
            ***

      • sir, that was a "low blow".
        Please see various other posts of mine, to see why that was uncalled for and unnecessary.
  • No offense guys, but unless you can make something that cuts the ping time in half, we won't be having any good FPS games against the Americans without increasing the ping from 60ms to 250ms or higher. 249ms won't cut it. It just won't.
    • Damn you Americans and your low latency connections, I'm lucky to get under 300ms to most game servers (500+ for World of Warcraft) and I am on adsl with no interleaving.

    • You'll never go faster than the speed of light (Score:4, Informative)

      by NSash (711724) on Monday October 22 2007, @06:19PM (#21078929) Journal
      "There is an old network saying: Bandwidth problems can be cured with money. Latency problems are harder because the speed of light is fixed - you can't bribe God."

      A beam of light takes roughly 1/7 of a second to travel around the world. That means that if you're playing on a server on the other side of the world, your ping will always be at least 143 ms. That's a hard physical limit: the only way to decrease that time would be to drill a hole through the Earth, or move closer.

      • Better living through physics... (Score:1, Insightful)

        by Anonymous Coward
        Electrical impulses propogate much, _much_ slower than the speed of light when run through copper (and perhaps fiber optics since the light beam has to bounce around so much that the path is many times longer?), so your hard limit may be ~143 ms, but only if your signal went through vacuum all the way to the server and back.

        The real latency should be actually much higher due to switching or forwarding overhead and any monitoring that the NSA does. 300ms at least.

        • Re: (Score:2)

          by Agripa (139780)

          Electrical impulses propogate much, _much_ slower than the speed of light when run through copper (and perhaps fiber optics since the light beam has to bounce around so much that the path is many times longer?)

          "Much much slower" is a pretty big exaggeration. Propagation velocity using a solid polyethylene dielectric is 66% and that is about as slow as it gets for electrical signals. Some glasses are as low as 50%. Store and forward ethernet switches at 100 Mbits/s have to add at least 150 microseconds pe

      • That's why I only attend LAN parties in the vacuum of outer space.

      • A beam of light takes roughly 1/7 of a second to travel around the world.
        So the hard limit would be 1/14 to get a packet halfway around the planet (approx 71 ms) which actually is pretty fast!
        • and another 1/14 to get it back again making a total ping time of 1/7 of a second.

          and light in fiber moves slower than light in free space.

          Of course UK to USA isn't as much as halfway round the world.

    • Re: (Score:2)

      by dodobh (65811)
      Just host your own server and make the Americans play against you.
  • Argh, the buzzwording! It burns us! Make it stop!!

    More seriously though, the article might benefit from a little bit more context. As mentioned above, taking 1ms off network latency is meaningless across long connections, where you expect 40ms latency just from the routers, speed of light, etc. Taking 1ms off when microseconds count, when your latency is 5ms, within a system where automated transactions and big money are involved, then the situation is different.

    The readers do not recognize the requirem

  • RIPE NCC Test Traffic services (Score:4, Informative)

    by jjgm (663044) on Monday October 22 2007, @05:25PM (#21078337)

    The RIPE NCC [ripe.net]'s Special Projects [ripe.net] group have been offering sub-microsecond latency/jitter/analytical services to ISPs for years. Their data is invaluable and unique, since it measures latency, jitter and packetloss in a single direction (unlike ICMP Ping, which is a round-trip measurement over an asymmetric path) and goes back at least to 2000. The paper claims accuracy to 0.0006 ms, which was good for the time when the product was designed.

    Read about the project here [ripe.net] and the paper on TTM [ripe.net] [pdf] that was presented at the PAM2001 conference [ripe.net].

    (This isn't what Corvil do.)

  • "For every packet [that the appliance records], we compute a signature and a time stamp"

    "When it exits on the other side of the WAN, we time-stamp it, and we can correlate the data across the whole network"

    Well, doh ... how about NMAP and Wireshark ...

    'Byrne said Corvil's customer base is "more than 10 but less than 100."'

    What ever happened to that perpetual motion outfit ..

    • Re:Oh goodie! (Score:5, Informative)

      by mccalli (323026) on Monday October 22 2007, @05:12PM (#21078183) Homepage
      Now I can get those random stock tips in my email in less milliseconds! I will be rich one day, I will!

      Milliseconds count. Maybe not to your stock tips, but trust me as someone who has spent about a decade in this kind of environment now - sub-millisecond latencies certainly count in automated trading between investment banks/hedge funds/whatever. To the point where people are prepared to pay fortunes to have their machines located physically closer to an exchange.

      For fun, check out arbitrage [wikipedia.org], and then ponder again why reducing latency might be important in a competitive environment. Think about highly liquid markets, such as spot foreign exchange.

      Cheers,
      Ian

      • buffering ......... (Score:3, Interesting)

        by khasim (1285)

        Milliseconds count. Maybe not to your stock tips, but trust me as someone who has spent about a decade in this kind of environment now - sub-millisecond latencies certainly count in automated trading between investment banks/hedge funds/whatever. To the point where people are prepared to pay fortunes to have their machines located physically closer to an exchange.

        A more logical reason would be to reduce the possible traffic issues.

        If I'm sitting on the network with a 100Mb/s connection straight to the serve

        • Re: (Score:3, Interesting)

          by mccalli (323026)
          So looking to shave a micro-second/milli-second off of a packet isn't that important or realistic. Humans do NOT make decisions that fast. You'd do better improving the speed of your code or throwing faster hardware at it.

          Humans make decisions at a minimum of around 200ms I think - that's from memory, so I expect someone to be along and give the real figure soon. But I'm not speaking about humans, I'm speaking about algorithmic trading in a competitive environment. It truly is that significant to remove

          • But I'm not speaking about humans, I'm speaking about algorithmic trading in a competitive environment.

            So get liquid nitrogen and overclock your processor.

            Speeding up the computer running the algorithm is more productive than trying to get your packet through 1 millisecond faster.

            Think about clustered pricing engines and similar, all trying to price as fast as possible to both a) capture business and b) avoid arbitrage.

            I understand that.

            I also understand that if you're LOSING because you're one millisecond

            • Re:Because you can buy faster hardware. (Score:5, Informative)

              by mccalli (323026) on Monday October 22 2007, @06:00PM (#21078715) Homepage
              What is the buffer capacity of the server's NIC?... How long does it take to empty it?... What was the guy just before you doing? Did he fill it?

              Sorry, but do you really think people don't do that level of analysis as well as trying to improve the network speed?

              My point is that if you're looking at spending money for a 1 millisecond gain, you've already lost sight of the goal.

              The goal in this kind of app is low latency - every millisecond counts. There are other goals of course, throughput, guaranteed maximums as well as low minimums...but in this case we were specifically discussing latency.

              And that's not even counting a router or everything that can slow down your Internet connection

              Internet connection? Who's talking about an internet connection? Dedicated leased lines direct to the exchange, internal transfer between machines...this kind of stuff isn't One Man And His PC sitting at home trying to day-trade. Yes there's variability, but even so engineering it out as much as possible is certainly an aim.

              All the levels of analysis you describe, from the algorithm right through to the NIC, are already being done. At some point they will be done better, because of a change in available tools. This appears to be one of those tools.

              Cheers,
              Ian
            • In a financial trading arbitrage environment, that millisecond would literally be worth millions of dollars. Yes, it matters that much. Some of the best and most expensive brains in the software and systems engineering world are paid a whole lot of money to try to gain that millisecond. At least one "dorm room to gazillionaire" story was built on just such an edge in the early 1990's. The resulting trading firm has $10 billion in net profits since 1998. Warning, there be flash here! http://www.citadelgroup.com/ [citadelgroup.com]

              Do not assume that the people interested in this level of performance are idiots. There's always the possibility they know more about what they're doing than you do.

              doc

          • I used to work in Derivatives Market Making, specifically in their connectivity group. Latency is a *very* big deal. Arbitrage aside, essentially market making works like this- whoever has the best price gets the trade. For the sake of simplicity, lets assume everyone prices an option the same way all the time. Lets also assume that everyone can also price an option at the same exact speed. A new price tick comes in, forcing the options to all reprice at once. The guy who gets his quote out there first is g

        • Re: (Score:2, Insightful)

          by Passresv (1177973)
          Hey Khasim. Can I say, I have to disagree. Analysing latency on large networks is incredibly important. There are often so many latency-inducing mechanisms between a packet leaving an application and arriving at the other end (misconfigured stacks, poor cabling, misconfigured routers and switches (#1 cause!), greedy applications, messy applications, multicast floods etc etc), that it is impossible to tell what the problem is. Modern networks are straining with latency problems that added bandwidth doesn't a

      • Re: (Score:2)

        by Eivind (15695)
        Yeah, sure. But that's zero-sum games. Having low latency here doesn't help at all. What helps is having lower latency than the *other* traders.

        If *everyone* improves their latency equally much, they're all back to square one.

        Which is typical of speculation (as opposed to investment) -- its fundamentally stupid playing a zero-sum game when positive-sum games are around.

    • Re: (Score:1, Insightful)

      by Anonymous Coward
      Or you work for a shitty company (ie: small, unimportant, does nothing) that can't get demo units from other companies.

All trademarks and copyrights on this page are owned by their respective owners. Comments are owned by the Poster. The Rest © 1997-2010 Geeknet, Inc.