Forgot your password?
typodupeerror
The Internet

Robust Timing Over the Internet 178

Posted by kdawson
from the no-present-like-the-time dept.
ChelleChelle writes "The NTP (Network Time Protocol) system for synchronizing computer clocks has been around for decades and has worked well for most general-purpose timing uses. However, new developments, such as the increasingly precise timing demands of the finance industry, are driving the need for a more precise and reliable network timing system. Julien Ridoux and Darryl Veitch from the University of Melbourne are working on such a system as part of the Radclock Project. In this article they share some of their expertise on synchronizing network clocks. The authors tackle the key challenge — taming delay variability — and provide useful guidelines for designing robust network timing algorithms."
This discussion has been archived. No new comments can be posted.

Robust Timing Over the Internet

Comments Filter:
  • by Anonymous Coward on Sunday May 02, 2010 @02:22AM (#32061884)

    let's do away with the arbitrage, gambling, and bullshit from wall street. make them own a stock for ONE WHOLE DAY. No more of this low-latency trading bullshit. They're just skimming money off the top of the financial markets--away from the regular folks. EVERY PENNY THESE GUYS MAKE COMES OUT OF OUR POCKET.

    • Re: (Score:3, Insightful)

      by IorDMUX (870522)

      EVERY PENNY THESE GUYS MAKE COMES OUT OF OUR POCKET.

      I see you have not the foggiest idea how "investments" and "fluidity" make the economy work.

      Hint: When the Great Depression hit, the money lost by the traders on Wall Street did not suddenly spew forth into the hands of "us".

      • Re: (Score:3, Insightful)

        by 1s44c (552956)

        Hint: When the Great Depression hit, the money lost by the traders on Wall Street did not suddenly spew forth into the hands of "us".

        Because the money you mention never really existed.

        If I had a thousand dollars in paper money under my bed before the great depression it would have had a certain buying power. In the middle of the great depression it would have had a much greater buying power. The value lost on Wall Street spewed forth into paper money and real physical assets.

        • by lennier (44736)

          Because the money you mention never really existed.

          If something doesn't exist, how come it has an effect, either good or bad, on the real world of goods and services?

          • Re: (Score:3, Insightful)

            by RockDoctor (15477)

            If something doesn't exist, how come it has an effect, either good or bad, on the real world of goods and services?

            Examine the concept of "god" and you'll see the answer.
            "god" does not exist in reality ; the concept exists perfectly well, but in physical reality there ain't no such puppy ; however even an convinced atheist can't deny that the non-existent entity "god" has real effects on life, the universe and 42. Richard Dawkins, amongst others, makes a steady income selling books pointing out how much dam

          • by sjames (1099)

            It's a sort of shared dream (or lately nightmare) that fools refuse to wake from. Consider, before the bubble popped and after, we had the same people available to work, the same consumer demands and the same needs and transportation channels. The bubble popping didn't change the consumer demand, transportation channels, need, availability to work, or any other factor that should have any effect on the economy, but suddenly it's in the dumper.

            It's as if we have put 5 year olds in charge of the country and n

    • Re: (Score:3, Interesting)

      by drsmithy (35869)

      let's do away with the arbitrage, gambling, and bullshit from wall street. make them own a stock for ONE WHOLE DAY. No more of this low-latency trading bullshit.

      A Capital Gains Tax that started at 95% and decreased ~2.5% per month would go a long way towards fixing many problems (/avoiding them in the first place).

      • by Sycraft-fu (314770) on Sunday May 02, 2010 @05:37AM (#32062502)

        While I think that's a bit extreme, I think you and the GP are both on to the right track. I think stocks should have a fairly short minimum hold time. A day would probably be fine, just something such that you aren't trying to gamble with them more or less, because that's why high speed day trading really is. It is gambling in every way that Vegas is.

        I also thing a sliding tax would make sense. If you execute quick trades, you are subjected to a fairly high tax. 95% is a bit excessive, but something still fairly high. Then, as you say, the tax drops on a monthly basis until it hits zero. That would act to encourage investing, rather than gambling. It rewards people who put their money away for the future, and do so in a manner that allows it to help the economy grow. It would also still allow for quicker trades, but the government gets to take a cut in that case.

        I really do think we need to start looking at something like this for three reasons:

        1) Because there are companies that try to game the system. That's never a good thing. You game the system on a large scale, you'll break it. When you are talking about something highly important to the economy, that's not acceptable.

        2) Because there are too many gamblers. There are plenty of traders who basically gamble on the market. They make day trades not based on anything about the company, but just to try and grab money from trends. This is not useful, and isn't what the stock market is for. It leads to unnecessary instability and thus should be discouraged.

        3) To help combat people's fear response. A non-trivial part of the big drop in the stock market last year was people being fearful. They panicked because of the economic troubles and wanted all their money out. That means even if the fund managers thought it was a bad idea, they had no choice. You have to do what your client wants. That of course lead to more drops and more panic and so on. Perhaps tax incentives could help stave that off with people.

        • While I think that's a bit extreme, I think you and the GP are both on to the right track. I think stocks should have a fairly short minimum hold time. A day would probably be fine, just something such that you aren't trying to gamble with them more or less, because that's why high speed day trading really is. It is gambling in every way that Vegas is.

          What does timespan have to do with it? Surely if something is unethical to do over an hour, it's unethical to do over a day or a week, or six months. The pr

        • by brianerst (549609) on Sunday May 02, 2010 @01:30PM (#32065040) Homepage

          I think people outside of the industry don't really understand the need for day traders.

          I write trading software for the futures and options industries (I was the lead architect for the CBOT's Order Routing System, and have written trading systems for other exchanges and banks). The type of day trading that you and the GP have issues with is an absolute requirement for liquid markets and the efficient working of those markets.

          Let's say you are a firm that needs to buy a large number of shares or contracts (in the futures world, you buy contracts for the future delivery of product). In the type of buy and hold market that seems "fair", that large order is going to go into a market of other buy and hold traders that are only going to trade out of their positions if the trade makes sense in their long-term strategy. Huge effective tax rates are going to make that trade unprofitable unless you've held the position a very long time (increasing your risk of market volatility) or you charge a premium (increase the price) in order to offset the tax hit.

          This type of "illiquid" market raises prices dramatically. You have to factor in the time value of money, the risk exposure of holding shares for a long time (increasing the likelihood of a bad market event) and tax overhead. All that means that buying shares/contracts is riskier and selling them is more expensive.

          Day trading fixes this. Day traders don't really care about the long-term direction of the market - they make money on minor intra-day price fluctuations. Because of this, they are nearly always willing to take on and shed positions. And because there are a lot of them chasing the same tiny fluctuations, they "shrink the spread" - they're going to give you a very good price because they shrink the difference between the bid and offer (price to buy/price to sell).

          Now, when that original firm wants to buy that large number of shares, they are doing that in a market where there is almost always a large number of shares being offered at competitive prices. The day traders will sell off shares and pick up shares all day, just trying to end the day flat (holding nothing) to reduce after market risk.

          Day trading does extract a price (they are making money) on the market, but the efficiency gains that they bring outweighs that price in almost all markets. They distribute a lot of the risk involved in trading among a bunch of smaller firms, rather than concentrating all the risk in big funds. This is generally a good thing - the government and the market in general are better off when a small firm goes bust than when one of the "too big to fail" firms starts having problems.

          Now, the rise of algorithmic trading is causing some stress in this pattern - I can see some real issues coming in the next few years as these algorithms become more and more precise. I'm less concerned about a rogue algorithm dragging a market down (or up) than of all the algos converging to a point where the market gets stuck because the algos are just too damn efficient. But that's a post for another day...

          • by CecilPL (1258010)

            This is the first actually well-informed post in this thread.

          • I wish I had mod points. Would the OP rather have the government perform this function? Is that even possible?
          • by lennier (44736)

            Day trading fixes this. Day traders don't really care about the long-term direction of the market - they make money on minor intra-day price fluctuations. Because of this, they are nearly always willing to take on and shed positions. And because there are a lot of them chasing the same tiny fluctuations, they "shrink the spread" - they're going to give you a very good price because they shrink the difference between the bid and offer (price to buy/price to sell).

            So basically, at the poker table of share trading, day traders are the designated suckers?

            • by lennier (44736)

              hmm, or is it more like there's a big high-stakes poker game going on (the long-term investors), but then there's a secondary game of bookies and punters (the day traders) betting on the outcome of each hand? And with the ability for the high rollers to borrow funds from the bookies, and/or buy each other's hands?

              But then there are syndicates among the bookies (the hedge funds) which are themselves so big that they become high rollers in themselves... they have no stake in the main game but are looking to c

          • by sjames (1099)

            Or we could just dump that entire "market". I use scare quotes because the people in it do not have any capacity whatsoever to take delivery of the goods or to actually provide them on demand (other than to order someone else to send 'their' commodity to another destination). In fact, if even a tiny fraction of what they buy and sell were actually delivered to them, it would be an un-mitigated disaster.

            Howsabout if company A needs a commodity they buy it from one or more producers of that commodity?

            Our econ

      • by micheas (231635)

        let's do away with the arbitrage, gambling, and bullshit from wall street. make them own a stock for ONE WHOLE DAY. No more of this low-latency trading bullshit.

        A Capital Gains Tax that started at 95% and decreased ~2.5% per month would go a long way towards fixing many problems (/avoiding them in the first place).

        So you short against the box. (buy N shares and sell N shares short collaterallizing the type two account (the account that buys the stock) with the cash in the type 3 account (the account that is short the stock)

        Worst case, you have to deliver the borrowed stock before 38 months have passed, and get hit with a tax bill when you unwind the trade in a private transaction between your two accounts.

        I am sure that there are other workarounds.

        • Re: (Score:2, Insightful)

          by swamp_ig (466489)

          Easy Answer: (in most tax laws)

          Any arangement who's primary purpose is to avoid taxation is illegal, and will be charged tax at the rate at which it would normally have been taxed.

    • Agreed. When are some of these people going to realize that greed is bad. Trying to game a system is inherently wrong. Not just morally, but wrong systemically. It will always result in harm to the system. These people soothe their conscience with excuses- "We're spreading the risk and that's a good thing", "It's only fractions of pennies off of many people, nobody will notice." , etc. Until the whole thing comes crashing down because in reality the financial system is core dependent on actual investment an

      • Re: (Score:3, Interesting)

        by mgblst (80109)

        Who the hell isn't greedy. If you were offered more money at work, would you turn it down. Most people wouldn't. And all your colleagues are going to get paid more.

        This is all these people are doing, they are doing there jobs, and they get paid bonuses for doing it well.

        It is ignorant and stupid to pretend that this situation is just that simple.

        • by kerrbear (163235)

          The offer for more money at work is part of the system. You provide a valuable service and they compensate you for it. They trust you to do useful labor, and you trust them to compensate you for it. It works because they make money off your labor and everybody is happy. If somebody was to write a program to take the .00001 cents rounded off from a company's transactions (a la Office Space) then they are trying to game the system. They are taking money without providing anything useful in return. It amounts

        • by sjames (1099)

          No, some of them knowingly set the company on the path to ruin in order to maximize their position. They don't care because they plan to deploy the golden parachute long before the crash.

          Most people would accept a freely offered raise consequence free. Fewer would take the raise if they were told one of them would be fired to pay for it. Fewer still would demand the raise and suggest firing someone, particularly if they already had enough money to retire comfortably on.

          So, no, most people aren't that greedy

    • Re: (Score:3, Insightful)

      by 1s44c (552956)

      let's do away with the arbitrage, gambling, and bullshit from wall street. make them own a stock for ONE WHOLE DAY. No more of this low-latency trading bullshit. They're just skimming money off the top of the financial markets--away from the regular folks. EVERY PENNY THESE GUYS MAKE COMES OUT OF OUR POCKET.

      You are absolutely, painfully right and it pains me deeply that I don't have any mod points.

      These jokers want time more accurate than NTP can provide but I'll bet most of them still believe time is a universal invariant. They don't need time more accurate than NTP can provide, they need to be hit in the head with a hammer until they stop moving.

    • Re: (Score:2, Interesting)

      by alexhard (778254)

      That's a ridiculous idea. It would increase volatility tremendously, increase spreads tremendously, increase mispricing tremendously, and of course destroy any semblance of liquidity for half the securities out there. Saying that arbitrageurs are skimming is like saying market makers are skimming. Partly true, but it completely ignores the positive effects of their actions for the market.

      • by atomic777 (860023) on Sunday May 02, 2010 @11:57AM (#32064288)

        Parent characterises the complete detachment from reality that characterises the modern "innovative" financial industry.

        I operate a small business. I don't give a fsck about your claims that we need "liquidity" when I can't get a simple business loan for less than 8-9%, despite being well-capitalised, while my personal savings attract all of r pct, where 0.01 < r < 0.1. Quite the scam there. As a result, I use only retained earnings to pay for anything, which while a prudent thing to do given my situation, is hardly a recipe for large-scale economic growth. The financial industry does not fulfil its most basic imperative of funnelling capital to productive use, why should I care about whether some institutions can hold a stock for 1ms or 1 day?

        I suppose things could be worse. Maybe in addition to being completely starved for capital, I may be subject to "protection" payments in the near future just to be able to stay in business at all! Financial innovation, 'Ndrangheta style! Wouldn't that be a boon for Wall Street and the City of London.

        • Parent characterises the complete detachment from reality that characterises the modern "innovative" financial industry.

          Actually the parent you refer to understands the difference between the equity market and the debt market. Imposing a restriction that stocks must be held for a day or other restrictions on equity trading would have costs that those proposing them don't understand. That is not directly connected to the consumer portion of debt markets you later refer to when you say you can't get a business loan at a rate you like or earn an interest rate you want. Actually since debt instruments can be bundled and sold lik

          • by atomic777 (860023)

            You are right, my rant did go astray and I realize that debt and equity are not the same thing. I am, however, trying to explain things from the perspective of a small business owner, you know, the little guy that the financial industry is walking all over.

            If we reverted to a very old fashioned banking system, 3% on deposits, 6% loans, bankers on the golf course at 3pm, i would be better off. With all the "innovation" and technology giving us "liquidity" in the debt and equity markets, what do i see? My d

    • Mod parent down. (Score:5, Insightful)

      by Anonymous Coward on Sunday May 02, 2010 @05:54AM (#32062556)

      Actually this kind of trading (especially arbitrage) narrows the spread making it cheaper for ordinary people to enter the market.
      So if you ban it you go back to only the big boys being able to afford to execute trades.

      However, using any system, high frequency or otherwise, to deliberately manipulate the market is a different thing.

      I'm sorry I'm posting as AC but it could affect my job.

      • by the eric conspiracy (20178) on Sunday May 02, 2010 @10:15AM (#32063610)

        Mod parent up. The efficiency and liquidity of world wide stock markets increases their usefulness to all involved. My ability as an individual investor to buy and sell at will is a huge factor in my willingness to participate.

        My father used to pay $100 to get a simple trade executed, and it would take hours to get it done. I can do the same trade for $5.00 and get it executed in a second or less.

        So what if some financial institution is making a profit on this. It is a win-win situation.

        The problems in the system are elsewhere - where people in the system are taking advantage of their position to trade based on information not available to everyone. Fortunately most of that is illegal.

    • Yes, an end to day-trading would be good. In fact, I would vote for mandates requiring even longer periods of ownership. It makes "pump and dump" schemes much more difficult to pull off and at vastly greater risk to the would-be criminal.
  • I like the trust yourself part. I actually implemented a dirty script to restrict how much ntp can change my clock speed and I call every minute from crond ;-))

    Limits are hardcoded and depend on the machine ;-))

    $ cat setclockfrequency
    #!/bin/sh

    # 251616 = 3.839
    # 269120 = 4.106
    # 280448 = 4.279
    # 288544 = 4.403
    # 9372208 = 143.009
    # 10703936 = 163.329
    # 11308784 = 172.558

    ###let LOWLIMIT=243520
    ###let HILIMIT=286496

    let LOWLIMIT=10072208
    let HILIMIT=10408784

    let FREQ=`adjtimex -p | grep frequency | cut -d ":" -f 2 | cut

    • Can you please explain what the issue is you are trying to solve with this?

      • Re:Trust yourself... (Score:5, Interesting)

        by ls671 (1122017) * on Sunday May 02, 2010 @03:44AM (#32062158) Homepage

        Network issues. If say, on average, once every 25 polls, ntpd doesn't get realistic data from higher stratum servers because of the network, don't let ntpd crank up the frequency to ridiculous values like it does when this occurs.

        There are realistic values for the frequency on every machine with a good clock. It is ridiculous to set the frequency below or above these values.

        Last time I checked, there is no way in ntpd to configure these values. The typical ntp guru reply will be: "Get a decent network connection". The author in TFA noted that such a "decent connection" is virtually impossible to achieve because of "variable delay". He also noted that it is sometime better to trust yourself, which is kind of what that script does.
         

        • Re: (Score:3, Interesting)

          by ls671 (1122017) *

          Author of TFA also mention:

          "More generally, the algorithm should be designed never to overreact to anything. Remember, its view of the world is always approximate and may be wrong, so why try to be too clever when inaction works so well? Unfortunately, feedback algorithms such as ntpd have more reactive strategies that drive the clock more strongly in the direction of their opinions. This is a major source of their nonrobustness to disruptive events."

          I fully agree with this and I realized it several years a

  • TFA already points out that there are existing solutions to this problem... sure, they're expensive, but when their example of the finance industry is involved, people will happily pay six, seven figures to get an edge over the competition anyways. If it's not worth the cost, you probably don't need that level of precision. Nice to improve the system, but how much benefit will anyone see from it?
    • by Z00L00K (682162)

      In my opinion the finance industry have to live with some minor differences in timing.

      What this has really done now is to provide an idea of an opening for fraudulent behavior in the finance industry. Just imagine what would happen if someone was able to inject timing faults into the servers when their favorite transaction was executed.

      • Are you talking of the situation where the stock goes up and I buy it, but I frig it so it looks like I bought it at the old (lower) price?

        The time doesn't matter, as long as the sequence is known. But I don't think the system works like that; quantity and price are always included, AFAIK, plus it wouldn't match the counterparty's books.

        • by compro01 (777531)

          One example goes kinda like this

          1. News breaks overnight that will make people want to buy company FOO stock.
          2. The instant (as determined by the very accurate clocks on their servers) the market opens, trading firm BAR buys up all the available FOO stock and then instantly puts it back up for sale for a little bit more.
          3. The rush of people buying the stock buy it from firm BAR.

          Rinse and repeat every day and you're talking about billions of dollars made by firm BAR that people without ultra accurate clock

  • GPS (Score:4, Interesting)

    by dracocat (554744) on Sunday May 02, 2010 @02:31AM (#32061918)

    How about GPS receiver + exact known location?

    I would think just eliminating an unknown latency and replacing it with a known one (radio waves generally travel at a consistent rate) would get you a pretty accurate time. If you add one more known variable which is the exact location of the device, then you should be able to get an even more accurate time.

    • by at10u8 (179705)
      to get what accuracy? the ionosphere can mess up GPS signals by about 100 nanoseconds
      • by dkf (304284)

        the ionosphere can mess up GPS signals by about 100 nanoseconds

        That's variable. You use the GPS time signal over a long period of time (days) to calculate the rate at which your clock drifts, at which point you can do a reasonably good correction for things and get your clock pretty accurate. Asking for perfect timing a second or two after switching the device on is just unfair!

    • by Bios_Hakr (68586)

      GPS is okay. But I would think that a financial institution would prefer to invest in a good cesium-beam frequency standard. These units are calibrated at the factory and drift something like one second every few thousand years. Plus, there is no need to have an external antenna to pick up GPS signals. If you *absolutely* need stable and accurate clocking, cesium is the only way to go.

    • by SEWilco (27983)
      You don't need to worry about the radio wave propagation, just wait for the GPS device to lock on and start querying its time. Much of the effort of the GPS device is in synchronizing its clock with the GPS clocks, resulting in probably less than 1 microsecond of error.
    • How about GPS receiver + exact known location?

      I would think just eliminating an unknown latency and replacing it with a known one (radio waves generally travel at a consistent rate) would get you a pretty accurate time.

      While radio waves do travel a constant speed, what they don't do is travel a consistent path between the satellite and the receiver. Some of this is due to the satellite's orbital motion, and the timing errors resulting from that can be eliminated. However, the path is also altered

    • Would make sense (Score:5, Informative)

      by Sycraft-fu (314770) on Sunday May 02, 2010 @05:26AM (#32062472)

      GPS is accurate to about 50 nanoseconds. All kinds of devices that need precision time get it from GPS. You don't need much more than a standard receiver, just one that is designed to place a high priority on time updates. The GPS system itself keeps very accurate time since each satellite has an atomic clock, and they all sync to the master clock.

      To me that would seem the best way, if accuracy is really important and the systems are high end. Have each device have its own receiver and just have them sync to that. Don't sync them to each other, since you aren't going to get anything more accurate.

    • by Zoxed (676559)

      > How about GPS receiver + exact known location ?

      But if you roll-your-own do not forget that GPS time does not have the leap seconds that UTC has !!

    • by vxice (1690200)
      how about this, slow down. If you are going fast enough that 'propagation' times affect your decisions you don't need more accurate clocks but therapy for OCD.
  • by at10u8 (179705) on Sunday May 02, 2010 @02:34AM (#32061926)
    Microsoft refuses to consider the notion [microsoft.com] of clocks accurate to within 2 seconds.
    • by Bios_Hakr (68586) <xptical.gmail@com> on Sunday May 02, 2010 @02:49AM (#32062000) Homepage

      I don't really see a problem with that. You wouldn't use Notepad for a mission-critical editing, why would you use W32Time for mission-critical timing?

      That being said, I don't know what 3rd-party system you would use for sub-second timing.

      • I don't really see a problem with that. You wouldn't use Notepad for a mission-critical editing, why would you use W32Time for mission-critical timing?

        Time synchronisation is crucial to Active Directory security (since it uses kerberos). In fact, I think a major part of joining windows desktops to a windows server/domain is that they all start syncing time.

  • PTPd? (Score:5, Interesting)

    by Jeremi (14640) on Sunday May 02, 2010 @02:49AM (#32061996) Homepage

    I'm surprised the article didn't mention PTPd [wikipedia.org], which is an implementation of the IEEE 1588 precision time-synchronization standard. I was under the impression that was the standard way to solve this sort of problem when NTP wasn't enough.

    • Re:PTPd? (Score:5, Informative)

      by apharov (598871) on Sunday May 02, 2010 @05:11AM (#32062416)
      (disclaimer: just finished my Master's thesis on a related subject) PTPd is ok, but not in itself up-to-date at the moment. It doesn't implement the most recent IEEE 1588-2008 standard, which has significant improvements compared to the 1588-2002. About the 1588 in general: its main selling point is the ability to do hardware timestamping (when using hardware with support!) of the two-way timing messages between master and slave. This eliminates the very significant timing jitter that happens in the software stack before the messages are timestamped. For reference, commercially available master-slave implementations using IEEE 1588 achieve synchronisation within tens of nanoseconds within LAN, and microseconds to tens of microseconds within WAN, depending on network conditions. So overall I think that while RADclock might be ok as an alternative between NTP and IEEE 1588, it doesn't really bring anything new to the table. Some of the stuff in the Rideaux/Veitch paper has also been used with IEEE 1588 for quite some time, for instance the filtering for fast timing packets is a necessity for accurate synchronisation with IEEE 1588.
      • by iritant (156271)

        Thanks for the informative post that was on topic. One question: with 1588 what sort of hierarchy do you set up? Does everyone have a rubidium or cesium clock attached?

        • by pyite (140350)

          One question: with 1588 what sort of hierarchy do you set up? Does everyone have a rubidium or cesium clock attached?

          A typical way is to use a GPS GrandMaster clock that outputs PTP. Some of these have good oscillators in them to keep good time if GPS dies for whatever reason. The tricky part is getting this time to many machines without bad drift.

      • PTPd is ok, but not in itself up-to-date at the moment.

        Maybe someone should adjust its clock? ;)

  • by Anonymous Coward

    It's not clear to me the financial system needs such high precision timing.

    My cynicism tells me they need it now to implement fraudulent micro-second trading, where see what you're buying, correlate with other people's buys and then buy a fraction of a second ahead of you.

    At any rate, it's not clear they need it, or that giving it to them won't lead to instability.

    Maybe someone should actually design a financial system, and design one that can use imprecise, or precision timing.

    • If you're talking about "front running", you don't need accurate timing for that. You (the broker) just slip your order in front of the customer's.

      • The purpose of this degree of accuracy is to measure the performance of your system.

        Here I am, team leader for Goldman Sacks Front-Runner IT services division and I'm trying to squeeze a couple of microseconds out of my trading network. Well I can't do that unless I can measure the performance of my distributed application very accurately.

    • by pyite (140350)

      It's not clear to me the financial system needs such high precision timing.

      A lot of it has to do with analytics and making sure your system is operating properly. Being able to compute network and software stack latencies often requires the clocks of multiple machines being very closely synced. NTP can't do it well enough in many cases. Without good synchronization, you end up (according to timestamps) receiving a packet before it was sent.

    • by JamesP (688957)

      Just add jitter

      Say 'your orders will be processed with a delay varying between 0 and 5s'

  • Ridiculous (Score:3, Interesting)

    by GNUALMAFUERTE (697061) <almafuerte@nOspAM.gmail.com> on Sunday May 02, 2010 @02:58AM (#32062036)

    NTP is accurate to within 10 milliseconds in any decent connection. Get a really good, stable, low latency connection and hook yourself up to a stratum 1 server and you can cut that number down tenfold ...

    Can anyone explain what kind of financial application would require bigger accuracy?

    I understand that certain scientific experiments require more accuracy than that, but those guys are probably already hooked up to stratum 0 servers directly.

    Why on earth would any financial system (yes, even the stock market) require 10ms time accuracy? They were able to rip us off with fucking sand clocks. NTP should be enough for them.

    • Re:Ridiculous (Score:5, Informative)

      by Anonymous Coward on Sunday May 02, 2010 @03:18AM (#32062080)

      This sort of thing probably has something to do with it:

      http://www.nytimes.com/2009/07/24/business/24trading.html [nytimes.com]

      • Re: (Score:3, Insightful)

        by MichaelSmith (789609)

        Its old fashioned day trading accelerated by a factor of a few million. Of course, you need to back it up with a good AI because humans can't react fast enough and a flatline with his ass in a can will react in a predictable way.

    • Re: (Score:3, Informative)

      by Todd Knarr (15451)

      These guys are probably trying to do sub-millisecond trading. They need to keep their clocks sync'd to the microsecond to keep different machines coordinated.

      Me, I wonder why they're bothering. Run GPS-based stratum-1 servers at each site, and sync directly to them across the LAN. Or get a cesium-beam primary time source. End of problem. Or take heed of the current financial meltdown and acknowledge that if you try to create money out of thin air, sooner or later you're going to pay the piper and it's going

    • The decline of /. (Score:3, Insightful)

      by swillden (191260)

      NTP is accurate to within 10 milliseconds in any decent connection. Get a really good, stable, low latency connection and hook yourself up to a stratum 1 server and you can cut that number down tenfold ...

      Can anyone explain what kind of financial application would require bigger accuracy?

      Where have all the nerds gone?

      Why are we debating whether or not the people who think they need highly accurate time really need it? What's cool about this article is the nerdly reaching for every higher degrees of accuracy, of digging deeply into a seriously difficult technical challenge and achieving new levels. What's cool for those of us who don't work on super-accurate timescales is the opportunity to read about how those who do think about this stuff, to see what math they use and what the issues

      • I don't think this is part of the decline of /.

        The trolls, the vacuous m$/apple/OS/FS/whatever mindless bashing/fanboys, the crappy editors, and other stuff like that is what's causing the decline of Slashdot.

        We are in the year 2010. Thinking just about the technical aspects isn't good enough anymore. What we have learn from RMS (Whether you like him or not) is that hacktivism is very important, if we want to continue hacking happily. Our world is at danger. We are heading towards an Orwellian future of tec

        • by swillden (191260)

          Thinking just about the technical aspects isn't good enough anymore.

          Clearly. Did you read the last sentence of my post?

          My point is that there was no discussion of the technical aspects.

  • ... is there a patch or driver hack for Windows that will allow the hardware clock to be set to GMT/UTC while the time zone offset is handled by the OS as it should be? It is such a pain in my ass that I run Linux with the occasional boot to Windows and having the time wrong for Windows. And of course if I set it, the time becomes wrong for Linux. Linux is doing it "right" and Windows is doing it wrong in my opinion. But what is the fix? I'd prefer Windows change its ways, not Linux.

  • I don't understand the need for wide-area highly accurate (less than 1ms accurate) timing. Once you get beyond a reasonable distance, the speed of light starts playing into the equation.
    • by 1s44c (552956) on Sunday May 02, 2010 @05:15AM (#32062426)

      I don't understand the need for wide-area highly accurate (less than 1ms accurate) timing. Once you get beyond a reasonable distance, the speed of light starts playing into the equation.

      Like most smart people you are forgetting how dumb most people are.

      Wall street traders don't understand much about time. They only understand how to obfuscate financial transactions like a high speed shell game. To them more speed means more chances to skim off a percentage and the laws of physics don't even get considered.

      • by renoX (11677)

        Uh? I've read here that some traders are paying to put their hardware the closest they can to the trading servers so that they can have low latency..
        So I'd guess that this is more a case of an article with a flawed example than anything else..

    • Re: (Score:3, Insightful)

      by Linker3000 (626634)

      I vote that all traders - and lawyers and estate agents for that matter - are sent such a reasonable distance from the planet that the speed of light indeed starts playing into the equation.

      ALl in the name of research, of course.

  • Hi!

    Last few years, most cellular operators are rolling out IP to the base stations.
    For HSPA/LTE speeds, E1/T1 PDH/ATM is out of the question,
    and as IP is rolled out, GSM gets there as well.

    Accurate timing is essential: even transmitter frequency is synchronized over IP,
    and keeping 2 GHz accurate to 100 Hz is not simple.
    Local oscillator can do it short term, but external correction is needed as well.

    Beside GPS (used in IS-95), several systems are used.
    IEEE 1588v2 is obvious one. Works.

    Some vendors do it with

  • Don't diss NTP! (Score:3, Informative)

    by Terje Mathisen (128806) on Sunday May 02, 2010 @10:19AM (#32063628)

    The authors of the RADclock article don't seem to realize that most of the issues they "solve" with their algorithm has been considered, and also more or less solved, by the NTP hackers severeal years ago.

    I.e. when the article shows that in a variable-latency environment you need to filter the measurements by the minimum rtt: This is of course default behaviour for ntp as well, but only over a limited number of polling intervals (8). If this isn't sufficient then there are a number of configuration options available, like the huff-puff filter which can be configured to maintain rtt history long enough to "coast" past any expected periods of large and asymmetrical network delays.

    They also give out some _really_ bad advice, like claiming that "One server is enough": This is only true if you know that this particular server can never be wrong!

    If you want to survive the situation where a single servers starts to lie, you shold configure at least 4 independent sources.

    Personally I have seen 3 different gps-based stratum-1 black box ntp servers go crazy, suddenly returning timestamps far into the future.

    OTOH, my freebsd-based servers with Motorola Oncore UT+ timing optimized gps firmware has never returned bad time, but I still have 6 such servers spread around our corporate network.

    Terje

  • by m.dillon (147925) on Sunday May 02, 2010 @05:07PM (#32066458) Homepage

    The only really major issue with NTP is figuring out the time offset due to network latency in asymmetric network environments. The NTP protocol itself can frequency lock down to a few ppm over the internet without any problem at all. Offset errors are another matter. Over the internet 10 ms is about the best you can do. Over a LAN with a local GPS signal driving the protocol you can reduce the offset error to less than a microsecond and can probably get it down to less than 100ns without too much trouble (and without needing any hardware time stamping or netif queue prioritization), and you can frequency lock down to a few nanoseconds.

    Other issues include the fact that most motherboards do not have temperature compensated timebases, and they tend to float around a few ppm (for the better MBs) to a few hundred ppm (for the stupid MBs that put crystals next to heat sources), as well as bugs in the operating systems themselves which don't get noticed until you actually try to frequency lock your timebase. This creates a multiplicative effect if you are not connected to a stratum 1 or 2 server because each server in the chain is trying to do a frequency lock against a drifting MB timebase. The result is the leaf nodes in the chain can get frequency locks but the locks are to a drifted frequency instead of to the correct frequency because the server itself can't instantly correct for its drifting timebase. This is why you see the frequency lock jump around by +/- 50ppm in a seemingly uncontrollable fashion once you get past stratum 2.

    NTPD was famous for not being able to frequency lock while doing an offset correction at the same time. The NTPD program has also had many issues over years, such as not using a proper dual-staggered linear regression to control offset and frequency corrections, which is why I wrote dntpd for DragonFly. I wonder how many of these old issues with NTPD have been fixed over the years, maybe they have.

    Network jitter is largely irrelevant. It isn't a problem if you use a proper linear regression over a long enough period of time. The linear regression can be used to calculate what the jitter is and thus form a good knowledge of the baseline accuracy of the protocol. A medium sized run over a few minutes will get you down to 10-15 ppm on your frequency lock and you can get down to 1-5 ppm within about 10 minutes (assuming the motherboard has an accurate timebase of its own).

    The sad thing about all of this is that it takes just a single resistor to temperature compensate a crystal, a resistor most MB manufacturers don't put in. Some don't even use crystals anymore for their PLL input (and the crystal on the RTC doesn't help much because the RTC doesn't have a fine enough grain timer to poll easily). Sigh.

    -Matt

  • This is still NTP (Score:2, Informative)

    by Mr. Suck (62745)

    Although the summary and the article itself seem to take pains not to mention it, a visit to the RADclock homepage (http://www.cubinlab.ee.unimelb.edu.au/radclock/) will tell you that what's actually being offered here is an improved NTP client. No changes to the NTP servers, server software or NTP protocol are required or are proposed. The client improvements are in an improved filter topology (feed forward with quality assessment) and introduction of separate concepts of absolute and difference clocks opt

Pause for storage relocation.

Working...