Robust Timing Over the Internet 178
ChelleChelle writes "The NTP (Network Time Protocol) system for synchronizing computer clocks has been around for decades and has worked well for most general-purpose timing uses. However, new developments, such as the increasingly precise timing demands of the finance industry, are driving the need for a more precise and reliable network timing system. Julien Ridoux and Darryl Veitch from the University of Melbourne are working on such a system as part of the Radclock Project. In this article they share some of their expertise on synchronizing network clocks. The authors tackle the key challenge — taming delay variability — and provide useful guidelines for designing robust network timing algorithms."
make all wall street traders own stock for 1 day (Score:4, Insightful)
let's do away with the arbitrage, gambling, and bullshit from wall street. make them own a stock for ONE WHOLE DAY. No more of this low-latency trading bullshit. They're just skimming money off the top of the financial markets--away from the regular folks. EVERY PENNY THESE GUYS MAKE COMES OUT OF OUR POCKET.
Re:make all wall street traders own stock for 1 da (Score:3, Insightful)
EVERY PENNY THESE GUYS MAKE COMES OUT OF OUR POCKET.
I see you have not the foggiest idea how "investments" and "fluidity" make the economy work.
Hint: When the Great Depression hit, the money lost by the traders on Wall Street did not suddenly spew forth into the hands of "us".
Re:make all wall street traders own stock for 1 da (Score:2, Insightful)
Phew that's OK then, it's only everyone with a fucking superannuation.
Re:make all wall street traders own stock for 1 da (Score:1, Insightful)
Re:Ridiculous (Score:3, Insightful)
Its old fashioned day trading accelerated by a factor of a few million. Of course, you need to back it up with a good AI because humans can't react fast enough and a flatline with his ass in a can will react in a predictable way.
Re:make all wall street traders own stock for 1 da (Score:3, Insightful)
Agreed. When are some of these people going to realize that greed is bad. Trying to game a system is inherently wrong. Not just morally, but wrong systemically. It will always result in harm to the system. These people soothe their conscience with excuses- "We're spreading the risk and that's a good thing", "It's only fractions of pennies off of many people, nobody will notice." , etc. Until the whole thing comes crashing down because in reality the financial system is core dependent on actual investment and trust, not on gambling. To quote Chinatown:
Jake Gittes: How much are you worth?
Noah Cross: I have no idea. How much do you want?
Jake Gittes: I just wanna know what you're worth. More than 10 million?
Noah Cross: Oh my, yes!
Jake Gittes: Why are you doing it? How much better can you eat? What could you buy that you can't already afford?
Re:make all wall street traders own stock for 1 da (Score:3, Insightful)
let's do away with the arbitrage, gambling, and bullshit from wall street. make them own a stock for ONE WHOLE DAY. No more of this low-latency trading bullshit. They're just skimming money off the top of the financial markets--away from the regular folks. EVERY PENNY THESE GUYS MAKE COMES OUT OF OUR POCKET.
You are absolutely, painfully right and it pains me deeply that I don't have any mod points.
These jokers want time more accurate than NTP can provide but I'll bet most of them still believe time is a universal invariant. They don't need time more accurate than NTP can provide, they need to be hit in the head with a hammer until they stop moving.
Re:make all wall street traders own stock for 1 da (Score:3, Insightful)
Hint: When the Great Depression hit, the money lost by the traders on Wall Street did not suddenly spew forth into the hands of "us".
Because the money you mention never really existed.
If I had a thousand dollars in paper money under my bed before the great depression it would have had a certain buying power. In the middle of the great depression it would have had a much greater buying power. The value lost on Wall Street spewed forth into paper money and real physical assets.
Re:Speed of light and relativity (Score:5, Insightful)
I don't understand the need for wide-area highly accurate (less than 1ms accurate) timing. Once you get beyond a reasonable distance, the speed of light starts playing into the equation.
Like most smart people you are forgetting how dumb most people are.
Wall street traders don't understand much about time. They only understand how to obfuscate financial transactions like a high speed shell game. To them more speed means more chances to skim off a percentage and the laws of physics don't even get considered.
Re:make all wall street traders own stock for 1 da (Score:4, Insightful)
While I think that's a bit extreme, I think you and the GP are both on to the right track. I think stocks should have a fairly short minimum hold time. A day would probably be fine, just something such that you aren't trying to gamble with them more or less, because that's why high speed day trading really is. It is gambling in every way that Vegas is.
I also thing a sliding tax would make sense. If you execute quick trades, you are subjected to a fairly high tax. 95% is a bit excessive, but something still fairly high. Then, as you say, the tax drops on a monthly basis until it hits zero. That would act to encourage investing, rather than gambling. It rewards people who put their money away for the future, and do so in a manner that allows it to help the economy grow. It would also still allow for quicker trades, but the government gets to take a cut in that case.
I really do think we need to start looking at something like this for three reasons:
1) Because there are companies that try to game the system. That's never a good thing. You game the system on a large scale, you'll break it. When you are talking about something highly important to the economy, that's not acceptable.
2) Because there are too many gamblers. There are plenty of traders who basically gamble on the market. They make day trades not based on anything about the company, but just to try and grab money from trends. This is not useful, and isn't what the stock market is for. It leads to unnecessary instability and thus should be discouraged.
3) To help combat people's fear response. A non-trivial part of the big drop in the stock market last year was people being fearful. They panicked because of the economic troubles and wanted all their money out. That means even if the fund managers thought it was a bad idea, they had no choice. You have to do what your client wants. That of course lead to more drops and more panic and so on. Perhaps tax incentives could help stave that off with people.
Mod parent down. (Score:5, Insightful)
Actually this kind of trading (especially arbitrage) narrows the spread making it cheaper for ordinary people to enter the market.
So if you ban it you go back to only the big boys being able to afford to execute trades.
However, using any system, high frequency or otherwise, to deliberately manipulate the market is a different thing.
I'm sorry I'm posting as AC but it could affect my job.
Re:Speed of light and relativity (Score:3, Insightful)
I vote that all traders - and lawyers and estate agents for that matter - are sent such a reasonable distance from the planet that the speed of light indeed starts playing into the equation.
ALl in the name of research, of course.
Re:Ridiculous (Score:1, Insightful)
Is there a problem with rampant abuse of deleting fresh files? Is there a problem with rampant abuse of returning products shortly after buying them (as opposed to one day later)?
There is a problem with algorithmic stock trading in millisecond intervals, which allows a few traders to game the market instead of participating on even ground. Even a most die-hard market liberal must agree that the market should work to eliminate arbitrage, not create it.
Re:OK, but... (Score:4, Insightful)
You don't need accurate timing for that, you just need to be first.
Re:make all wall street traders own stock for 1 da (Score:2, Insightful)
Easy Answer: (in most tax laws)
Any arangement who's primary purpose is to avoid taxation is illegal, and will be charged tax at the rate at which it would normally have been taxed.
Re:make all wall street traders own stock for 1 da (Score:1, Insightful)
I'm not greedy because money can't buy really cool things like immortality or space trips to other star systems.
More useless stuff is all we can buy with money.
Re:Mod parent down. (Score:5, Insightful)
Mod parent up. The efficiency and liquidity of world wide stock markets increases their usefulness to all involved. My ability as an individual investor to buy and sell at will is a huge factor in my willingness to participate.
My father used to pay $100 to get a simple trade executed, and it would take hours to get it done. I can do the same trade for $5.00 and get it executed in a second or less.
So what if some financial institution is making a profit on this. It is a win-win situation.
The problems in the system are elsewhere - where people in the system are taking advantage of their position to trade based on information not available to everyone. Fortunately most of that is illegal.
Re:make all wall street traders own stock for 1 da (Score:4, Insightful)
Parent characterises the complete detachment from reality that characterises the modern "innovative" financial industry.
I operate a small business. I don't give a fsck about your claims that we need "liquidity" when I can't get a simple business loan for less than 8-9%, despite being well-capitalised, while my personal savings attract all of r pct, where 0.01 < r < 0.1. Quite the scam there. As a result, I use only retained earnings to pay for anything, which while a prudent thing to do given my situation, is hardly a recipe for large-scale economic growth. The financial industry does not fulfil its most basic imperative of funnelling capital to productive use, why should I care about whether some institutions can hold a stock for 1ms or 1 day?
I suppose things could be worse. Maybe in addition to being completely starved for capital, I may be subject to "protection" payments in the near future just to be able to stay in business at all! Financial innovation, 'Ndrangheta style! Wouldn't that be a boon for Wall Street and the City of London.
Re:Ridiculous (Score:3, Insightful)
Arbitrage trading is a possible application for very accurate timing in financial markets. That is the mechanism that eliminates arbitrage & it creates more liquidity in the markets, as well as more stable & accurate prices.
By imposing a minimum time limit to hold onto stock, you'd prevent a lot of arbitrage trading, which would create more arbitrage. It'd greatly reduce the liquidity in the markets too, making it harder for everybody to buy & sell.
Re:make all wall street traders own stock for 1 da (Score:5, Insightful)
I think people outside of the industry don't really understand the need for day traders.
I write trading software for the futures and options industries (I was the lead architect for the CBOT's Order Routing System, and have written trading systems for other exchanges and banks). The type of day trading that you and the GP have issues with is an absolute requirement for liquid markets and the efficient working of those markets.
Let's say you are a firm that needs to buy a large number of shares or contracts (in the futures world, you buy contracts for the future delivery of product). In the type of buy and hold market that seems "fair", that large order is going to go into a market of other buy and hold traders that are only going to trade out of their positions if the trade makes sense in their long-term strategy. Huge effective tax rates are going to make that trade unprofitable unless you've held the position a very long time (increasing your risk of market volatility) or you charge a premium (increase the price) in order to offset the tax hit.
This type of "illiquid" market raises prices dramatically. You have to factor in the time value of money, the risk exposure of holding shares for a long time (increasing the likelihood of a bad market event) and tax overhead. All that means that buying shares/contracts is riskier and selling them is more expensive.
Day trading fixes this. Day traders don't really care about the long-term direction of the market - they make money on minor intra-day price fluctuations. Because of this, they are nearly always willing to take on and shed positions. And because there are a lot of them chasing the same tiny fluctuations, they "shrink the spread" - they're going to give you a very good price because they shrink the difference between the bid and offer (price to buy/price to sell).
Now, when that original firm wants to buy that large number of shares, they are doing that in a market where there is almost always a large number of shares being offered at competitive prices. The day traders will sell off shares and pick up shares all day, just trying to end the day flat (holding nothing) to reduce after market risk.
Day trading does extract a price (they are making money) on the market, but the efficiency gains that they bring outweighs that price in almost all markets. They distribute a lot of the risk involved in trading among a bunch of smaller firms, rather than concentrating all the risk in big funds. This is generally a good thing - the government and the market in general are better off when a small firm goes bust than when one of the "too big to fail" firms starts having problems.
Now, the rise of algorithmic trading is causing some stress in this pattern - I can see some real issues coming in the next few years as these algorithms become more and more precise. I'm less concerned about a rogue algorithm dragging a market down (or up) than of all the algos converging to a point where the market gets stuck because the algos are just too damn efficient. But that's a post for another day...
The decline of /. (Score:3, Insightful)
NTP is accurate to within 10 milliseconds in any decent connection. Get a really good, stable, low latency connection and hook yourself up to a stratum 1 server and you can cut that number down tenfold ...
Can anyone explain what kind of financial application would require bigger accuracy?
Where have all the nerds gone?
Why are we debating whether or not the people who think they need highly accurate time really need it? What's cool about this article is the nerdly reaching for every higher degrees of accuracy, of digging deeply into a seriously difficult technical challenge and achieving new levels. What's cool for those of us who don't work on super-accurate timescales is the opportunity to read about how those who do think about this stuff, to see what math they use and what the issues are.
I don't mind a little discussion of social implications of technology, but it'd be nice to see at least a little discussion of the technology itself.
NTP issues -- really only one major issue (Score:4, Insightful)
The only really major issue with NTP is figuring out the time offset due to network latency in asymmetric network environments. The NTP protocol itself can frequency lock down to a few ppm over the internet without any problem at all. Offset errors are another matter. Over the internet 10 ms is about the best you can do. Over a LAN with a local GPS signal driving the protocol you can reduce the offset error to less than a microsecond and can probably get it down to less than 100ns without too much trouble (and without needing any hardware time stamping or netif queue prioritization), and you can frequency lock down to a few nanoseconds.
Other issues include the fact that most motherboards do not have temperature compensated timebases, and they tend to float around a few ppm (for the better MBs) to a few hundred ppm (for the stupid MBs that put crystals next to heat sources), as well as bugs in the operating systems themselves which don't get noticed until you actually try to frequency lock your timebase. This creates a multiplicative effect if you are not connected to a stratum 1 or 2 server because each server in the chain is trying to do a frequency lock against a drifting MB timebase. The result is the leaf nodes in the chain can get frequency locks but the locks are to a drifted frequency instead of to the correct frequency because the server itself can't instantly correct for its drifting timebase. This is why you see the frequency lock jump around by +/- 50ppm in a seemingly uncontrollable fashion once you get past stratum 2.
NTPD was famous for not being able to frequency lock while doing an offset correction at the same time. The NTPD program has also had many issues over years, such as not using a proper dual-staggered linear regression to control offset and frequency corrections, which is why I wrote dntpd for DragonFly. I wonder how many of these old issues with NTPD have been fixed over the years, maybe they have.
Network jitter is largely irrelevant. It isn't a problem if you use a proper linear regression over a long enough period of time. The linear regression can be used to calculate what the jitter is and thus form a good knowledge of the baseline accuracy of the protocol. A medium sized run over a few minutes will get you down to 10-15 ppm on your frequency lock and you can get down to 1-5 ppm within about 10 minutes (assuming the motherboard has an accurate timebase of its own).
The sad thing about all of this is that it takes just a single resistor to temperature compensate a crystal, a resistor most MB manufacturers don't put in. Some don't even use crystals anymore for their PLL input (and the crystal on the RTC doesn't help much because the RTC doesn't have a fine enough grain timer to poll easily). Sigh.
-Matt
Re:make all wall street traders own stock for 1 da (Score:3, Insightful)
Examine the concept of "god" and you'll see the answer.
"god" does not exist in reality ; the concept exists perfectly well, but in physical reality there ain't no such puppy ; however even an convinced atheist can't deny that the non-existent entity "god" has real effects on life, the universe and 42. Richard Dawkins, amongst others, makes a steady income selling books pointing out how much damage this concept has caused.
The power of the non-existent thing "god" to have effects in the real world comes from the fact that some people think that "god" exists, and this changes their behaviour.
Similarly, the belief of market people, economists etc in the existence of "money" affects their behaviour and their behaviour has real-world consequences.