Robust Timing Over the Internet 178
ChelleChelle writes "The NTP (Network Time Protocol) system for synchronizing computer clocks has been around for decades and has worked well for most general-purpose timing uses. However, new developments, such as the increasingly precise timing demands of the finance industry, are driving the need for a more precise and reliable network timing system. Julien Ridoux and Darryl Veitch from the University of Melbourne are working on such a system as part of the Radclock Project. In this article they share some of their expertise on synchronizing network clocks. The authors tackle the key challenge — taming delay variability — and provide useful guidelines for designing robust network timing algorithms."
make all wall street traders own stock for 1 day (Score:4, Insightful)
let's do away with the arbitrage, gambling, and bullshit from wall street. make them own a stock for ONE WHOLE DAY. No more of this low-latency trading bullshit. They're just skimming money off the top of the financial markets--away from the regular folks. EVERY PENNY THESE GUYS MAKE COMES OUT OF OUR POCKET.
Re: (Score:3, Insightful)
EVERY PENNY THESE GUYS MAKE COMES OUT OF OUR POCKET.
I see you have not the foggiest idea how "investments" and "fluidity" make the economy work.
Hint: When the Great Depression hit, the money lost by the traders on Wall Street did not suddenly spew forth into the hands of "us".
Re: (Score:3, Insightful)
Hint: When the Great Depression hit, the money lost by the traders on Wall Street did not suddenly spew forth into the hands of "us".
Because the money you mention never really existed.
If I had a thousand dollars in paper money under my bed before the great depression it would have had a certain buying power. In the middle of the great depression it would have had a much greater buying power. The value lost on Wall Street spewed forth into paper money and real physical assets.
Re: (Score:2)
Because the money you mention never really existed.
If something doesn't exist, how come it has an effect, either good or bad, on the real world of goods and services?
Re: (Score:3, Insightful)
Examine the concept of "god" and you'll see the answer.
"god" does not exist in reality ; the concept exists perfectly well, but in physical reality there ain't no such puppy ; however even an convinced atheist can't deny that the non-existent entity "god" has real effects on life, the universe and 42. Richard Dawkins, amongst others, makes a steady income selling books pointing out how much dam
Re: (Score:2)
It's a sort of shared dream (or lately nightmare) that fools refuse to wake from. Consider, before the bubble popped and after, we had the same people available to work, the same consumer demands and the same needs and transportation channels. The bubble popping didn't change the consumer demand, transportation channels, need, availability to work, or any other factor that should have any effect on the economy, but suddenly it's in the dumper.
It's as if we have put 5 year olds in charge of the country and n
Re: (Score:3, Interesting)
let's do away with the arbitrage, gambling, and bullshit from wall street. make them own a stock for ONE WHOLE DAY. No more of this low-latency trading bullshit.
A Capital Gains Tax that started at 95% and decreased ~2.5% per month would go a long way towards fixing many problems (/avoiding them in the first place).
Re:make all wall street traders own stock for 1 da (Score:4, Insightful)
While I think that's a bit extreme, I think you and the GP are both on to the right track. I think stocks should have a fairly short minimum hold time. A day would probably be fine, just something such that you aren't trying to gamble with them more or less, because that's why high speed day trading really is. It is gambling in every way that Vegas is.
I also thing a sliding tax would make sense. If you execute quick trades, you are subjected to a fairly high tax. 95% is a bit excessive, but something still fairly high. Then, as you say, the tax drops on a monthly basis until it hits zero. That would act to encourage investing, rather than gambling. It rewards people who put their money away for the future, and do so in a manner that allows it to help the economy grow. It would also still allow for quicker trades, but the government gets to take a cut in that case.
I really do think we need to start looking at something like this for three reasons:
1) Because there are companies that try to game the system. That's never a good thing. You game the system on a large scale, you'll break it. When you are talking about something highly important to the economy, that's not acceptable.
2) Because there are too many gamblers. There are plenty of traders who basically gamble on the market. They make day trades not based on anything about the company, but just to try and grab money from trends. This is not useful, and isn't what the stock market is for. It leads to unnecessary instability and thus should be discouraged.
3) To help combat people's fear response. A non-trivial part of the big drop in the stock market last year was people being fearful. They panicked because of the economic troubles and wanted all their money out. That means even if the fund managers thought it was a bad idea, they had no choice. You have to do what your client wants. That of course lead to more drops and more panic and so on. Perhaps tax incentives could help stave that off with people.
Re: (Score:2)
What does timespan have to do with it? Surely if something is unethical to do over an hour, it's unethical to do over a day or a week, or six months. The pr
Re:make all wall street traders own stock for 1 da (Score:5, Insightful)
I think people outside of the industry don't really understand the need for day traders.
I write trading software for the futures and options industries (I was the lead architect for the CBOT's Order Routing System, and have written trading systems for other exchanges and banks). The type of day trading that you and the GP have issues with is an absolute requirement for liquid markets and the efficient working of those markets.
Let's say you are a firm that needs to buy a large number of shares or contracts (in the futures world, you buy contracts for the future delivery of product). In the type of buy and hold market that seems "fair", that large order is going to go into a market of other buy and hold traders that are only going to trade out of their positions if the trade makes sense in their long-term strategy. Huge effective tax rates are going to make that trade unprofitable unless you've held the position a very long time (increasing your risk of market volatility) or you charge a premium (increase the price) in order to offset the tax hit.
This type of "illiquid" market raises prices dramatically. You have to factor in the time value of money, the risk exposure of holding shares for a long time (increasing the likelihood of a bad market event) and tax overhead. All that means that buying shares/contracts is riskier and selling them is more expensive.
Day trading fixes this. Day traders don't really care about the long-term direction of the market - they make money on minor intra-day price fluctuations. Because of this, they are nearly always willing to take on and shed positions. And because there are a lot of them chasing the same tiny fluctuations, they "shrink the spread" - they're going to give you a very good price because they shrink the difference between the bid and offer (price to buy/price to sell).
Now, when that original firm wants to buy that large number of shares, they are doing that in a market where there is almost always a large number of shares being offered at competitive prices. The day traders will sell off shares and pick up shares all day, just trying to end the day flat (holding nothing) to reduce after market risk.
Day trading does extract a price (they are making money) on the market, but the efficiency gains that they bring outweighs that price in almost all markets. They distribute a lot of the risk involved in trading among a bunch of smaller firms, rather than concentrating all the risk in big funds. This is generally a good thing - the government and the market in general are better off when a small firm goes bust than when one of the "too big to fail" firms starts having problems.
Now, the rise of algorithmic trading is causing some stress in this pattern - I can see some real issues coming in the next few years as these algorithms become more and more precise. I'm less concerned about a rogue algorithm dragging a market down (or up) than of all the algos converging to a point where the market gets stuck because the algos are just too damn efficient. But that's a post for another day...
Re: (Score:2)
This is the first actually well-informed post in this thread.
Re: (Score:2)
Re: (Score:2)
Day trading fixes this. Day traders don't really care about the long-term direction of the market - they make money on minor intra-day price fluctuations. Because of this, they are nearly always willing to take on and shed positions. And because there are a lot of them chasing the same tiny fluctuations, they "shrink the spread" - they're going to give you a very good price because they shrink the difference between the bid and offer (price to buy/price to sell).
So basically, at the poker table of share trading, day traders are the designated suckers?
Re: (Score:2)
hmm, or is it more like there's a big high-stakes poker game going on (the long-term investors), but then there's a secondary game of bookies and punters (the day traders) betting on the outcome of each hand? And with the ability for the high rollers to borrow funds from the bookies, and/or buy each other's hands?
But then there are syndicates among the bookies (the hedge funds) which are themselves so big that they become high rollers in themselves... they have no stake in the main game but are looking to c
Re: (Score:2)
Or we could just dump that entire "market". I use scare quotes because the people in it do not have any capacity whatsoever to take delivery of the goods or to actually provide them on demand (other than to order someone else to send 'their' commodity to another destination). In fact, if even a tiny fraction of what they buy and sell were actually delivered to them, it would be an un-mitigated disaster.
Howsabout if company A needs a commodity they buy it from one or more producers of that commodity?
Our econ
Re: (Score:2)
let's do away with the arbitrage, gambling, and bullshit from wall street. make them own a stock for ONE WHOLE DAY. No more of this low-latency trading bullshit.
A Capital Gains Tax that started at 95% and decreased ~2.5% per month would go a long way towards fixing many problems (/avoiding them in the first place).
So you short against the box. (buy N shares and sell N shares short collaterallizing the type two account (the account that buys the stock) with the cash in the type 3 account (the account that is short the stock)
Worst case, you have to deliver the borrowed stock before 38 months have passed, and get hit with a tax bill when you unwind the trade in a private transaction between your two accounts.
I am sure that there are other workarounds.
Re: (Score:2, Insightful)
Easy Answer: (in most tax laws)
Any arangement who's primary purpose is to avoid taxation is illegal, and will be charged tax at the rate at which it would normally have been taxed.
Re: (Score:2)
Or they'd come up with a workaround where X holds the stock forever, and traders swap "X points".
Re:make all wall street traders own stock for 1 da (Score:3, Insightful)
Agreed. When are some of these people going to realize that greed is bad. Trying to game a system is inherently wrong. Not just morally, but wrong systemically. It will always result in harm to the system. These people soothe their conscience with excuses- "We're spreading the risk and that's a good thing", "It's only fractions of pennies off of many people, nobody will notice." , etc. Until the whole thing comes crashing down because in reality the financial system is core dependent on actual investment an
Re: (Score:3, Interesting)
Who the hell isn't greedy. If you were offered more money at work, would you turn it down. Most people wouldn't. And all your colleagues are going to get paid more.
This is all these people are doing, they are doing there jobs, and they get paid bonuses for doing it well.
It is ignorant and stupid to pretend that this situation is just that simple.
Re: (Score:2)
The offer for more money at work is part of the system. You provide a valuable service and they compensate you for it. They trust you to do useful labor, and you trust them to compensate you for it. It works because they make money off your labor and everybody is happy. If somebody was to write a program to take the .00001 cents rounded off from a company's transactions (a la Office Space) then they are trying to game the system. They are taking money without providing anything useful in return. It amounts
Re: (Score:2)
No, some of them knowingly set the company on the path to ruin in order to maximize their position. They don't care because they plan to deploy the golden parachute long before the crash.
Most people would accept a freely offered raise consequence free. Fewer would take the raise if they were told one of them would be fired to pay for it. Fewer still would demand the raise and suggest firing someone, particularly if they already had enough money to retire comfortably on.
So, no, most people aren't that greedy
Re: (Score:2)
Are you sure about that? If you are young right now, there's a good chance you will see some pretty amazing improvements in medicine and technology. It might not be enough to acheive immortality, right away, but perhaps the average lifespan will be increased dramatically enough you will.
Singularity is upon us! Personally, I don't want to gamble on it being free.
Re: (Score:3, Insightful)
let's do away with the arbitrage, gambling, and bullshit from wall street. make them own a stock for ONE WHOLE DAY. No more of this low-latency trading bullshit. They're just skimming money off the top of the financial markets--away from the regular folks. EVERY PENNY THESE GUYS MAKE COMES OUT OF OUR POCKET.
You are absolutely, painfully right and it pains me deeply that I don't have any mod points.
These jokers want time more accurate than NTP can provide but I'll bet most of them still believe time is a universal invariant. They don't need time more accurate than NTP can provide, they need to be hit in the head with a hammer until they stop moving.
Re: (Score:2, Interesting)
That's a ridiculous idea. It would increase volatility tremendously, increase spreads tremendously, increase mispricing tremendously, and of course destroy any semblance of liquidity for half the securities out there. Saying that arbitrageurs are skimming is like saying market makers are skimming. Partly true, but it completely ignores the positive effects of their actions for the market.
Re:make all wall street traders own stock for 1 da (Score:4, Insightful)
Parent characterises the complete detachment from reality that characterises the modern "innovative" financial industry.
I operate a small business. I don't give a fsck about your claims that we need "liquidity" when I can't get a simple business loan for less than 8-9%, despite being well-capitalised, while my personal savings attract all of r pct, where 0.01 < r < 0.1. Quite the scam there. As a result, I use only retained earnings to pay for anything, which while a prudent thing to do given my situation, is hardly a recipe for large-scale economic growth. The financial industry does not fulfil its most basic imperative of funnelling capital to productive use, why should I care about whether some institutions can hold a stock for 1ms or 1 day?
I suppose things could be worse. Maybe in addition to being completely starved for capital, I may be subject to "protection" payments in the near future just to be able to stay in business at all! Financial innovation, 'Ndrangheta style! Wouldn't that be a boon for Wall Street and the City of London.
Re: (Score:2)
Parent characterises the complete detachment from reality that characterises the modern "innovative" financial industry.
Actually the parent you refer to understands the difference between the equity market and the debt market. Imposing a restriction that stocks must be held for a day or other restrictions on equity trading would have costs that those proposing them don't understand. That is not directly connected to the consumer portion of debt markets you later refer to when you say you can't get a business loan at a rate you like or earn an interest rate you want. Actually since debt instruments can be bundled and sold lik
Re: (Score:2)
You are right, my rant did go astray and I realize that debt and equity are not the same thing. I am, however, trying to explain things from the perspective of a small business owner, you know, the little guy that the financial industry is walking all over.
If we reverted to a very old fashioned banking system, 3% on deposits, 6% loans, bankers on the golf course at 3pm, i would be better off. With all the "innovation" and technology giving us "liquidity" in the debt and equity markets, what do i see? My d
Mod parent down. (Score:5, Insightful)
Actually this kind of trading (especially arbitrage) narrows the spread making it cheaper for ordinary people to enter the market.
So if you ban it you go back to only the big boys being able to afford to execute trades.
However, using any system, high frequency or otherwise, to deliberately manipulate the market is a different thing.
I'm sorry I'm posting as AC but it could affect my job.
Re:Mod parent down. (Score:5, Insightful)
Mod parent up. The efficiency and liquidity of world wide stock markets increases their usefulness to all involved. My ability as an individual investor to buy and sell at will is a huge factor in my willingness to participate.
My father used to pay $100 to get a simple trade executed, and it would take hours to get it done. I can do the same trade for $5.00 and get it executed in a second or less.
So what if some financial institution is making a profit on this. It is a win-win situation.
The problems in the system are elsewhere - where people in the system are taking advantage of their position to trade based on information not available to everyone. Fortunately most of that is illegal.
Re: (Score:2)
Re: (Score:2, Insightful)
Phew that's OK then, it's only everyone with a fucking superannuation.
Re: (Score:2)
it only comes out of the pockets of those who regularly buy and sell.
Right; 'cos this being Slashdot most of us can hold of buying food for a month or two and just buy electronics. Seriously; that's the whole point of this. When the price of food falls; the financial guys involved in arbitrage know in advance of the people who are buying for supermarkets and they pocket part of the money which the supermarkets otherwise wouldn't have to pay the producers.
The only way to avoid this is to ensure that they pay a transaction cost for each transaction.
Re: (Score:2)
The only way to avoid this is to ensure that they pay a transaction cost for each transaction.
Pay it to whom?
Trust yourself... (Score:2)
I like the trust yourself part. I actually implemented a dirty script to restrict how much ntp can change my clock speed and I call every minute from crond ;-))
Limits are hardcoded and depend on the machine ;-))
$ cat setclockfrequency
#!/bin/sh
# 251616 = 3.839
# 269120 = 4.106
# 280448 = 4.279
# 288544 = 4.403
# 9372208 = 143.009
# 10703936 = 163.329
# 11308784 = 172.558
###let LOWLIMIT=243520
###let HILIMIT=286496
let LOWLIMIT=10072208
let HILIMIT=10408784
let FREQ=`adjtimex -p | grep frequency | cut -d ":" -f 2 | cut
Re: (Score:2)
Can you please explain what the issue is you are trying to solve with this?
Re:Trust yourself... (Score:5, Interesting)
Network issues. If say, on average, once every 25 polls, ntpd doesn't get realistic data from higher stratum servers because of the network, don't let ntpd crank up the frequency to ridiculous values like it does when this occurs.
There are realistic values for the frequency on every machine with a good clock. It is ridiculous to set the frequency below or above these values.
Last time I checked, there is no way in ntpd to configure these values. The typical ntp guru reply will be: "Get a decent network connection". The author in TFA noted that such a "decent connection" is virtually impossible to achieve because of "variable delay". He also noted that it is sometime better to trust yourself, which is kind of what that script does.
Re: (Score:3, Interesting)
Author of TFA also mention:
"More generally, the algorithm should be designed never to overreact to anything. Remember, its view of the world is always approximate and may be wrong, so why try to be too clever when inaction works so well? Unfortunately, feedback algorithms such as ntpd have more reactive strategies that drive the clock more strongly in the direction of their opinions. This is a major source of their nonrobustness to disruptive events."
I fully agree with this and I realized it several years a
Re: (Score:2)
I know very well how ntpd works. On the other hand, you do not understand what this script is for. Please read the other posts I have made on this tread for further understanding.
http://slashdot.org/comments.pl?sid=1638098&cid=32062158 [slashdot.org]
http://slashdot.org/comments.pl?sid=1638098&cid=32062314 [slashdot.org]
OK, but... (Score:2)
Re: (Score:2)
In my opinion the finance industry have to live with some minor differences in timing.
What this has really done now is to provide an idea of an opening for fraudulent behavior in the finance industry. Just imagine what would happen if someone was able to inject timing faults into the servers when their favorite transaction was executed.
Re: (Score:2)
Are you talking of the situation where the stock goes up and I buy it, but I frig it so it looks like I bought it at the old (lower) price?
The time doesn't matter, as long as the sequence is known. But I don't think the system works like that; quantity and price are always included, AFAIK, plus it wouldn't match the counterparty's books.
Re: (Score:2)
One example goes kinda like this
1. News breaks overnight that will make people want to buy company FOO stock.
2. The instant (as determined by the very accurate clocks on their servers) the market opens, trading firm BAR buys up all the available FOO stock and then instantly puts it back up for sale for a little bit more.
3. The rush of people buying the stock buy it from firm BAR.
Rinse and repeat every day and you're talking about billions of dollars made by firm BAR that people without ultra accurate clock
Re:OK, but... (Score:4, Insightful)
You don't need accurate timing for that, you just need to be first.
Re: (Score:2)
And you be first by jumping exactly when the market opens, which is why the accurate timing is needed.
Re: (Score:2)
You can trade at any time, provided you can find a counterparty to deal with.
Re: (Score:2)
GPS (Score:4, Interesting)
How about GPS receiver + exact known location?
I would think just eliminating an unknown latency and replacing it with a known one (radio waves generally travel at a consistent rate) would get you a pretty accurate time. If you add one more known variable which is the exact location of the device, then you should be able to get an even more accurate time.
Re: (Score:2)
Re: (Score:2)
the ionosphere can mess up GPS signals by about 100 nanoseconds
That's variable. You use the GPS time signal over a long period of time (days) to calculate the rate at which your clock drifts, at which point you can do a reasonably good correction for things and get your clock pretty accurate. Asking for perfect timing a second or two after switching the device on is just unfair!
Re: (Score:2)
GPS is okay. But I would think that a financial institution would prefer to invest in a good cesium-beam frequency standard. These units are calibrated at the factory and drift something like one second every few thousand years. Plus, there is no need to have an external antenna to pick up GPS signals. If you *absolutely* need stable and accurate clocking, cesium is the only way to go.
Re: (Score:2)
I believe that the core clocks used in GPS are cesium-beam. Still, if you happen to have a data center in a high-flying (lower gravity well) or fast moving (probably Mach 2 or higher), then you will need to re-sync with GPS occasionally.
Re: (Score:2)
The clocks in orbit are either Rubidium or Cesium based. The US Primary Time Standard is a Cesium Fountain.
http://tf.nist.gov/cesium/fountain.htm [nist.gov]
The GPS clocks are synchronized to the Master Clock.
http://tycho.usno.navy.mil/clocks.html [navy.mil]
http://tycho.usno.navy.mil/mc_to.html [navy.mil]
Re: (Score:2)
Re: (Score:2)
While radio waves do travel a constant speed, what they don't do is travel a consistent path between the satellite and the receiver. Some of this is due to the satellite's orbital motion, and the timing errors resulting from that can be eliminated. However, the path is also altered
Would make sense (Score:5, Informative)
GPS is accurate to about 50 nanoseconds. All kinds of devices that need precision time get it from GPS. You don't need much more than a standard receiver, just one that is designed to place a high priority on time updates. The GPS system itself keeps very accurate time since each satellite has an atomic clock, and they all sync to the master clock.
To me that would seem the best way, if accuracy is really important and the systems are high end. Have each device have its own receiver and just have them sync to that. Don't sync them to each other, since you aren't going to get anything more accurate.
Re: (Score:2)
> How about GPS receiver + exact known location ?
But if you roll-your-own do not forget that GPS time does not have the leap seconds that UTC has !!
Re: (Score:2)
Re: (Score:2)
I thought the Data Signal, repeated every 15 minutes roughly contained date/time and clock corrections? You might be drifting for 15 minutes at a time I guess...
Re: (Score:2)
The GPS satellites all have synchronized atomic clocks on board. If your GPS receiver also had an atomic clock, then the distance to each satellite could be computed by subtracting the times and multiplying by the speed of light. So, with 3 satellites, you could calculate your location.
However, GPS receivers are cheap, and don't have atomic clocks. But if you get a signal from *four* satellites, then you can use the redundant information to deduce the correct time, and then use that time to find your loc
can't trust self if microsoft (Score:5, Interesting)
Re:can't trust self if microsoft (Score:4, Funny)
I don't really see a problem with that. You wouldn't use Notepad for a mission-critical editing, why would you use W32Time for mission-critical timing?
That being said, I don't know what 3rd-party system you would use for sub-second timing.
Re: (Score:2)
Time synchronisation is crucial to Active Directory security (since it uses kerberos). In fact, I think a major part of joining windows desktops to a windows server/domain is that they all start syncing time.
Re:can't trust self if microsoft (Score:5, Informative)
The problem isnt "windows"
Accurate timing on the modern PC is ridiculously difficult due to the hardware situation. The original IBM PC and its clones came with Intel's 8253 or 8254 Programmable Interval Timer (PIT) chip and back then it was pretty good. This timer chip had an internal frequency matching the CPU's of the time of 4.77mhz, and ticks were evenly distributed between 4 internal counters, so each counter had 1193181.81818181.... ticks per second. Pretty damn good.
Many of the old-school people might be familiar with the 18.2 ticks per second of the DOS clock. This rate was a direct consequence of configuring one of those counters to emit an interrupt every 65536 ticks, the longest possible interrupt interval due to these chips using 16-bit words.
Anyways, over time many manufacturers stopped using Intel's PIT's and started emulating them with other hardware (such as PMT.) None of these emulations can be configured to tick at exactly the same rate as the Intel PIT, and even among these emulated timers there wasn't any consistency.
To counter the serious problem of accurate timing on the PC, Microsoft and a few other large companies moved to define a new standard named HPET. Many new motherboards will have an HPET timer, but not all of them, and there are some problems with some HPET implementations as well (such as appearing to tick backwards sometimes.)
The issue of accurate timing on PC's has considerable effect on multi-player game developers.
A reference to some of the issues, and some of the attempts to fix them (source code) [gamedev.net]
Re: (Score:2)
Having worked with a county sheriff's office and their 911 system, I can tell you that it is possible to regulate the Windows system clock quite well, as they require it. It just isn't "high frequency trading" quality.
But I doubt you'd use the Linux kernel time for high frequency trading either. You'd probably opt for something specifically designed for just that task, and designed to do it particularly well.
Re: (Score:3, Interesting)
Addendum: I thought I'd add that they use hardware GPS modules to receive the current time, as NTP is considered not reliable enough for them.
PTPd? (Score:5, Interesting)
I'm surprised the article didn't mention PTPd [wikipedia.org], which is an implementation of the IEEE 1588 precision time-synchronization standard. I was under the impression that was the standard way to solve this sort of problem when NTP wasn't enough.
Re:PTPd? (Score:5, Informative)
Re: (Score:2)
Thanks for the informative post that was on topic. One question: with 1588 what sort of hierarchy do you set up? Does everyone have a rubidium or cesium clock attached?
Re: (Score:2)
One question: with 1588 what sort of hierarchy do you set up? Does everyone have a rubidium or cesium clock attached?
A typical way is to use a GPS GrandMaster clock that outputs PTP. Some of these have good oscillators in them to keep good time if GPS dies for whatever reason. The tricky part is getting this time to many machines without bad drift.
Re: (Score:2)
Maybe someone should adjust its clock? ;)
Wants vs. needs vs. design (Score:2, Interesting)
It's not clear to me the financial system needs such high precision timing.
My cynicism tells me they need it now to implement fraudulent micro-second trading, where see what you're buying, correlate with other people's buys and then buy a fraction of a second ahead of you.
At any rate, it's not clear they need it, or that giving it to them won't lead to instability.
Maybe someone should actually design a financial system, and design one that can use imprecise, or precision timing.
Re: (Score:2)
If you're talking about "front running", you don't need accurate timing for that. You (the broker) just slip your order in front of the customer's.
Re: (Score:2)
The purpose of this degree of accuracy is to measure the performance of your system.
Here I am, team leader for Goldman Sacks Front-Runner IT services division and I'm trying to squeeze a couple of microseconds out of my trading network. Well I can't do that unless I can measure the performance of my distributed application very accurately.
Re: (Score:2)
It's not clear to me the financial system needs such high precision timing.
A lot of it has to do with analytics and making sure your system is operating properly. Being able to compute network and software stack latencies often requires the clocks of multiple machines being very closely synced. NTP can't do it well enough in many cases. Without good synchronization, you end up (according to timestamps) receiving a packet before it was sent.
Re: (Score:2)
Just add jitter
Say 'your orders will be processed with a delay varying between 0 and 5s'
Ridiculous (Score:3, Interesting)
NTP is accurate to within 10 milliseconds in any decent connection. Get a really good, stable, low latency connection and hook yourself up to a stratum 1 server and you can cut that number down tenfold ...
Can anyone explain what kind of financial application would require bigger accuracy?
I understand that certain scientific experiments require more accuracy than that, but those guys are probably already hooked up to stratum 0 servers directly.
Why on earth would any financial system (yes, even the stock market) require 10ms time accuracy? They were able to rip us off with fucking sand clocks. NTP should be enough for them.
Re:Ridiculous (Score:5, Informative)
This sort of thing probably has something to do with it:
http://www.nytimes.com/2009/07/24/business/24trading.html [nytimes.com]
Re: (Score:3, Insightful)
Its old fashioned day trading accelerated by a factor of a few million. Of course, you need to back it up with a good AI because humans can't react fast enough and a flatline with his ass in a can will react in a predictable way.
Re:Ridiculous (Score:4, Informative)
Re: (Score:3, Informative)
These guys are probably trying to do sub-millisecond trading. They need to keep their clocks sync'd to the microsecond to keep different machines coordinated.
Me, I wonder why they're bothering. Run GPS-based stratum-1 servers at each site, and sync directly to them across the LAN. Or get a cesium-beam primary time source. End of problem. Or take heed of the current financial meltdown and acknowledge that if you try to create money out of thin air, sooner or later you're going to pay the piper and it's going
Re: (Score:2)
I fully support the rule that you can't sell stock when you bought the same stock less than a day earlier, unless the current stock price is more than 10% lower than when you last bought.
Let's have the government regulate everything. You can't return a product to the store that you bought less than 1 day earlier. You can't delete a file on your computer that you created less than 1 day earlier.
Re: (Score:3, Insightful)
Arbitrage trading is a possible application for very accurate timing in financial markets. That is the mechanism that eliminates arbitrage & it creates more liquidity in the markets, as well as more stable & accurate prices.
By imposing a minimum time limit to hold onto stock, you'd prevent a lot of arbitrage trading, which would create more arbitrage. It'd greatly reduce the liquidity in the markets too, making it harder for everybody to buy & sell.
Re: (Score:2)
How about simply making the hold requirement the minimum average length of time it would take an average off the street human to immediately buy and then sell the stock?
Have the SEC round up random samples of people and have them try to buy and then sell stocks as fast as possible on each broker's site. Take the average of that and make that the hold requirement.
The decline of /. (Score:3, Insightful)
NTP is accurate to within 10 milliseconds in any decent connection. Get a really good, stable, low latency connection and hook yourself up to a stratum 1 server and you can cut that number down tenfold ...
Can anyone explain what kind of financial application would require bigger accuracy?
Where have all the nerds gone?
Why are we debating whether or not the people who think they need highly accurate time really need it? What's cool about this article is the nerdly reaching for every higher degrees of accuracy, of digging deeply into a seriously difficult technical challenge and achieving new levels. What's cool for those of us who don't work on super-accurate timescales is the opportunity to read about how those who do think about this stuff, to see what math they use and what the issues
Re: (Score:2)
I don't think this is part of the decline of /.
The trolls, the vacuous m$/apple/OS/FS/whatever mindless bashing/fanboys, the crappy editors, and other stuff like that is what's causing the decline of Slashdot.
We are in the year 2010. Thinking just about the technical aspects isn't good enough anymore. What we have learn from RMS (Whether you like him or not) is that hacktivism is very important, if we want to continue hacking happily. Our world is at danger. We are heading towards an Orwellian future of tec
Re: (Score:2)
Thinking just about the technical aspects isn't good enough anymore.
Clearly. Did you read the last sentence of my post?
My point is that there was no discussion of the technical aspects.
Re: (Score:2)
Carriers with shorter fibre optic paths have been able to command price premiums upwards of $500/month per millisecond of decreased latency vs. their competition. Eventually the exchanges will just host trading houses' servers locally, everyone will be on a level playing field, and this fiasco will end.
You have to admit, the end state of this system will be an interesting variation of "Core Wars", at least for those who have the money and skill to participate.
Off topic I know but.... (Score:2)
... is there a patch or driver hack for Windows that will allow the hardware clock to be set to GMT/UTC while the time zone offset is handled by the OS as it should be? It is such a pain in my ass that I run Linux with the occasional boot to Windows and having the time wrong for Windows. And of course if I set it, the time becomes wrong for Linux. Linux is doing it "right" and Windows is doing it wrong in my opinion. But what is the fix? I'd prefer Windows change its ways, not Linux.
Re: (Score:2)
I run Linux often enough that I don't want to change it and regard its behavior as correct. I regard windows as incorrect. And you're right. I don't run the Windows that much -- just when I VPN into the office which is somewhat rare.
I don't usually "need" the time to be right... it's just annoying when it's not.
Speed of light and relativity (Score:2)
Re:Speed of light and relativity (Score:5, Insightful)
I don't understand the need for wide-area highly accurate (less than 1ms accurate) timing. Once you get beyond a reasonable distance, the speed of light starts playing into the equation.
Like most smart people you are forgetting how dumb most people are.
Wall street traders don't understand much about time. They only understand how to obfuscate financial transactions like a high speed shell game. To them more speed means more chances to skim off a percentage and the laws of physics don't even get considered.
Re: (Score:2)
Uh? I've read here that some traders are paying to put their hardware the closest they can to the trading servers so that they can have low latency..
So I'd guess that this is more a case of an article with a flawed example than anything else..
If investors really wanted to take risks (Score:2)
Wouldn't they just back a company they knew, trusted, and had some reason to believe in, and stay with that company throughout its life? Wearing the damage if that enterprise fails? (Because everyone else will - a failed enterprise is a failed enterprise, especially if it causes social and environmental damage on the way down.)
Why does supporting causes you believe in require liquidity? Why the need to jump in and out of positions in the first place? It takes years for companies to gear up infrastructure -
Re: (Score:3, Insightful)
I vote that all traders - and lawyers and estate agents for that matter - are sent such a reasonable distance from the planet that the speed of light indeed starts playing into the equation.
ALl in the name of research, of course.
Works fine in telecommunications (Score:2, Interesting)
Hi!
Last few years, most cellular operators are rolling out IP to the base stations.
For HSPA/LTE speeds, E1/T1 PDH/ATM is out of the question,
and as IP is rolled out, GSM gets there as well.
Accurate timing is essential: even transmitter frequency is synchronized over IP,
and keeping 2 GHz accurate to 100 Hz is not simple.
Local oscillator can do it short term, but external correction is needed as well.
Beside GPS (used in IS-95), several systems are used.
IEEE 1588v2 is obvious one. Works.
Some vendors do it with
Don't diss NTP! (Score:3, Informative)
The authors of the RADclock article don't seem to realize that most of the issues they "solve" with their algorithm has been considered, and also more or less solved, by the NTP hackers severeal years ago.
I.e. when the article shows that in a variable-latency environment you need to filter the measurements by the minimum rtt: This is of course default behaviour for ntp as well, but only over a limited number of polling intervals (8). If this isn't sufficient then there are a number of configuration options available, like the huff-puff filter which can be configured to maintain rtt history long enough to "coast" past any expected periods of large and asymmetrical network delays.
They also give out some _really_ bad advice, like claiming that "One server is enough": This is only true if you know that this particular server can never be wrong!
If you want to survive the situation where a single servers starts to lie, you shold configure at least 4 independent sources.
Personally I have seen 3 different gps-based stratum-1 black box ntp servers go crazy, suddenly returning timestamps far into the future.
OTOH, my freebsd-based servers with Motorola Oncore UT+ timing optimized gps firmware has never returned bad time, but I still have 6 such servers spread around our corporate network.
Terje
NTP issues -- really only one major issue (Score:4, Insightful)
The only really major issue with NTP is figuring out the time offset due to network latency in asymmetric network environments. The NTP protocol itself can frequency lock down to a few ppm over the internet without any problem at all. Offset errors are another matter. Over the internet 10 ms is about the best you can do. Over a LAN with a local GPS signal driving the protocol you can reduce the offset error to less than a microsecond and can probably get it down to less than 100ns without too much trouble (and without needing any hardware time stamping or netif queue prioritization), and you can frequency lock down to a few nanoseconds.
Other issues include the fact that most motherboards do not have temperature compensated timebases, and they tend to float around a few ppm (for the better MBs) to a few hundred ppm (for the stupid MBs that put crystals next to heat sources), as well as bugs in the operating systems themselves which don't get noticed until you actually try to frequency lock your timebase. This creates a multiplicative effect if you are not connected to a stratum 1 or 2 server because each server in the chain is trying to do a frequency lock against a drifting MB timebase. The result is the leaf nodes in the chain can get frequency locks but the locks are to a drifted frequency instead of to the correct frequency because the server itself can't instantly correct for its drifting timebase. This is why you see the frequency lock jump around by +/- 50ppm in a seemingly uncontrollable fashion once you get past stratum 2.
NTPD was famous for not being able to frequency lock while doing an offset correction at the same time. The NTPD program has also had many issues over years, such as not using a proper dual-staggered linear regression to control offset and frequency corrections, which is why I wrote dntpd for DragonFly. I wonder how many of these old issues with NTPD have been fixed over the years, maybe they have.
Network jitter is largely irrelevant. It isn't a problem if you use a proper linear regression over a long enough period of time. The linear regression can be used to calculate what the jitter is and thus form a good knowledge of the baseline accuracy of the protocol. A medium sized run over a few minutes will get you down to 10-15 ppm on your frequency lock and you can get down to 1-5 ppm within about 10 minutes (assuming the motherboard has an accurate timebase of its own).
The sad thing about all of this is that it takes just a single resistor to temperature compensate a crystal, a resistor most MB manufacturers don't put in. Some don't even use crystals anymore for their PLL input (and the crystal on the RTC doesn't help much because the RTC doesn't have a fine enough grain timer to poll easily). Sigh.
-Matt
This is still NTP (Score:2, Informative)
Although the summary and the article itself seem to take pains not to mention it, a visit to the RADclock homepage (http://www.cubinlab.ee.unimelb.edu.au/radclock/) will tell you that what's actually being offered here is an improved NTP client. No changes to the NTP servers, server software or NTP protocol are required or are proposed. The client improvements are in an improved filter topology (feed forward with quality assessment) and introduction of separate concepts of absolute and difference clocks opt
Re: (Score:2)
Running antenna cable to the servers requiring synchronization is often a problem.