Researchers Warn Computer Clocks Can Be Easily Scrambled Via NTP Flaws (networkworld.com) 132
alphadogg writes: Researchers at Boston University said this week that they've found flaws in the Network Time Protocol (NTP), a 30-year-old Internet protocol whose security shortcomings could undermine encrypted communications and even jam up bitcoin transactions. The importance of NTP was highlighted in a 2012 incident in which two servers run by the U.S. Navy rolled back their clocks 12 years, deciding it was the year 2000. Computers that checked in with the Navy's servers and adjusted their clocks accordingly had a variety of problems with their phones systems, routers and authentication systems.
There is at least one alternative out there, and reason to use it.
Bitcoin! (Score:1, Funny)
Oh boy not Bitcoin! Save us!
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Most NTP clients I've seen... (Score:5, Insightful)
have an option for ignoring server updates if the time differential is too great.
Re:Most users I've seen... (Score:4, Insightful)
Re:Most NTP clients I've seen... (Score:5, Insightful)
Not only that, but there are loads of other options, the most basic being that you can use your own personal stratum 1 device [probroadcastsupply.com], and point all your servers to that. Hell, you can even build your own [satsignal.eu] if you're truly paranoid.
IIRC by default the Windows AD infrastructure has member clients/servers using the Domain Controllers as their local source, and then you only need to re-point the DCs to whatever you want (the registry info documentation for NTP was a bit hinkey as late as Win7/Win2k8, so if you go mucking around in there, do it at your own risk). On the Linux/BSD/*nix side, there's a zillion options to beef up security, and they are drop-easy to enforce with any competent orchestration software (puppet, cfengine, chef, whatever). I only remember the Windows side because a previous employer had a distributed BI system that demanded that all component client devices (a mixture of Windows and Linux) must remain within 5 seconds of each other time-wise (else the whole thing threw an error and stopped).
Like sibling said though, most sysadmins don't dork around with NTP once they get a time source running - few will set up a local NTP relay of sorts, fewer still will have those sources use at least three different and vastly disparate sources to check against, and very few will set up a local authoritative stratum 1 box.
Lots of workarounds, and you really only have to set it up right once, with maybe an occasional (as in once-a-year-or-so) review and tuneup of the infrastructure.
Re: (Score:3)
An additional idea for a cheap personal stratum 1 device if you happen to live in the reception area of a suitable LW radioclock.
http://www.buzzard.me.uk/jonat... [buzzard.me.uk]
Had my own personal dual source (MSF/DF77) stratum 1 time server for over a decade now.
Re: (Score:2)
Re: Most NTP clients I've seen... (Score:2)
The idea that LW clocks lose 1s for every 300km is a laughable. I have two in operation one MSF and the other DCF77, the difference in the distances from me is well over 300km, more like 1000km. I have a decade of data showing them to be milliseconds apart. Sure GPS is better, but multiple sources are better still, and a lw radiology is better than some random time-served on the Internet; again I have a decade of data to prove that.
Re: (Score:2)
Hell, you can even build your own if you're truly paranoid.
It's a good idea, but the forty euro GPS shield can easily be replaced by a twenty euro GPS board, a NEO-6M. These are also favored for drones because they are teensy tiny and they are smart enough to store configuration settings such that they will wake up, acquire, and start spewing GPS data on their serial port at your chosen baudrate. They also feature a PPS pin. Buy one with a header strip on and you can connect it to the Pi without soldering. Plus, they are useful for other projects if you decide to r
Re:Most NTP clients I've seen... (Score:4, Informative)
Many devices such as tablets and cell phones contain GPS, another hard to spoof time source. Multi factor authentication checking NTP, Cell tower time, and GPS would make a very robust system that could detect tampering. On an IMSI catcher, Time wrong, GPS time difference would catch it. NTP spoof, Cell and GPS mismatch would catch it. Local area GPS spoof, Cell and NTP would catch it. Targeting a mobile device with 3 factor time authentication + internal clock for 4 factor would be very hard to spoof undetected.
If you want to roll your own, GPS receiver modules for RC drones are under $20 online. Adding GPS to your NTP corporate servers is not difficult. You can protect your network with a little hardware and software.
Re: (Score:2)
How is GPS a time source?
Re: (Score:2)
GPS Time [lammertbies.nl]
"The GPS system allows a maximum of 32 satellites around the earth which each transmit their own position and time on a regular interval to the earth."
Re: (Score:3)
Many GPS receivers will output a very accurate 1 sec tic. I've used this to check for multipath drift from WWV in maintaining a cesium beam atomic clock in one installation. The time standards were used to callibrate RF equipment to NIST standards. Multi source verification validates source drift and gives the degree of confidence for the certification paperwork. If your tracable standards are accurate to each other with jitter and drift to 8 9's, you can validate equipment as to being within 7 9's. Thi
Re: (Score:2)
Each GPS satellite transmits a time code along with its ID number. When the receiver is able to locate a signal from four or more satellites, it can determine the distances to those satellites. Some simple spherical trigonometry allows the latitude, longitude and altitude to be calculated for the receiver.
Re: (Score:2)
Adding a window to the server room, or running a GPS antenna cable through the cable trunking to the outside world is likely to be the harder part of the process. PARTICULARLY if you have anything resembling Management Of Change procedures for your physical cabling.
Re: (Score:1)
AC power is NOT precise. Install a logger and you'll see just how much it varies throughout the day.
Re: (Score:2)
In the short term they may seem all over the place, but the AC frequency is accurate when averaged over the entire day.
In the '60s, '70s, '80s the wall clocks in schools and other public places were simple AC motors. In the absence of a loss of power, those clocks were accurate to with a few seconds over a year. Even our old AC powered kitchen clock kept perfect time.
These days, you can't even trust a wall clock to be accurate to within a few minutes.
Re: (Score:2)
Besides which, doesn't IBM have a patent on inaccurate computer clocks? Tromp on that and you'll probably find them suing you for your car, home, savings, and first born son.
Re: (Score:2)
They can have my first born son! But they never get my Sun!
Re: (Score:2)
Re: (Score:1)
it's nontrivial in the first place, doing it repeatedly over a period of time would require a focused attacker. and nothing stops a focused attacker.
Re:Most NTP clients I've seen... (Score:4, Informative)
That's not how NTP is supposed to be used: It is designed to pull
time from multiple sources, weigh their accuracy/trustworthyness
over a longer time window against each other (with rather sophisticated
algorithms), and use the best ones as the time source to follow.
I usually go for 5-6 independent sources (with independent stratum0 masters!):
some national labs' PPS; GPS; etc. - this doesn't make an attack impossible,
but it mitigates the "the master time source is wrong" problem. This has
happend by accident before, so even without considering active attacks, it's
just the sensible thing to do.
Re: (Score:3)
That's good advice (it may actually be formal Best Practice for NTP configuration), but does violate Segal's Law: [wikipedia.org]
Which ultimately demonstrates that "a witty saying proves nothing."
Re: (Score:3)
This is why it pains me when I see two NTP servers setup. You NEVER know which one is right. Having just 1 is better than that.
To me, the minimum is 4: This gives you n+1, since you need at least 3 clocks to know if the time is right, and NTP will actually play ball with this logic: It will first look to see if 1 of the 3 are way off and exclude it, then it will sync time from the clock that is the lowest stratum and most stable.
Re: (Score:2)
As I routinely say to my trainees.
"One measurement is a datum. Two measurements is a disagreement. Three measurements is a majority opinion. Four measurements is statistics."
Re:Most NTP clients I've seen... (Score:5, Informative)
There is at least one alternative out there
Whoever proposed tlsdate as an alternative to NTP has no idea how either NTP or tlsdate work. What moving to tlsdate is doing is replacing a well-designed clock-synchronisation protocol talking to precise time servers with an opportunistic gimme-whatver-time-you've-got mechanism that returns a one-off estimate of an approximate time on a web server, assuming the server doesn't just set the time field to random bytes as many do. They're totally different things.
If you're really worried about this, run your own stratum 1 clock and serve NTP off that. If you're worried about the cost of a dedicated NTP server, build it yourself using any number of instructions on the Internet, e.g. these ones [satsignal.eu].
Re: (Score:2)
Yes, and if you read the article, you'll see they found ways around them.
Re: (Score:2)
Authenticated NTP? (Score:3)
Re: (Score:2)
What a great idea let every protocol invent its own crappy little authentication scheme.
Re: (Score:2)
Different protocols have different requirements; especially something as time sensitive as NTP.
Treating security with a 'one size fits all' approach is never a good idea.
Re: (Score:2)
Different protocols have different requirements; especially something as time sensitive as NTP.
DTLS adds no latency over UDP for purpose of time synchronization.
Treating security with a 'one size fits all' approach is never a good idea.
Unless it actually fits then its a great idea.
Re:Authenticated NTP? (Score:5, Funny)
Once an RFC is adopted by IETF (as the linked RFC is), it becomes a standard. Bro, do you even internet?
Re: (Score:3)
Once an RFC is adopted by IETF (as the linked RFC is), it becomes a standard. Bro, do you even internet?
Most RFCs are standards only in the minds of their authors.
Re: (Score:2)
Put the crack down and try again.
Re: (Score:2)
I also recall that there's like one person, and he's ancient, maintaining the package. He basically controls the keys. Fortunately, he's good and smart but he's pretty much the definition of the bus problem.
Re: (Score:2)
What "the package"? This about a specification.
Re: (Score:2)
IIRC he was maintaining the package for Linux/Unix and not the specs. 'Twas a /. article like a month or so ago. Obviously, I didn't RTFA. I'm no heretic.
Re: (Score:2)
Fortunately "the package" it isn't the only ntp implementation, so there's no bus problem.
Re: (Score:2)
You're not a heretic, just the next hairyfeet. God I'm sick of reading your bullshit anecdotes that never contribute anything.
Eric Raymond rewrite (Score:2)
Isn't Eric Raymond in the middle of a major rewrite of the NTP software, with emphasis on security?
Re:Eric Raymond rewrite (Score:5, Funny)
Isn't Eric Raymond in the middle of a major rewrite of the NTP software, with emphasis on security?
I heard it was going to become part of systemd.
Re:Eric Raymond rewrite (Score:4, Funny)
Time synchronisation is in systemd already:
http://lists.freedesktop.org/a... [freedesktop.org]
And it uses SNTP, not NTP.
Re: (Score:2)
Those consumed by hubris will continually reinvent what already exists, poorly.
"Scrambling" an SNTP client's clock doesn't require this exploit because SNTP doesn't retain any state (and all the complexities that come from that). The systemd time sync client is especially naive, even for SNTP.
Re: (Score:2)
Isn't Eric Raymond in the middle of a major rewrite of the NTP software, with emphasis on security?
I heard it was going to become part of systemd.
Doubtful, unless you're talking about fetchd?
Re: (Score:3)
No, but he and someone else are working to improve it a little, IIRC [catb.org]
OTOH, this guy maintains it [informationweek.com]
Re: (Score:1)
Yes, it is called ntpsec:
http://www.ntpsec.org/
http://esr.ibiblio.org/?p=6820
http://esr.ibiblio.org/?p=6846
Re:Eric Raymond rewrite (Score:4, Informative)
Re: (Score:3)
Isn't Eric Raymond in the middle of a major rewrite of the NTP software, with emphasis on security?
You're probably thinking of ntimed [nwtime.org].
Experience Y2K Again...and again... and again... (Score:2)
And around and around we go... (Score:1)
Researchers warn [X] is unsafe.
"We should replace [X] with [Y]!"
Later:
Researchers warn [Y] is unsafe.
Can we, like, audit something rather that just keep replacing things and letting the users figure out which new ones were broken? Maybe the researchers could take a look at [Y] and make sure that's working before we switch it out?
Ob (Score:5, Funny)
systemd automatically knows what time it is, but it'll only tell you in binary.
Re: (Score:2)
Re: (Score:2)
Yes.
Re: (Score:2)
There you go, using that fuzzy logic again...
So design things to not require synced clocks. (Score:4, Interesting)
So design things to not require synced clocks.
It's not like you couldn't include your idea of your local time (whatever it is) in your NFS requests, and then have the server take its idea of its local time, generate a delta, and apply that to all the timestamps that you are trying to set on a file. Or conversely, when you do a stat, the server could include its idea of the local time, the client could use that to generate a delta vs. its own idea of local time, and apply that delta to the time being reported up from the kernel to user space.
The whole idea of having to synchronize clocks between machines is rather moronic. When you have a billion mechanical computers wandering around in your body with robot bodies to e.g. fight a nasty cancer, do you really think there's going to be enough spare CPU cycles, RAM, or communications bandwidth for them to run NTP requests around to each other?
I recently fielded a request from someone who was building an embedded device; the trick was, it was going to be pre-programmed, then deployed everywhere, and not have local time beacons (i.e. it couldn't access local beacons, such as local cell towers, which send out "time is now" broadcasts). The question was: "How do I sync the time to the local time?".
My response was "Why?".
The reason finally boiled down to wanting to put the time in log files, and to display it on an LCD.
There was no reason for either of these: if the devices are Internet connected, just grab an HTTP header by hitting a known HTTP server, and log in UTC, since the time in the header will be reported as a UTC time + a zone delta. For the display: why the hell do you need to display the time on the small LCD? Because it was the only neutral thing he could think of to display on the LCD. "Can't I just look at my watch/iPhone/VCR/microwave/refrigerator/dishwasher/clock? Or just display it in UTC? Or display a PacMan animation instead of a clock?". "I guess so".
Problem solved with no need to sync clocks.
Non-synchronized clocks are only a problem if you let them be a problem/make them a problem.
Reasons for timestamps (Score:1)
...
Problem solved with no need to sync clocks.
Non-synchronized clocks are only a problem if you let them be a problem/make them a problem.
You have to be able to have an accurate time reference because a lot of devices interact with the real world. Logs, audit trails, signals intelligence, even photographs and surveillance cameras are a lot more meaningful if they have timestamps. If you can manipulate timestamps by using a flaw in NTP, then you can have an alibi for... anything.
Synced clocks are necessary (Score:4, Insightful)
So design things to not require synced clocks.
They do when that is a sane thing to do. Sometimes a precise notion of time isn't important. But many activities are impossible without a rather precise determination of the time across multiple devices.
It's not like you couldn't include your idea of your local time (whatever it is) in your NFS requests, and then have the server take its idea of its local time, generate a delta, and apply that to all the timestamps that you are trying to set on a file.
The only way to ensure the local time on your clock is correct is to synchronize with another clock. A clock providing arbitrary time stamps is worse than useless. In fact for many activities what you suggest would lead to accidents, fraud and all sorts of confusion.
Any time you have a measuring device where you care about its accuracy you have to compare it to a reference standard. That's why we have highly accurate atomic clocks maintained by standards organizations to calibrate our clocks to.
Non-synchronized clocks are only a problem if you let them be a problem/make them a problem.
Sorry my friend but that's simply not true for a lot of problems.
Re: (Score:2)
They do when that is a sane thing to do. Sometimes a precise notion of time isn't important. But many activities are impossible without a rather precise determination of the time across multiple devices.
[...]
The only way to ensure the local time on your clock is correct is to synchronize with another clock. A clock providing arbitrary time stamps is worse than useless. In fact for many activities what you suggest would lead to accidents, fraud and all sorts of confusion.
Any time you have a measuring device where you care about its accuracy you have to compare it to a reference standard. That's why we have highly accurate atomic clocks maintained by standards organizations to calibrate our clocks to.
These are reasonable arguments for a UTC time. Which you can get by hitting a known HTTP server, which I've already pointed out.
These are not reasonable arguments for trying to do NTP from an IoT device, and they are not reasonable arguments for trying to do NTP for most applications.
Re:So design things to not require synced clocks. (Score:4, Interesting)
So design things to not require synced clocks.
Practically all of modern humanity requires clocks so I'm not sure how we can design modern life without clocks.
The whole idea of having to synchronize clocks between machines is rather moronic.
Synchronizing clocks is necessary because clocks drift. Clocks have always drifted since the dawn of humanity. Humans have been able to make clocks that are more accurate but unless you want every machine to have an atomic clock built-in, synchronizing is one way to ensure they all agree.
When you have a billion mechanical computers wandering around in your body with robot bodies to e.g. fight a nasty cancer, do you really think there's going to be enough spare CPU cycles, RAM, or communications bandwidth for them to run NTP requests around to each other?
Um we are not talking about nanobots in the future. We are talking about today and servers: servers that run everything from email to shopping to banking. They all rely on correct clocks
There was no reason for either of these: if the devices are Internet connected, just grab an HTTP header by hitting a known HTTP server, and log in UTC, since the time in the header will be reported as a UTC time + a zone delta.
And how can your ensure that the web server that was hit has the correct time? That web server used an NTP server somewhere.
Problem solved with no need to sync clocks.
In your one example of an embedded device, you can question whether it needed to sync a clock. For the rest of the world, it is needed. Your sample size of one does not translate to the whole world.
Re: (Score:2)
Re: (Score:2)
So design things to not require synced clocks.
What happens when you can't or such a design would be more difficult than a global shared time reference?
It's not like you couldn't include your idea of your local time (whatever it is) in your NFS requests, and then have the server take its idea of its local time, generate a delta, and apply that to all the timestamps that you are trying to set on a file. Or conversely, when you do a stat, the server could include its idea of the local time, the client could use that to generate a delta vs. its own idea of local time, and apply that delta to the time being reported up from the kernel to user space.
What happens when every system can't just stamp their own "idea of local time" on to an event when it occurs? What happens when every systems "idea of local time" changes over time?
Non-synchronized clocks are only a problem if you let them be a problem/make them a problem.
Obviously it is best to avoid dependencies which require synchronization where possible. Unfortunately it isn't always possible.
Re: (Score:2)
So design things to not require synced clocks.
What happens when you can't or such a design would be more difficult than a global shared time reference?
What happens when every system can't just stamp their own "idea of local time" on to an event when it occurs? What happens when every systems "idea of local time" changes over time?
Non-synchronized clocks are only a problem if you let them be a problem/make them a problem.
Obviously it is best to avoid dependencies which require synchronization where possible. Unfortunately it isn't always possible.
In those cases, use NTP. Or use direct tier one information from GPS or cell tower data (if you are OK with the +/- 1 second on the cell tower's idea of the time). Or set the device's idea of local time as part of the handshake establishing the connection with the server (although this last one wouldn't work for mesh networks not communicating with servers).
Re: (Score:2)
Good luck on that.
Try getting a database spread across multiple servers without good clock sync. Try getting version control + build systems working well over a network without good clock sync.
It is kind of like having unguarded critical sections in your code. You might get something that works most of the time but you'll be left with mysterious and messy failures that are hard to explain, hard to reproduce, and make you want to gouge your own eyeballs out. I'd rather keep my eyeballs in their sockets wh
Re: (Score:2)
Any multi-server, replicated database that requires synchronized real-time clocks to maintain consistency is inherently broken. A system that lies about consistency is worse than a system that doesn't pretend to be consistent.
Google's Spanner database, for example, uses high-precision timestamps, but only for optimizing reads other housekeeping chores. Transactions are kept consistent and synchronized using proper consensus protocols (e.g. Paxos).
One of the major things that allows this optimization (and I agree, it's merely an optimization, and any data model that doesn't maintain consistency is inherently broken), id that all of the systems on which it as developed and deployed have working TSC's, meaning that they count while in low power states, and they are invariant under "speedboost" style short term clock accelerations, so once you get them synced up, they *stay* synced up. then reading the TSC is practically free.
Re: (Score:2)
Re: (Score:2)
The problem with delta's as you describe is how you account for network latency in your delta. Its impossible to tell clock drift from network latency when syncing two clocks in this way.
That a fair ask, assuming an asymmetric latency.
The easy way to handle this is to also send along your delta calculation with your packet as well, so that the other end can compare its delta calculation with your delta calculation.
Another alternative is to include this information in the establishment of the connection, as part of the encryption startup and the authentication handshake which establishes the credentials authorizing the server operations by the client. Although that would not account for var
Not for Windows? (Score:4, Interesting)
It appears to me that all the NTP patches and all the NTP alternatives are for UNIX or Linux systems.
I use SocketWatch on Windows 7 to synchronize my PC clock with external time servers around the world. I have it set to run every hour. It warns me whenever an adjustment to my PC clock is excessive (using my definition of "excessive").
The questions are: How do the reported problems with NTP affect me. Or do those problems only affect time servers?
Yes, I know SocketWatch is no longer being maintained. The developer is going out of business and will soon stop distributing it. As long as it works for me, I hope to keep using it.
Re: (Score:2)
Hasn't Windows automatically synched time out of the box since the first Lord of the Rings movie was released?
Re: (Score:2)
Hasn't Windows automatically synched time out of the box since the first Lord of the Rings movie was released?
ISTR it's had the feature since Win2k. It's pretty sloppy, though.
Re: (Score:3)
The Windows capability to synchronize my PC clock depends on a single time server. The default is Microsoft's own time.windows.com, which is not always up.
SocketWatch does not tie me to one particular time server the way the Windows capability does. SocketWatch has a list of servers, which I have edited. My list now has over 200 entries. Per my option settings, SocketWatch queries the top five entries from the list hourly, scoring each entry primarily on how quickly the server responds. The server with
Re: (Score:2)
The Windows capability to synchronize my PC clock depends on a single time server.
time.windows.com is a pool, it's not a single server. The only real risk you have is DNS going down.
My list includes some stratum 1 servers, which are atomic clocks.
Do you run a small cluster of stratum 2 servers for the express purpose of serving time to hundreds of other machines? If not you have no business syncing to stratum 1 servers. If you have a specific application for a single device that needs a high accuracy clock then why not spend $50 on a GPS unit with which you can sync your way to stratum 1 precision.
Re: (Score:2)
I only use those stratum 1 servers that (a) either serve my geographical area or are worldwide, (b) that have "open access", and (c) do not require me to notify them that I am using them. Also, I only use those stratum 2 servers that meet the same criteria.
The "Rules of Engagement" state: "There are many scenarios where the above rules may not apply, especially ... clients with intermittent connectivity ..." Given that I disconnect from the Internet whenever I walk away from my PC and I shut down my PC wh
Re: (Score:2)
OpenBSD's OpenNTPd with constraints (Score:2)
.
A quick description of OpenNTPd's constraints is here [marc.info].
Bad Example (Score:1)
Re: (Score:2)
How true. Wouldn't it be the fault of the system who accepted a time 12 years in the past and started up without so much as a "Hey, System Administrator! You might want to look at this! Do you really intend to go back 12 years in time here?" I know it wouldn't fix anything because such warnings would likely be ignored anyway, but at least the system Administrator would be to blame then.
So long as we're talking about NTPd (Score:2)
GPS and Tardis (Score:2)
Re: (Score:2)
Re: (Score:2)
If you're relying on a Tardis for accurate time, you're in for a world of pain.
Re: (Score:2)
Not Again! (Score:1)
Re: (Score:2)
I'm worried about 2038 myself... It's going to make Y2K look like a walk in the park and drop us all back into the dark ages of the 1970's...
You won't survive without your Fondue pots, plaid bell bottoms, leisure suits and disco moves....
Re: (Score:1)
Re: (Score:2)
Disco Stu will be born anew!
Re: (Score:2)
Is their a hoard common this whey?
Re: (Score:1)
tlsdate isn't a NTP replacement (Score:4, Informative)
The mentioned TLSdate isn't a NTP replacement.
It openly admits is roughly only good for a <1-5 second accuracy. That's crap. A typical NTP setup can easily maintain ~10-15 millisecond accuracy using public stratum 2 or 3 NTP servers from the Internet.
Sure, tlsdate is a simple, secure rdate replacement, and while many people without precise timing requirements it is good enough, it is simply not suitable for a huge range of applications that are time sensitive, or are timing / synchronization critical.
Re: (Score:3)
Not only that but:
TLS 1.3 and tlsdate
In the upcoming version of TLS, tlsdate might not function anymore.
Clocks (Score:1)
Re: (Score:2)
Are 3D Printers affected?? (Score:1)
Computer Clocks used to be scrambled via NTP flaws (Score:2)
'We thank the Network Time Foundation, NTPsec, Cisco, and RedHat's security team for quickly issuing patches for various issues described in this work'