8

Until recently, we had at work a small cluster of about 20 small Windows servers (which have now all been virtualized). They were all configured to synchronize with the local time server. It was on an 1Gb sub-network in our own DC. I never got them to be less than about 100ms away from each other, which I consider to be an incredibly big difference.

Is that a normal value? What is a realistic expectation of time difference between machines running on a 1Gb network, and all connected to the same time server, and updating frequently, say every 5 minutes?

I would like to know this as setting timeouts and other parameters in a distributed application requires to take that difference into consideration.

Improve this question
2
  • As you say, 100ms is very big. The question is: how do you measure this time difference ?
    – Gregory MOUSSAT
    Jun 2, 2012 at 16:21
  • I have a central "loadbalancer" that receives messages from all machines regularly. And it can display the timestamp value of the events it received from the other machines. When an event arrive 100ms before it was sent then I know that the clocks have at least 100ms differences, probably a lot more.
    – monster
    Jun 2, 2012 at 16:35

2 Answers 2

Reset to default
14

NTP usually hovers in the <10-20ms range on a "reasonable" network. If you're using Windows built-in NTP client however, that's a different story. The Windows built-in client is only meant to keep the time synchronized enough for Kerberos, 5 minutes. They had no intention of keeping very accurate time; so 100ms is doing pretty good.

If you need more accurate time than 100ms, install an actual NTP client, like the Meinberg binaries of ISC's Reference NTPd.

If you need ultra accurate time, you should be looking at Precision Time Protocol (PTP). Be warned that PTP is much more complicated than NTP, so unless you need ultra accurate timestamping or something similar, NTP is highly advised.

Improve this answer
6

NTP ought to be able to do a lot better than 100ms

Virtualized machines are notorious for having odd time sync issues.

You don't mention your virtualization environment, but check and make sure all the hypervisors are NTP sync'd as well. (Generally it is best for a guest VM machine to take its clock from its host hypervisor.)

See this 'Known Issue' in the NTP documentation -- it gives some more detail.

http://support.ntp.org/bin/view/Support/KnownOsIssues#Section_9.2.2.

See this paper from VMware for excruciating detail about problems with timekeeping on Windows hardware in general and additional problems with timekeeping in virtual machines:

http://www.vmware.com/files/pdf/Timekeeping-In-VirtualMachines.pdf

Improve this answer
5
  • The problem was on the hardware cluster, but it did get worst on VMWare.
    – monster
    Jun 2, 2012 at 16:31
  • then another question would be what sort of average ping latencies you get between hosts. Most ping times inside a datacenter should be under 1ms. How jittery are those pings? Also, to be clear you said you're seeing 100ms of difference between nodes. That's milli, not micro (u), right? Can I ask how you're measuring the time deltas?
    – Joel K
    Jun 2, 2012 at 16:39
  • Yes, milliseconds, not microseconds. And I measure it as I said in my comment to @Gregory. By comparing the local time to the "sent time" of the event I receive. If the sent time is older, I cannot say if it is clock drift or network latency. But if it is in the future, than I know that the clock of the sender is ahead by far.
    – monster
    Jun 2, 2012 at 16:50
  • Did you do some ping tests to identify if network jitter is a factor?
    – Joel K
    Jun 3, 2012 at 20:20
  • No. And I cannot do that anymore since that cluster was "retired".
    – monster
    Jun 4, 2012 at 10:56

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .