Linux clock loses 10 minutes every week

Question

One of my linux server's clock loses 10 minutes every now and then, nearly every week. I update the time so it stays correct, and although it doesn't really bother me, i'd like to fix it.

I've been searching around a bit. Nothing can be responsible in the crontab, and i can't find any related message in the logs. Some people seem to use ntp to fix that kind of issue, but i'd prefer not to use an unecessary component on it.

Uname result : Linux unis-monitor 2.6.32-5-686 #1 SMP Mon Feb 25 01:04:36 UTC 2013 i686 GNU/Linux

Cat message :

cat messages
Jul 14 06:25:06 unis-monitor rsyslogd: [origin software="rsyslogd" swVersion="4.6.4" x-pid="882" x-info="http://www.rsyslog.com"] rsyslogd was HUPed, type 'lightweight'.
Jul 15 06:25:05 unis-monitor rsyslogd: [origin software="rsyslogd" swVersion="4.6.4" x-pid="882" x-info="http://www.rsyslog.com"] rsyslogd was HUPed, type 'lightweight'.

Cat syslog

cat syslog
Jul 15 06:25:05 unis-monitor rsyslogd: [origin software="rsyslogd" swVersion="4.6.4" x-pid="882" x-info="http://www.rsyslog.com"] rsyslogd was HUPed, type 'lightweight'.
Jul 15 06:39:01 unis-monitor /USR/SBIN/CRON[15272]: (root) CMD (  [ -x /usr/lib/php5/maxlifetime ] && [ -d /var/lib/php5 ] && find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -delete)
Jul 15 07:09:01 unis-monitor /USR/SBIN/CRON[15465]: (root) CMD (  [ -x /usr/lib/php5/maxlifetime ] && [ -d /var/lib/php5 ] && find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -delete)
Jul 15 07:17:01 unis-monitor /USR/SBIN/CRON[15521]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Jul 15 07:39:01 unis-monitor /USR/SBIN/CRON[15662]: (root) CMD (  [ -x /usr/lib/php5/maxlifetime ] && [ -d /var/lib/php5 ] && find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -delete)
Jul 15 08:09:01 unis-monitor /USR/SBIN/CRON[15855]: (root) CMD (  [ -x /usr/lib/php5/maxlifetime ] && [ -d /var/lib/php5 ] && find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -delete)
Jul 15 08:17:01 unis-monitor /USR/SBIN/CRON[15911]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Jul 15 08:39:01 unis-monitor /USR/SBIN/CRON[16052]: (root) CMD (  [ -x /usr/lib/php5/maxlifetime ] && [ -d /var/lib/php5 ] && find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -delete)
Jul 15 09:09:01 unis-monitor /USR/SBIN/CRON[16273]: (root) CMD (  [ -x /usr/lib/php5/maxlifetime ] && [ -d /var/lib/php5 ] && find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -delete)

So if you have any clue of where to look or what i could use to monitor those date change ?

Here is some more infos : the server is a virtual server hosted on HyperV on a win 2012 server. Don't know if it changes anything, seen the other servers hosted don't have this issue...

Answer 1

HyperV is the disease, NTPd is the cure.

Where to look or what i could use to monitor those date change?

You can query the NTPd daemon (via the ntpq client) to get the difference between local clock and the NTPd servers reference clock. But that implies actually running NTPd so you are not monitoring your changes alone, you're monitoring the combined effect of your running local clock and NTPd keeping it in sync.

I don't actually know if you can configure NTPd to run (and give you the above mentioned metrics) but not to actually adjust the system clock. A different, and less efficient, way would be to periodically (cron?) run ntpdate -q against a set of reference NTPd servers and monitor its output, which will give you the difference between your clock and the reference without actually touching the local clock. Output will be like this:

$ ntpdate -q $YOUR_TLD.pool.ntp.org
[... list of queried servers ...]
17 Jul 12:14:11 ntpdate[42868]: adjust time server 109.168.106.59 offset -0.002517 sec

You could filter the last number and graph it to get a nice view of how much and when your clock jumps around:

$ OFFSET=$( ntpdate -q $YOUR_TLD.pool.ntp.org | grep adjust | awk '{ print $10 }' )
$ echo $OFFSET
0.002970

Answer 2

NTP is not an unnecessary component.

It's the only sane way to keep a system clock updated against an atomic time source.

You should install NTP, configure it to use your local pool time source, and everything will be fine.

Answer 3

In my tests I discovered that Hyper-V virtual machine clocks are very unreliable and frequently have a clock drift exceeding 500ppm. This is enough to cause even ntpd to fail. I had to switch to using chrony in these VMs to provide reasonably accurate wall clocks; it defaults to 1000ppm for this scenario and can be adjusted even further if necessary.

I no longer seriously consider Hyper-V for any application where timekeeping is especially important.

Answer 4

Clock drift in a linux vm running in Hyper-V is very common if you don't have the correct integration components installed and enabled. Hyper-V will adjust the hardware clock of any VM it runs, but linux by default does not rely on the hardware clock once the system is booted. The integration components gives the kernel hints about the actual clock time and eliminates this problem.

If integration components are not an option for your distro, then the correct solution is to install something like ntpd and sync to the Hyper-V host or your local clock pool.

Answer 5

The "problem" is that your running a version of Linux that does not support Pluggable Time Source Infrastructure. Several distros support it, but commonly only in their 64-bit OSes, and not in their 32-bit.

As others have mentioned, Integration Services allows for clock synchronization, but only if PTSI is supported. Most distros that support PTSI have the HV source already. See if there is a adjtimex port/package available for your distro.

Using NTP is a valid alternative, if you can't get PTSI working correctly. There are also some hacks including the tickadj command, and changing kernel boot variables - these should be avoided.