9

I have two linux machines (A and B) on an isolated network. They must be time-synchronized. Machine A is powered intermittently and must serve the time, as it is connected to an authoritative time source (GPS). Machine B is only powered if machine A is powered, but it is an embedded linux device and its power state will change frequently. Neither machine has access to other systems. It's a closed network.

I understand that this is quite a tall order for NTP, as NTP usually expects to have contact with several servers. I'm having trouble getting this to work properly on Machine B. Machine A synchs to the GPS just fine, and machine B can reach machine A and even do time queries, but Machine A is not trusted (perhaps by itself?). After a solid hour of machine A being up, this suddenly changed and machine B worked. However, when machine A went down (and thus machine B), machine B is once again unable to find a good time synch.

Here's some ntpdate info. Please note that even when machine A's stratum is 1, the operation fails with the same output at the end.

10.10.10.1: Server dropped: strata too high
server 10.10.10.1, port 123
stratum 16, precision -19, leap 11, trust 000
refid [10.10.10.1], delay 0.02614, dispersion 0.00000
transmitted 4, in filter 4
reference time:    00000000.00000000  Thu, Feb  7 2036  6:28:16.000
originate timestamp: d3a9bdc4.27ebb350  Thu, Jul 12 2012 21:19:00.155
transmit timestamp:  bc17c803.b42dfffe  Sat, Jan  1 2000  0:25:39.703
filter delay:  0.02625  0.02614  0.02618  0.02625 
         0.00000  0.00000  0.00000  0.00000 
filter offset: 39544160 39544160 39544160 39544160
         0.000000 0.000000 0.000000 0.000000
delay 0.02614, dispersion 0.00000
offset 395441600.451568

 1 Jan 00:25:39 ntpdate[677]: no server suitable for synchronization found

My guess is that machine A just doesn't trust itself for serving time. After 51 minutes (may have happened earlier, I don't know) of uptime and having its clock synch'd to GPS, machine A started to serve time correctly, and machine B picked it up. I need this to happen earlier. Like, within seconds if possible.

With the following configs (and a lot of waiting), it eventually succeeds.

Machine A ntp.conf:

server 127.127.28.0 prefer true minpoll 4 maxpoll 4
fudge 127.127.28.0 stratum 1 time1 0.420 refid GPS 

Machine B ntp.conf:

server 10.10.10.1 prefer true minpoll 4 maxpoll 4

ntpq -c peers on Machine B without good time fix:

     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
 10.10.10.1   .STEP.          16 u    9   16    0    0.000    0.000   0.000

ntp1 -c peers on Machine B with good time fix:

     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
*10.10.10.1   SHM(0)           2 u    7   16   17    0.669    2.597   1.808

So, now the question becomes: how do I make Machine A trust itself quickly?

Some debug output from Machine A before and after machine B decides that Machine A is good enough to use..

before..

~ # ntpq -c rv
associd=0 status=c418 leap_alarm, sync_uhf_radio, 1 event, no_sys_peer,
version="ntpd [email protected] Fri Feb 24 15:01:45 UTC 2012 (1)",
processor="armv7l", system="Linux/2.6.35.14", leap=11, stratum=2,
precision=-19, rootdelay=0.000, rootdisp=44.537, refid=SHM(0),
reftime=d3ab0053.43b44780  Fri, Jul 13 2012 20:15:15.264,
clock=d3ab0062.e7e03154  Fri, Jul 13 2012 20:15:30.905, peer=34819, tc=4,
mintc=3, offset=0.000, frequency=0.000, sys_jitter=3.853,
clk_jitter=36.492, clk_wander=0.000

after...

~ # ntpq -c rv
associd=0 status=0415 leap_none, sync_uhf_radio, 1 event, clock_sync,
version="ntpd [email protected] Fri Feb 24 15:01:45 UTC 2012 (1)",
processor="armv7l", system="Linux/2.6.35.14", leap=00, stratum=2,
precision=-19, rootdelay=0.000, rootdisp=41.278, refid=SHM(0),
reftime=d3ab0063.43b37856  Fri, Jul 13 2012 20:15:31.264,
clock=d3ab006d.9ee53ec2  Fri, Jul 13 2012 20:15:41.620, peer=34819, tc=4,
mintc=3, offset=0.000, frequency=43.896, sys_jitter=0.762,
clk_jitter=36.953, clk_wander=0.000
6
  • 1
    Could we see the ntp.conf files and output from ntpq -p when machine B is NOT getting good time from machine A? It could be marking machine A as a false ticker or something. When machine B isn't trusting machine A, is machine A synchronized with the GPS? (Output of ntpstat on machine A.) Jul 12, 2012 at 22:07
  • I've heard that chrony is more suitable for this application. "If your computer connects to the 'net for 5 minutes once a day (or something like that), or you turn your (Linux v2.0) computer off when you're not using it, or you want to use NTP on an isolated network with no hardware clocks in sight, chrony will work much better for you." Jul 12, 2012 at 23:30
  • @AaronCopley I can post those in a few (10 or 12) hours. Machine A becomes sychronized to the GPS within a minute of booting Machine B has problems synching to machine A for quite a long period of time. Jul 12, 2012 at 23:34
  • @DavidSchwartz Thanks. I'll look into it, but I'm a little reluctant to change much beyond configurations if I can help it. It's a chore to cross-build anything for Machine B at this time. Jul 12, 2012 at 23:36
  • @AaronCopley Updated. Jul 13, 2012 at 10:32

1 Answer 1

9

NTP should work fine. Look at some of the options for fast synchronization on start-up. Look at the burst and iburst options for the system B. Look at the true option for the GPS clock source.

Consider using the hardware clock as a backup time source on both systems. Set a higher stratum system B. Something like the following should work:

server  127.127.1.0
fudge   127.127.1.0 stratum 8

Watch the output of ntpq -c peers to see when you get a trusted clock source. Normally ntp wants a number of responses from a trusted time source before it trusts it. This is indicated by the first character on each line.

While NTP likes more sources, any odd number of time sources within one stratum level should work well. As you only have two servers and a GPS clock the priority (stratum) of the sources should increase from GPS, clock on server A, clock on server B. Increasing the stratum between each by three or four levels will ensure priorities are respected.

EDIT: If you have the busybox NTP server on server A, it may be worthwhile installing the full ntp server package. Understanding what is happening with server A should go a long way to solving your problem. You will need at least one trusted time source there before server B should trust it. If ntpq -c peers doesn't work, then you can try ntpdc peers. Both these commands allow you to query other hosts. A peerstats log could also be useful.

On server B use ntpclient as documented the busybox ntp howto to log what is happening on it

The clocks should be reasonably close to the correct time if the servers haven't been down for long. If you need to sync the two systems, that should be sufficient. The GPS will bring the time into sync with the real world eventually.

'ntpd -q' synchronizes quickly, but exits (ntpdate behaviour). It needs to be followed by an ntpd command without the quit option to have continuous synchronization.

EDIT2: I check my server and found one of the servers was off by a second. While fixing this I played with the settings. iburst gets a server trusted very quickly. true ensured the clock driver was trusted if there weren't multiple other trusted sources. The clock took a little more than a minute before it was locally trusted and could be trusted remotely.

When testing you should be able to restart the ntpd process once it is synchronized and test how fast settings work. In the above case Server B may need to be restarted to test how fast it synchronizes. When monitoring ntpd changes I use a line like:

while ntpq -c peers localhost; do sleep 10; done

The hostname and sleep time are adjusted as require. In some cases I chain two or more ntpq command lines in the loop. When doing so I use an echo and/or date command to provide an indication of where sets of data change.

7
  • Adding burst to the conf file did not improve the situation. Each of these machines is a busybox machine, and the "-c" option is unknown to ntpq. Also, the clocks cannot be trust on these devices until they are synched with the GPS. Just a limitation of the systems. Thanks. Jul 13, 2012 at 12:24
  • I actually made one small mistake, I already had the full version of ntpd running on Machine A. Machine B is the only one running the BusyBox version (and if I had a way to build programs for it, I'd do the same there). Eventually, everything works. I think it's a sever trust problem. Could you give some insight to my edits? Thanks. Jul 13, 2012 at 17:31
  • Also, if you get a chance to edit your answer again, could you @ me so the system notifies me? Thanks. Jul 13, 2012 at 17:41
  • @SanJacinto I have added a second edit with results from my system. I don't have the busybox ntpd client so I can't vouch for the results with it. I would try adding both true and iburst to server B.
    – BillThor
    Jul 13, 2012 at 23:20
  • +1 from me for your effort, but it's not solving my problem. A solution I've found (and please suggest something else if you wish and I'll try it) is to kill ntpd on machine A after it synchs to the GPS, and then restart it. This seems to let machine B synch to machine A within seconds. My guess is that a 42-year jump in time on Machine A (always boots from the Epoch) is making it nervous about sharing its time, but when it starts and the clock is already set, it's as though the clock wasn't far off to being with, so minor adjustments make it feel good about sharing its time. I did allow ntp.. Jul 16, 2012 at 12:09

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .