1

Some of my tests on EC2 servers began failing. When I dug in, the reason was serious time mismatches between my servers:

Sun Jun 22 14:43:43 UTC 2014  # Laptop
Sun Jun 22 14:46:36 UTC 2014  # Server #1
Sun Jun 22 07:50:19 UTC 2014  # Server #2

The difference is significant - 3 minutes on one server, about 7 hours on another one.

So, I tossed up a little script which calculated the time difference between one of the server and some other servers:

$ $ ./time_diff.py
+--------+---------------+----------------------+----------------------+-----------------------+
| Number |       ip      | difference (seconds) | local time (seconds) | remote time (seconds) |
+--------+---------------+----------------------+----------------------+-----------------------+
|   1    |  172.31.45.17 |        -142.73       |     1403448641.8     |     1403448499.07     |
|   2    |  172.31.35.69 |      -24977.82       |     1403448642.7     |     1403423664.87     |
|   3    | 172.31.47.152 |        -141.79       |    1403448643.51     |     1403448501.72     |
|   4    | 172.31.47.151 |        -119.05       |    1403448644.35     |     1403448525.31     |
|   5    |  172.31.32.31 |        -124.29       |     1403448645.2     |     1403448520.91     |
|   6    | 172.31.26.134 |        -122.59       |    1403448646.43     |     1403448523.85     |
|   7    | 172.31.27.224 |        -161.85       |    1403448647.72     |     1403448485.87     |
|   8    | 172.31.17.112 |        -156.70       |    1403448649.03     |     1403448492.32     |
+--------+---------------+----------------------+----------------------+-----------------------+

This is very surprising: I thought that the virtual machines get their time from the underlying physical machine, and that the physical machine time is well-kept by an Amazon NTP server.

Update - ntpdate I have tried using ntpdate, and the relevant UDP ports are open, but I can't get it to fix the drift:

$ sudo ntpdate -dv 0.ubuntu.pool.ntp.org
22 Jun 15:20:56 ntpdate[11889]: ntpdate 4.2.6p5@1.2349-o Wed Oct  9 19:08:07 UTC 2013 (1)
Looking for host 0.ubuntu.pool.ntp.org and service ntp
host found : hydrogen.constant.com
transmit(108.61.73.243)
transmit(108.61.73.244)
transmit(173.255.118.107)
transmit(208.87.104.40)
transmit(108.61.73.243)
transmit(108.61.73.244)
transmit(173.255.118.107)
transmit(208.87.104.40)
transmit(108.61.73.243)
transmit(108.61.73.244)
transmit(173.255.118.107)
transmit(208.87.104.40)
transmit(108.61.73.243)
transmit(108.61.73.244)
transmit(173.255.118.107)
transmit(208.87.104.40)
transmit(108.61.73.243)
transmit(108.61.73.244)
transmit(173.255.118.107)
transmit(208.87.104.40)
108.61.73.243: Server dropped: no data
108.61.73.244: Server dropped: no data
173.255.118.107: Server dropped: no data
208.87.104.40: Server dropped: no data
server 108.61.73.243, port 123
stratum 0, precision 0, leap 00, trust 000
refid [108.61.73.243], delay 0.00000, dispersion 64.00000
transmitted 4, in filter 4
reference time:    00000000.00000000  Mon, Jan  1 1900  0:00:00.000
originate timestamp: 00000000.00000000  Mon, Jan  1 1900  0:00:00.000
transmit timestamp:  d75172df.1d895ce9  Sun, Jun 22 2014 15:21:03.115
filter delay:  0.00000  0.00000  0.00000  0.00000
         0.00000  0.00000  0.00000  0.00000
filter offset: 0.000000 0.000000 0.000000 0.000000
         0.000000 0.000000 0.000000 0.000000
delay 0.00000, dispersion 64.00000
offset 0.000000

server 108.61.73.244, port 123
stratum 0, precision 0, leap 00, trust 000
refid [108.61.73.244], delay 0.00000, dispersion 64.00000
transmitted 4, in filter 4
reference time:    00000000.00000000  Mon, Jan  1 1900  0:00:00.000
originate timestamp: 00000000.00000000  Mon, Jan  1 1900  0:00:00.000
transmit timestamp:  d75172df.50bc6bd2  Sun, Jun 22 2014 15:21:03.315
filter delay:  0.00000  0.00000  0.00000  0.00000
         0.00000  0.00000  0.00000  0.00000
filter offset: 0.000000 0.000000 0.000000 0.000000
         0.000000 0.000000 0.000000 0.000000
delay 0.00000, dispersion 64.00000
offset 0.000000

server 173.255.118.107, port 123
stratum 0, precision 0, leap 00, trust 000
refid [173.255.118.107], delay 0.00000, dispersion 64.00000
transmitted 4, in filter 4
reference time:    00000000.00000000  Mon, Jan  1 1900  0:00:00.000
originate timestamp: 00000000.00000000  Mon, Jan  1 1900  0:00:00.000
transmit timestamp:  d75172df.83eeeb52  Sun, Jun 22 2014 15:21:03.515
filter delay:  0.00000  0.00000  0.00000  0.00000
         0.00000  0.00000  0.00000  0.00000
filter offset: 0.000000 0.000000 0.000000 0.000000
         0.000000 0.000000 0.000000 0.000000
delay 0.00000, dispersion 64.00000
offset 0.000000

server 208.87.104.40, port 123
stratum 0, precision 0, leap 00, trust 000
refid [208.87.104.40], delay 0.00000, dispersion 64.00000
transmitted 4, in filter 4
reference time:    00000000.00000000  Mon, Jan  1 1900  0:00:00.000
originate timestamp: 00000000.00000000  Mon, Jan  1 1900  0:00:00.000
transmit timestamp:  d75172df.b722b2ea  Sun, Jun 22 2014 15:21:03.715
filter delay:  0.00000  0.00000  0.00000  0.00000
         0.00000  0.00000  0.00000  0.00000
filter offset: 0.000000 0.000000 0.000000 0.000000
         0.000000 0.000000 0.000000 0.000000
delay 0.00000, dispersion 64.00000
offset 0.000000

22 Jun 15:21:05 ntpdate[11889]: no server suitable for synchronization found

How can I sync the time on my Ubuntu servers on EC2?

0

To keep time in sync with ntp servers simply install package ntp and restart the service with

/etc/init.d/ntp restart
  • Tried that. It did not work, probably for the same reason ntpdate does not work. – Adam Matan Jun 22 '14 at 15:26
  • have you set up outbound firewall rules on EC2? – Lluís Jun 22 '14 at 15:27
  • Yes. I will double-check them. – Adam Matan Jun 22 '14 at 15:29
  • ups, just checked EC2 and seem there's no outbound firewall rules, so have you configured iptables to block outbound traffic on your server? – Lluís Jun 22 '14 at 15:32
  • No, I did not change anything in iptables. – Adam Matan Jun 23 '14 at 7:24
0

Solved by using ntpdate -u.

It seems that EC2 machine can not connect to an NTP server outside their Availability Zone without this option.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.