1

Some of my tests on EC2 servers began failing. When I dug in, the reason was serious time mismatches between my servers:

Sun Jun 22 14:43:43 UTC 2014  # Laptop
Sun Jun 22 14:46:36 UTC 2014  # Server #1
Sun Jun 22 07:50:19 UTC 2014  # Server #2

The difference is significant - 3 minutes on one server, about 7 hours on another one.

So, I tossed up a little script which calculated the time difference between one of the server and some other servers:

$ $ ./time_diff.py
+--------+---------------+----------------------+----------------------+-----------------------+
| Number |       ip      | difference (seconds) | local time (seconds) | remote time (seconds) |
+--------+---------------+----------------------+----------------------+-----------------------+
|   1    |  172.31.45.17 |        -142.73       |     1403448641.8     |     1403448499.07     |
|   2    |  172.31.35.69 |      -24977.82       |     1403448642.7     |     1403423664.87     |
|   3    | 172.31.47.152 |        -141.79       |    1403448643.51     |     1403448501.72     |
|   4    | 172.31.47.151 |        -119.05       |    1403448644.35     |     1403448525.31     |
|   5    |  172.31.32.31 |        -124.29       |     1403448645.2     |     1403448520.91     |
|   6    | 172.31.26.134 |        -122.59       |    1403448646.43     |     1403448523.85     |
|   7    | 172.31.27.224 |        -161.85       |    1403448647.72     |     1403448485.87     |
|   8    | 172.31.17.112 |        -156.70       |    1403448649.03     |     1403448492.32     |
+--------+---------------+----------------------+----------------------+-----------------------+

This is very surprising: I thought that the virtual machines get their time from the underlying physical machine, and that the physical machine time is well-kept by an Amazon NTP server.

Update - ntpdate I have tried using ntpdate, and the relevant UDP ports are open, but I can't get it to fix the drift:

$ sudo ntpdate -dv 0.ubuntu.pool.ntp.org
22 Jun 15:20:56 ntpdate[11889]: ntpdate 4.2.6p5@1.2349-o Wed Oct  9 19:08:07 UTC 2013 (1)
Looking for host 0.ubuntu.pool.ntp.org and service ntp
host found : hydrogen.constant.com
transmit(108.61.73.243)
transmit(108.61.73.244)
transmit(173.255.118.107)
transmit(208.87.104.40)
transmit(108.61.73.243)
transmit(108.61.73.244)
transmit(173.255.118.107)
transmit(208.87.104.40)
transmit(108.61.73.243)
transmit(108.61.73.244)
transmit(173.255.118.107)
transmit(208.87.104.40)
transmit(108.61.73.243)
transmit(108.61.73.244)
transmit(173.255.118.107)
transmit(208.87.104.40)
transmit(108.61.73.243)
transmit(108.61.73.244)
transmit(173.255.118.107)
transmit(208.87.104.40)
108.61.73.243: Server dropped: no data
108.61.73.244: Server dropped: no data
173.255.118.107: Server dropped: no data
208.87.104.40: Server dropped: no data
server 108.61.73.243, port 123
stratum 0, precision 0, leap 00, trust 000
refid [108.61.73.243], delay 0.00000, dispersion 64.00000
transmitted 4, in filter 4
reference time:    00000000.00000000  Mon, Jan  1 1900  0:00:00.000
originate timestamp: 00000000.00000000  Mon, Jan  1 1900  0:00:00.000
transmit timestamp:  d75172df.1d895ce9  Sun, Jun 22 2014 15:21:03.115
filter delay:  0.00000  0.00000  0.00000  0.00000
         0.00000  0.00000  0.00000  0.00000
filter offset: 0.000000 0.000000 0.000000 0.000000
         0.000000 0.000000 0.000000 0.000000
delay 0.00000, dispersion 64.00000
offset 0.000000

server 108.61.73.244, port 123
stratum 0, precision 0, leap 00, trust 000
refid [108.61.73.244], delay 0.00000, dispersion 64.00000
transmitted 4, in filter 4
reference time:    00000000.00000000  Mon, Jan  1 1900  0:00:00.000
originate timestamp: 00000000.00000000  Mon, Jan  1 1900  0:00:00.000
transmit timestamp:  d75172df.50bc6bd2  Sun, Jun 22 2014 15:21:03.315
filter delay:  0.00000  0.00000  0.00000  0.00000
         0.00000  0.00000  0.00000  0.00000
filter offset: 0.000000 0.000000 0.000000 0.000000
         0.000000 0.000000 0.000000 0.000000
delay 0.00000, dispersion 64.00000
offset 0.000000

server 173.255.118.107, port 123
stratum 0, precision 0, leap 00, trust 000
refid [173.255.118.107], delay 0.00000, dispersion 64.00000
transmitted 4, in filter 4
reference time:    00000000.00000000  Mon, Jan  1 1900  0:00:00.000
originate timestamp: 00000000.00000000  Mon, Jan  1 1900  0:00:00.000
transmit timestamp:  d75172df.83eeeb52  Sun, Jun 22 2014 15:21:03.515
filter delay:  0.00000  0.00000  0.00000  0.00000
         0.00000  0.00000  0.00000  0.00000
filter offset: 0.000000 0.000000 0.000000 0.000000
         0.000000 0.000000 0.000000 0.000000
delay 0.00000, dispersion 64.00000
offset 0.000000

server 208.87.104.40, port 123
stratum 0, precision 0, leap 00, trust 000
refid [208.87.104.40], delay 0.00000, dispersion 64.00000
transmitted 4, in filter 4
reference time:    00000000.00000000  Mon, Jan  1 1900  0:00:00.000
originate timestamp: 00000000.00000000  Mon, Jan  1 1900  0:00:00.000
transmit timestamp:  d75172df.b722b2ea  Sun, Jun 22 2014 15:21:03.715
filter delay:  0.00000  0.00000  0.00000  0.00000
         0.00000  0.00000  0.00000  0.00000
filter offset: 0.000000 0.000000 0.000000 0.000000
         0.000000 0.000000 0.000000 0.000000
delay 0.00000, dispersion 64.00000
offset 0.000000

22 Jun 15:21:05 ntpdate[11889]: no server suitable for synchronization found

How can I sync the time on my Ubuntu servers on EC2?

Adam Matan
  • 13,194
  • 19
  • 55
  • 75

2 Answers2

0

To keep time in sync with ntp servers simply install package ntp and restart the service with

/etc/init.d/ntp restart
Lluís
  • 425
  • 1
  • 4
  • 22
0

Solved by using ntpdate -u.

It seems that EC2 machine can not connect to an NTP server outside their Availability Zone without this option.

Community
  • 1
Adam Matan
  • 13,194
  • 19
  • 55
  • 75