date command displays different time

Question

I have one centos 5.8 Linux server. I am experiencing a bit strange issue and figured out it is something with system time. When I check date command, outputs are really strange. Here is example:

[root@web ~]# date
Thu Feb 19 21:21:59 JST 2015
[root@web ~]# date
Thu Feb 19 21:23:43 JST 2015
[root@web ~]# date
Thu Feb 19 21:22:02 JST 2015
[root@web ~]# date
Thu Feb 19 21:23:45 JST 2015
[root@web ~]#

As you can see each date command outputs different time. It seems there are two different system time? Does anybody know what causes this and potential fix to this?

UPDATE

This server is not virtual machine. It has two x64 Dual Core Processors. Same processors.

UPDATE 2

hwclock -r result:

[root@web ~]# hwclock -r --utc
Mon Dec 29 21:37:14 2014  -101.846756 seconds
[root@web ~]# hwclock -r --utc
Mon Dec 29 21:37:15 2014  -101.908926 seconds
[root@web ~]# hwclock -r --utc
Mon Dec 29 21:37:16 2014  -102.393569 seconds
[root@web ~]# hwclock -r --utc
Mon Dec 29 21:37:19 2014  -101.846761 seconds
[root@web ~]# hwclock -r --utc
Mon Dec 29 21:37:20 2014  -101.846721 seconds
[root@web ~]# hwclock -r --utc
Mon Dec 29 21:37:21 2014  -102.065391 seconds
[root@web ~]# hwclock -r --utc
Mon Dec 29 21:37:22 2014  -102.581079 seconds

update 3

date; hwclock -r result

[root@web ~]# date; hwclock -r --utc
Thu Feb 19 21:42:38 JST 2015
Mon Dec 29 21:42:38 2014  -0.016165 seconds
[root@web ~]# date; hwclock -r --utc
Thu Feb 19 21:42:38 JST 2015
Mon Dec 29 21:42:39 2014  -0.156442 seconds
[root@web ~]# date; hwclock -r --utc
Thu Feb 19 21:42:39 JST 2015
Mon Dec 29 21:42:40 2014  -0.125509 seconds
[root@web ~]# date; hwclock -r --utc
Thu Feb 19 21:42:40 JST 2015
Mon Dec 29 21:42:41 2014  101.812663 seconds
[root@web ~]# date; hwclock -r --utc
Thu Feb 19 21:42:42 JST 2015
Mon Dec 29 21:42:42 2014  -0.016169 seconds
[root@web ~]#

UPDATE 4

some logs related to ntpd:

Feb 19 20:01:33 web ntpd[1759]: time reset -103.081446 s
Feb 19 20:07:16 web ntpd[1759]: synchronized to 210.188.224.14, stratum 2
Feb 19 20:26:31 web ntpd[1759]: time reset -103.082073 s
Feb 19 20:31:36 web ntpd[1759]: synchronized to LOCAL(0), stratum 10
Feb 19 20:32:40 web ntpd[1759]: synchronized to 210.188.224.14, stratum 2
Feb 19 20:49:39 web ntpd[1759]: time reset -103.081776 s
Feb 19 20:54:50 web ntpd[1759]: synchronized to LOCAL(0), stratum 10
Feb 19 20:55:30 web ntpd[1759]: synchronized to 210.188.224.14, stratum 2
Feb 19 21:18:11 web ntpd[1759]: time reset -103.082092 s
Feb 19 21:23:43 web ntpd[1759]: synchronized to LOCAL(0), stratum 10
Feb 19 21:24:49 web ntpd[1759]: synchronized to 210.188.224.14, stratum 2
Feb 19 21:40:57 web ntpd[1759]: time reset -103.082297 s
Feb 19 21:46:05 web ntpd[1759]: synchronized to 210.188.224.14, stratum 2

UPDATE 5

I have another identical server with exact same setup, but only this one has this issue at this moment. The logs above only appears in the server with the issue.

No virtual machine. We are not doing any sync such as ntpdate. I believe this machine has two CPU's — mask8, Feb 19 '15 at 12:29
sorry actually ntpd is running. I am not sure if it is syncing system time as I am not the one who set this server up. I added hwclock result — mask8, Feb 19 '15 at 12:34
ntp if doing adjustments should log something. But ntp should not make time skip forwards and backwards. Only if you run something like ntpdate in a loop — Dan, Feb 19 '15 at 12:40
I added `date; hwclock -r` result. quite interesting. I am not sure what is going on. btw thank you for your help @Dan — mask8, Feb 19 '15 at 12:43
actually "Mon Dec 29 21:37:22 2014" isn't right is it. hardware clock isn't working? — mask8, Feb 19 '15 at 12:47
hw clock (bios clock) is only used at boot to set system time. And updated at shutdown. Any entries in logs that contain the string ntpdate? Any funny cronjobs that mess with time? — Dan, Feb 19 '15 at 12:53
I added crond related logs. nothing with ntpdate. no special cronjobs. just basic jobs that come with the centos distribution as far as I can see. — mask8, Feb 19 '15 at 13:00
Those ntp time reset lines show the problem. Ntp keeps moving the clock 100 seconds backwards and somebody else keeps moving it forward. Now you have to find that somebody else — Dan, Feb 19 '15 at 13:04
I see. But that clock refresh only occurs every 25 minutes or so. Does that still cause the date command issue? meantime I will check who's causing it. I need to leave now but I will keep investigate when I'm back. thanks a lot — mask8, Feb 19 '15 at 13:10
Maybe something is messing with the clock more frequent than ntp catches is, no idea. — Dan, Feb 19 '15 at 13:12
Do you have the local clock listed as a server in ntp.conf? If your motherboard clock is bad it can cause problems with NTP. — Boscoe, Feb 20 '15 at 17:05
@Boscoe right. that's probably it. I'm asking the hosting company to replace hardware. — mask8, Feb 27 '15 at 09:09
They replaced the hardware other than hard disk drive, and the issue is totally gone. It seems that hardware clock wasn't working properly, and the NTP with local clock was overriding system time. Thanks dan and boscoe and all. I would like to vote you guys up if you can add answer — mask8, Mar 16 '15 at 06:39
@mask8 I found this question very helpful even though I found it wasn't the answer to the problem that brought me here. In the absence of anyone else answering can I suggest you add the answer yourself, just to make sure the question doesn't disappear for lack of activity? — Doug McLean, Nov 17 '15 at 16:27
@DougMcLean It was simply hardware trouble. The host had been running about 5 years, and hosting guy needed to replace hardware. After that, the error has gone.. thank you for all your supports — mask8, Dec 04 '15 at 06:33

score 0 · Answer 1 · answered Dec 04 '15 at 06:33

0

I thought I should post to answer to my question. It was simply hardware trouble. The host had been running about 5 years, and hosting guy needed to replace hardware. After that, the error has gone.. thank you for all your supports

answered Dec 04 '15 at 06:33

mask8

169
5

date command displays different time

1 Answers1