2

At my organisation we have a number of virtual machines on ESXi hosts in a number of sites.

For the past few months, around once or twice a week one virtual machine, seemingly at random, will update the date to something weeks in the future for a few minutes, only for W32Time to notice and change it back. This is affecting the applications running on the server and we cannot figure out what could be causing it.

This is not the usual time change of a few minutes, it's shifting the date and time to something completely different, and there's no pattern to what it moves to.

The latest example can be seen in the below event log pulled from the server:

"Event 4616, Microsoft Windows security auditing. The system time was changed. Subject: Security ID: LOCAL SERVICE Account Name: LOCAL SERVICE Account Domain: NT AUTHORITY Logon ID: 0x3E5 Process Information: Process ID: 0x174 Name: C:\Windows\System32\svchost.exe Previous Time: ‎2022‎-‎10‎-‎11T19:28:42.731783900Z New Time: ‎2022‎-‎12‎-‎06T05:38:25.439000000Z This event is generated when the system time is changed. It is normal for the Windows Time Service, which runs with System privilege, to change the system time on a regular basis. Other system time changes may be indicative of attempts to tamper with the computer."

Any help or advice would be greatly appreciated!

Improve this question
5
  • Is the VM set to sync time with the host? What is w32Time using as it's source? What is it's configuration?
    – joeqwerty
    Oct 11, 2022 at 23:47
  • It is not set to sync time with the host and w32Time is set to use its local Read-only DC, which as far as we can tell, always has the correct time. For what it's worth - The RODCs look for their time from Azure.
    – Mayzed
    Oct 12, 2022 at 7:47
  • @Mayzed have you find the reason why this happens? We have got somewhat similar issue, it's very frustrating, and we still cannot find a way to solve it unfortunately.
    – MyUserName
    Nov 1, 2022 at 10:19
  • Are you changing the locale ?
    – Rohit Gupta
    Nov 1, 2022 at 11:04
  • @RohitGupta no, we don't. These VMs run the server apps, so there is no need in such changes for us.
    – MyUserName
    Nov 1, 2022 at 14:36

1 Answer 1

Reset to default
1

Most likely, the VM is being 'updated' to the wrong time by the ESXi host.

When a VM is 'stunned' (on pause, snapshot, migration, etc.) it inherits the host's system time, regardless of the VM settings. Possibly the 'seemingly at random' times are when snapshots are taken, backups are running or similar. And very likely the other VMs are having the same problem but you just haven't noticed it yet (been there myself).

Therefore, it's very important to keep your hosts updated by NTP at all times and check their correct system time on a regular basis.

Improve this answer
2
  • Hello! Thanks for the reply. We went down this rabbit hole too, and our ESX hosts are indeed showing the correct time whenever we check them. So while we don’t sync time with the host, in events where the VM is in the midst of a snapshot, the time it gets from the host should be correct. It certainly would never be getting a date weeks into the future! So unfortunately this still remains a mystery to us.
    – Mayzed
    Jan 6 at 16:15
  • Then I'd definitely monitor the network connection for any wanted - and unwanted - NTP and SNTP traffic. Perhaps there's a funny upstream stratum host. If the time offset always the same or is it fluctuating?
    – Zac67
    Jan 6 at 16:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.