Over the weekend (Sunday March 27th) Daylight Savings Time kicked in here in Ireland and so all the servers (all win2k3 server) in our Production Environment updated their Times +1 Hour. However something which I cannot explain caused 2 of our cluster nodes to update @ 02:08 and 02:38 rather than @ 00:59.
We have 2 Domains in our production environment.
- DOMAIN1 contains PDC1 + 6 or 7 Virtual Machines. (Standalone Web, Load Bal, File servers)
- DOMAIN2 contains PDC2 + 2 Physical Servers which are clustered together for SQL Server
In DOMAIN1, everything seemed to work perfectly with all machines moving forward 1 hour at exacly
02:00:00. This can be confirmed by viewing timestamps in logfiles on any of those servers, or viewing timestamp data written to DB tables by those servers.
In DOMAIN2 however, something odd happened.
- Jobs in the SQL Server Agent Instance on DB_NODE_1 show the time change occuring @ 02:08. i.e. there are per minute jobs which show start times of 02:08, then 03:09
- Jobs in the SQL Server Agent Instance on DB_NODE_2 show the time change occuring @ 02:38. i.e. there are per minute jobs which show start times of 02:38, then 03:39
Same goes for DB Inserts on both instances that relied on the SQL
GetDate() function. Data shows the getdate() inserts jumping forward +1 hour at
02:38 respectively on the two instances.
So my question is WHY?
All Servers (PDC, Physical & VM) have the exact same Locale & Timezone Settings (GMT, Ireland(English)) All Servers have the "Automatically Adjust for Daylight Savings Time" option selected. All Non-PDC Servers are setup to use the Windows Time Service to get time updates from their PDC if there's any variances All Servers have the exact same W32Time Registry Settings
So I am completely stumped why these 2 cluster nodes decided to wait 68 minutes & 98 minutes to do their DST updates.
Thanks if anyone can shine a light on this one.