Hyper-V Machine drifts time all over, even with NTP

From your description, it sounds like there is an actual hardware problem with the RTC (http://en.wikipedia.org/wiki/Real-time_clock) on the motherboard of server S2.

The Hyper-V guest gets it's clock initially from the host (HYV1), but as you have Hyper-V time sync disabled, it gets all further clock updates from NIST (which is working fine). Your Linux VM is not integrated with Hyper-V, so it is getting it's time from the domain, which is also working fine. Your other physical machines are working fine, it is just a single physical server that is having 1 second of drift every 20 seconds (which is a crazy amount of drift). The time is drifting much quicker than the network time sync can reset the clock to the right time (which if I recall correctly takes place every 8 hours).

If you want to rule out Hyper-V as a cause for the error on S2, create a "no Hypervisor" boot entry, reboot without Hyper-V, and see if the time drift persists. Instructions here: http://blogs.msdn.com/virtual_pc_guy/archive/2008/04/14/creating-a-no-hypervisor-boot-entry.aspx

-Sean

The problem is with the virtual implementation of the various clock sources (tsc, jiffies, acpi_pm, cmos_trc). The best way I have found to fix this problem with HyperV is to turn off the HyperV provided clock sync for your guest machine, then use adjtimex to adjust the time. On an Ubuntu guest OS do this...

# rm /var/log/clocks.log
# /etc/init.d/ntp-server stop
# ntpdate ntp.ubuntu.com
# hwclock -u --systohc
# adjtimex -l -u -h ntp.ubuntu.com

and answer No to both questions

# while [ /bin/true ] ; do yes | adjtimex -l -u -h ntp.ubuntu.com ; sleep 60 ; done

leave that to run for a few hours to calibrate, hit Ctrl-C to exit it.

# adjtimex -r -a -u -h ntp.ubuntu.com

this will do a least squares analysis of your clock and will find the right adjustment

# ntpdate ntp.ubuntu.com
# hwclock -u --systohc
# /etc/init.d/ntp-server start

this will resync the time on your machine and ntp should then be able to keep it in sync because it shouldn't drift too much anymore.

This seems to be a very common issue with VM's. See the following websites:

http://www.vmwareinfo.com/2008/04/enabling-ntp-on-esx-servers.html

http://social.technet.microsoft.com/Forums/en-US/winserverhyperv/thread/6fff3eef-1b5b-4059-8618-22ab3f5c293c

My suggestion would be to sync with just an external time server and disable any integration time sync'ing

Hopefully this helps.