There's a timeout parameter used by NTP that if things get out of sync by too much, it won't sync them at all.
The default is something low, I think just 128 msec, which is pretty small under some circumstances.
edit the /etc/sysconfig/ntpd on the control VM to sync hardware. Default is no. Once done, please cycle ntpd. You will see an extra line for hardware clock. If possible, all VMs and XenServer should use a local physical time server.