Bug 506505 - Xen - DomU clock incorrect during boot
Xen - DomU clock incorrect during boot
Status: CLOSED INSUFFICIENT_DATA
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: xen (Show other bugs)
5.3
x86_64 Linux
low Severity medium
: rc
: ---
Assigned To: Xen Maintainance List
Virtualization Bugs
:
Depends On:
Blocks: 514498
  Show dependency treegraph
 
Reported: 2009-06-17 11:05 EDT by Joel Koon
Modified: 2010-12-06 08:00 EST (History)
9 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2010-12-06 08:00:43 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Joel Koon 2009-06-17 11:05:48 EDT
Description of problem:
Fully virtualized DomU RHEL 5VM's start up w/ 4 hour time difference from Dom0.

On DomU, hwclock --utc shows correct time, hwclock --localtime shows incorrect time.

Dom0:
# date && cat /etc/adjtime && hwclock --show --debug && cat /etc/sysconfig/clock 
Wed Jun 17 10:04:37 EDT 2009
1.527610 1245244376 0.000000
1245244376
LOCAL
hwclock from util-linux-2.13-pre7
Using /dev/rtc interface to clock.
Last drift adjustment done at 1245244376 seconds after 1969
Last calibration done at 1245244376 seconds after 1969
Hardware clock is on local time
Assuming hardware clock is kept in local time.
Waiting for clock tick...
/dev/rtc does not have interrupt functions. Waiting in loop for time from /dev/rtc to change
...got clock tick
Time read from Hardware Clock: 2009/06/17 10:04:38
Hw clock time : 2009/06/17 10:04:38 = 1245247478 seconds since 1969
Wed 17 Jun 2009 10:04:38 AM EDT  -0.735927 seconds
# The ZONE parameter is only evaluated by system-config-date.
# The timezone of the system is defined by the contents of /etc/localtime.
ZONE="America/New_York"
UTC=false
ARC=false

DomU:
# date && cat /etc/adjtime && hwclock --show --debug && cat /etc/sysconfig/clock 
Wed Jun 17 14:06:50 EDT 2009
0.000000 1245247530 0.000000
1245247530
LOCAL
hwclock from util-linux-2.13-pre7
Using /dev/rtc interface to clock.
Last drift adjustment done at 1245247530 seconds after 1969
Last calibration done at 1245247530 seconds after 1969
Hardware clock is on local time
Assuming hardware clock is kept in local time.
Waiting for clock tick...
/dev/rtc does not have interrupt functions. Waiting in loop for time from /dev/rtc to change
...got clock tick
Time read from Hardware Clock: 2009/06/17 14:06:50
Hw clock time : 2009/06/17 14:06:50 = 1245262010 seconds since 1969
Wed 17 Jun 2009 02:06:50 PM EDT  -1.018045 seconds
ZONE="America/New_York"
UTC=false
ARC=false

Version-Release number of selected component (if applicable):
RHEL5 Host:
xen-libs-3.0.3-80.el5_3.2
xen-3.0.3-80.el5_3.2
kernel-xen-2.6.18-128.el5
xen-libs-3.0.3-80.el5_3.2
kernel-xen-2.6.18-128.1.10.el5

RHEL5 Host and Guest VM:
util-linux-2.13-0.50.el5

How reproducible:
Reboot DomU VM.
On DomU, set utc=true in /etc/sysconfig/clock gets DomU clock corrected during boot. Why do I have to use UTC instead of local?

Steps to Reproduce:
1. set utc=false in /etc/sysconfig/clock on DomU and reboot, time incorrect.
2. set utc=true in /etc/sysconfig/clock on DomU and reboot, time correct.
Comment 2 Ethan Baker 2009-06-17 11:58:02 EDT
Here is the output of virsh:

<domain type='xen' id='26'>
  <name>XEN_DOMU_01</name>
  <uuid>b187aa0d-64dd-f0a8-a8d8-5ff5f3e21ba4</uuid>
  <os>
    <type>hvm</type>
    <loader>/usr/lib/xen/boot/hvmloader</loader>
    <boot dev='hd'/>
  </os>
  <memory>4194304</memory>
  <vcpu>4</vcpu>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <clock offset='utc'/>
  <devices>
    <emulator>/usr/lib64/xen/bin/qemu-dm</emulator>
    <interface type='bridge'>
      <source bridge='xenbr0'/>
      <target dev='vif26.0'/>
      <mac address='00:16:3e:22:ef:ab'/>
      <script path='vif-bridge'/>
    </interface>
    <disk type='file' device='disk'>
      <driver name='file'/>
      <source file='/var/lib/xen/images/XEN_DOMU_01.img'/>
      <target dev='hda'/>
    </disk>
    <input type='mouse' bus='ps2'/>
    <graphics type='vnc' port='5926' keymap='en-us'/>
    <console tty='/dev/pts/4'/>
  </devices>
</domain>

Here is the output of xm list:

(domain
    (domid 26)
    (uuid b187aa0d-64dd-f0a8-a8d8-5ff5f3e21ba4)
    (vcpus 4)
    (cpu_weight 1.0)
    (memory 4096)
    (shadow_memory 36)
    (maxmem 4096)
    (features )
    (name XEN_DOMU_01)
    (on_poweroff destroy)
    (on_reboot restart)
    (on_crash restart)
    (image
        (hvm
            (kernel /usr/lib/xen/boot/hvmloader)
            (device_model /usr/lib64/xen/bin/qemu-dm)
            (vcpus 4)
            (boot c)
            (acpi 1)
            (apic 1)
            (pae 1)
            (usb 1)
            (serial pty)
            (vnc 1)
            (vncunused 1)
            (keymap en-us)
        )
    )
    (device
        (vif
            (backend 0)
            (script vif-bridge)
            (bridge xenbr0)
            (mac 00:16:3e:22:ef:ab)
        )
    )
    (device
        (vbd
            (backend 0)
            (dev hda:disk)
            (uname file:/var/lib/xen/images/XEN_DOMU_01.img)
            (mode w)
        )
    )
    (state -b----)
    (shutdown_reason poweroff)
    (cpu_time 92.499350989)
    (online_vcpus 4)
    (up_time 281.606243134)
    (start_time 1245253785.96)
    (store_mfn 983038)
)
Comment 3 Rik van Riel 2009-06-17 12:16:25 EDT
Ethan,

does changing the UTC line in /etc/sysconfig/clock to "UTC=true" in the guest make things work correctly?
Comment 4 Daniel Berrange 2009-06-17 12:28:47 EDT
Ok, so the XML / xm list output shows the guest definitely does *not* have the 'localtime' setting enabled. Thus the guest should be synchronized to the host hardware clock.

You original comment though shows that the host's hardware clock is stored as localtime. I expect this is probably what's confusing Xen, and causing the guest to see localtime when it should in fact be seeing UTC.

Can you change your host hardware clock to be synced to UTC and see if that corrects the guest problems ?
Comment 5 Ethan Baker 2009-06-17 12:33:18 EDT
and yet in the configuration file, localtime=1 is set.  It would appear that sometimes this value goes through, and system time is correct, whereas other times it does not go through.

here is the xen config:

name = "XEN_DOMU_01"
uuid = "b187aa0d-64dd-f0a8-a8d8-5ff5f3e21ba4"
maxmem = 4096
memory = 4096
vcpus = 4
localtime=1
builder = "hvm"
kernel = "/usr/lib/xen/boot/hvmloader"
boot = "c"
pae = 1
acpi = 1
apic = 1
on_poweroff = "destroy"
on_reboot = "restart"
on_crash = "restart"
device_model = "/usr/lib64/xen/bin/qemu-dm"
sdl = 0
vnc = 1
vncunused = 1
keymap = "en-us"
disk = [ "file:/var/lib/xen/images/XEN_DOMU_01.img,hda,w" ]
vif = [ "mac=00:16:3e:22:ef:ab,bridge=xenbr0,type=ioemu" ]
serial = "pty"
Comment 6 Ethan Baker 2009-06-17 14:23:32 EDT
Daniel,

Should I set the system hardware clock to UTC/GMT and then the localtime on Dom0 to localtime?  Or are you requesting that everything be UTC, even local timezone?
Comment 7 Daniel Berrange 2009-06-17 14:31:19 EDT
Typically the host RTC would be stored in UTC.  Then as a general rule the guest config files  should also be set to utc (ie don't add a localtime=1 setting). This allows the guest admin to decide what their localtime setting is independantly of the host. The only time you'd normally want to set localtime=1 for a guest config is for Windows fullyvirt machines, since Windows expects the BIOS RTC to be stored in localtime.
Comment 8 Ethan Baker 2009-06-17 15:01:33 EDT
Daniel,

Unfortunately, our division doesn't do the "typical" thing. It is division policy that RTC be stored in localtime.  Similarly, all systems then also use localtime for their timezones.   I don't see  how to make this work with that situation.
Comment 9 Daniel Berrange 2009-06-17 16:19:54 EDT
For guests that would be fine - simply make sure all your guests, both linux & windows have localtime=1 in their Xen config, then the guest OS can be made to deal with this as you have with UTC=false in the sysconfig file.

For the host OS though, things are more complicated. The hypervisor will default to giving the guest a clock synced to the host OS RTC. The dom0  mgmt tools assume the host RTC is synced to UTC.  Then when localtime=1 is set in guest config, the tools will apply an offset to the guest clock. 

The trouble with having the host RTC in localtime, is that the time exposed to the guest by default is already offset.  So in fact the tools are doing precisely the opposite of what they need todo. They would in fact have to be setting a negative offset for when localtime=0, and not doing any offset for localtime=1. The hard thing is that I'm not sure the dom0 mgmt tools are able to easily determine whether the host RTC is UTC vs localtime. So it may not be possible to make it do the right thing here.

So at least for the immediate future I don't see any option other than to make your host keep its RTC in UTC, not localtime.
Comment 10 Ethan Baker 2009-06-18 08:28:17 EDT
It seems that ntpd does not function in HVM guests unless acpi=0 is in the xen config for that guest.  Otherwise, the clock does not stay synchronized to the proper time source, but instead to the LOCAL(0) clock instead, skewing a lot each day.
Comment 12 Michal Novotny 2010-03-22 08:10:42 EDT
(In reply to comment #10)
> It seems that ntpd does not function in HVM guests unless acpi=0 is in the xen
> config for that guest.  Otherwise, the clock does not stay synchronized to the
> proper time source, but instead to the LOCAL(0) clock instead, skewing a lot
> each day.    

Is the solution in comment #9 working fine for you?

Michal
Comment 13 John Feeney 2010-09-10 16:35:22 EDT
Can we close this given we have been waiting for more than a year for feedback?
Comment 14 Miroslav Rezanina 2010-12-06 08:00:43 EST
As there's no answer for more than 8 month, closing this issue. Feel free to open in case you have any new data on this problem.

Note You need to log in before you can comment on or make changes to this bug.