Bug 236307 - inconsistent system time
inconsistent system time
Status: CLOSED WONTFIX
Product: Fedora
Classification: Fedora
Component: kernel-xen (Show other bugs)
6
x86_64 Linux
medium Severity high
: ---
: ---
Assigned To: Eduardo Habkost
Virtualization Bugs
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2007-04-12 19:35 EDT by Ask Bjørn Hansen
Modified: 2009-12-14 15:38 EST (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2008-02-26 19:04:40 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Ask Bjørn Hansen 2007-04-12 19:35:54 EDT
Description of problem:

With the latest FC6 Xen kernel the system time is completely unreliable, jumping back and forth(!) even 
in dom0.

[ask@app2 ~]$ uname -a
Linux app2.la.sol 2.6.20-1.2933.fc6xen #1 SMP Mon Mar 19 11:17:46 EDT 2007 x86_64 x86_64 
x86_64 GNU/Linux
[ask@app2 ~]$ while `/bin/true`; do date; sleep 1; done
Thu Apr 12 16:33:16 PDT 2007
Thu Apr 12 16:33:17 PDT 2007
Thu Apr 12 16:33:18 PDT 2007
Thu Apr 12 16:33:18 PDT 2007
Thu Apr 12 16:33:20 PDT 2007
Thu Apr 12 16:33:21 PDT 2007
Thu Apr 12 16:33:22 PDT 2007
Thu Apr 12 16:33:23 PDT 2007
Thu Apr 12 16:33:24 PDT 2007
Thu Apr 12 16:33:24 PDT 2007
Thu Apr 12 16:33:26 PDT 2007
Thu Apr 12 16:33:27 PDT 2007
Thu Apr 12 16:33:28 PDT 2007
Thu Apr 12 16:33:28 PDT 2007
Thu Apr 12 16:33:30 PDT 2007

(the seconds that appears to be okay are only that because it seems to more or less average out to be 
correct over time - each line is very "jumpy").

On the non-xen kernel I get:

[root@app3 ~]# uname -a
Linux app3.la.sol 2.6.20-1.2933.fc6 #1 SMP Mon Mar 19 11:00:19 EDT 2007 x86_64 x86_64 x86_64 
GNU/Linux
[root@app3 ~]# while `/bin/true`; do date; sleep 1; done
Thu Apr 12 16:20:28 PDT 2007
Thu Apr 12 16:20:29 PDT 2007
Thu Apr 12 16:20:30 PDT 2007
Thu Apr 12 16:20:31 PDT 2007


Version-Release number of selected component (if applicable):


How reproducible:

Always.
Comment 1 Ask Bjørn Hansen 2007-04-12 19:43:34 EDT
The two boxes above are dual quad-core Intel Xeon 2.33Ghz; I see the same problem on a dual dual-core 
Intel Xeon 2.00Ghz.

http://www.siliconmechanics.com/i5917/Xeon-Server.php

dmesg of xen kernel at http://tmp.askask.com/2007/04/app2-xen.dmesg.txt

dmesg of no-xen kernel at http://tmp.askask.com/2007/04/app3-noxen.dmesg.txt
Comment 2 Ask Bjørn Hansen 2007-04-12 19:45:00 EDT
FWIW, it seems to get better if I disable apic ...
Comment 3 Eduardo Habkost 2007-04-13 08:29:06 EDT
A test using 'date --rfc-3339 ns' in the place of the 'date' command may make 
the problem more visible. Could you do this test?
Comment 4 Ask Bjørn Hansen 2007-04-13 15:22:48 EDT
very cool, didn't know about that option.

Here's a non-xen kernel:

[root@app3 ~]# while `/bin/true`; do date --rfc-3339 ns; sleep 1; done
2007-04-13 12:19:42.766020000-07:00
2007-04-13 12:19:43.770184000-07:00
2007-04-13 12:19:44.774319000-07:00
2007-04-13 12:19:45.777746000-07:00
2007-04-13 12:19:46.781122000-07:00
2007-04-13 12:19:47.790459000-07:00
2007-04-13 12:19:48.794074000-07:00
2007-04-13 12:19:49.797861000-07:00
2007-04-13 12:19:50.801986000-07:00
2007-04-13 12:19:51.805114000-07:00

Here's from the xen kernel:

[root@app2 ~]# while /bin/true; do date --rfc-3339 ns; sleep 1; done
2007-04-13 12:20:15.672158000-07:00
2007-04-13 12:20:16.643917000-07:00
2007-04-13 12:20:17.672203000-07:00
2007-04-13 12:20:17.643942000-07:00
2007-04-13 12:20:19.006156000-07:00
2007-04-13 12:20:19.671671000-07:00
2007-04-13 12:20:20.672290000-07:00
2007-04-13 12:20:21.671717000-07:00
2007-04-13 12:20:21.656193000-07:00
2007-04-13 12:20:22.644064000-07:00
2007-04-13 12:20:23.672360000-07:00
2007-04-13 12:20:24.672386000-07:00
2007-04-13 12:20:25.582588000-07:00
2007-04-13 12:20:26.672925000-07:00
2007-04-13 12:20:27.672468000-07:00
2007-04-13 12:20:28.582667000-07:00
2007-04-13 12:20:28.672996000-07:00
2007-04-13 12:20:29.671917000-07:00
2007-04-13 12:20:29.656400000-07:00
2007-04-13 12:20:31.006121000-07:00
2007-04-13 12:20:31.672586000-07:00
2007-04-13 12:20:33.672630000-07:00


I booted the xen kernel with apic=off to see if it made a difference, but if it did it wasn't much:

title Fedora Core (2.6.20-1.2933.fc6xen)
        root (hd0,0)
        kernel /xen.gz-2.6.20-1.2933.fc6 com1=9600,8n1 com2=115200,8n1 apic=off
        module /vmlinuz-2.6.20-1.2933.fc6xen ro root=/dev/vg0/root console=tty0 console=ttyS0
        module /initrd-2.6.20-1.2933.fc6xen.img

Comment 5 Ask Bjørn Hansen 2007-04-17 02:04:43 EDT
Here's with the new kernel (notice how it starts out going backwards in time ...)

[root@app2 ~]# while /bin/true; do date --rfc-3339 ns; sleep 1; done
2007-04-16 23:02:48.981407000-07:00
2007-04-16 23:02:48.930131000-07:00
2007-04-16 23:02:50.820784000-07:00
2007-04-16 23:02:50.930164000-07:00
2007-04-16 23:02:51.791174000-07:00
2007-04-16 23:02:51.953771000-07:00
2007-04-16 23:02:52.819448000-07:00
2007-04-16 23:02:53.019187000-07:00
2007-04-16 23:02:53.797581000-07:00
2007-04-16 23:02:54.981535000-07:00
2007-04-16 23:02:54.953835000-07:00
2007-04-16 23:02:55.930285000-07:00
2007-04-16 23:02:56.791280000-07:00
2007-04-16 23:02:56.797647000-07:00
2007-04-16 23:02:57.981602000-07:00
2007-04-16 23:02:57.953897000-07:00
2007-04-16 23:02:58.930349000-07:00
2007-04-16 23:02:59.791346000-07:00
2007-04-16 23:02:59.904411000-07:00
2007-04-16 23:03:00.930392000-07:00
2007-04-16 23:03:01.791389000-07:00
2007-04-16 23:03:02.797772000-07:00
2007-04-16 23:03:02.821041000-07:00
2007-04-16 23:03:03.954014000-07:00
2007-04-16 23:03:04.904499000-07:00
2007-04-16 23:03:04.930476000-07:00
2007-04-16 23:03:05.791478000-07:00
2007-04-16 23:03:06.797861000-07:00
2007-04-16 23:03:06.821126000-07:00
2007-04-16 23:03:07.954106000-07:00
Comment 6 Ask Bjørn Hansen 2007-04-17 02:06:11 EDT
[root@app2 ~]# uname -a
Linux app2.la.sol 2.6.20-1.2944.fc6xen #1 SMP Tue Apr 10 18:03:37 EDT 2007 x86_64 x86_64 x86_64 
GNU/Linux

If it can help I think we can arrange for something from RedHat to get access to one of these systems.  
(The boxes have an IPMI KVM and a serial console and IPDU power, so they are easy to mess with 
remotely).


 - ask
Comment 7 Eduardo Habkost 2007-06-05 18:08:57 EDT
Hi,

Sorry for taking so long to take a look on this bug.

Could you check the results of the same command (while /bin/true; do 
date --rfc-3339 ns; sleep 1; done), using the kernel command line 
option "independent_wallclock"?

Also, could you attach dmesg output when using the following command line 
options:
- Using both: independent_wallclock permitted_clock_jitter=0
- Using only: permitted_clock_jitter=0

Probably it will flood your console with messages, but the information from 
them will be useful.
Comment 8 Red Hat Bugzilla 2007-07-24 21:40:27 EDT
change QA contact
Comment 9 Ask Bjørn Hansen 2007-08-07 21:33:00 EDT
Hi Eduardo,

Darn - I missed your comment earlier.

FWIW changing independent_wallclock in the dom0 (with sysctl) doesn't seem to make a difference.  I'll 
try with permitted_clock_jitter=0 later.


 - ask

Linux app3.la.sol 2.6.20-2925.13.fc7xen #1 SMP Tue Jul 17 10:38:19 EDT 2007 x86_64 x86_64 
x86_64 GNU/Linux

[ask@app3 ~]$  while /bin/true; do date --rfc-3339 ns; sleep 1; done
2007-08-07 18:31:35.155215000-07:00
2007-08-07 18:31:37.070061000-07:00
2007-08-07 18:31:38.081759000-07:00
2007-08-07 18:31:39.317794000-07:00
2007-08-07 18:31:41.120665000-07:00
2007-08-07 18:31:42.932862000-07:00
2007-08-07 18:31:44.005449000-07:00
2007-08-07 18:31:45.029116000-07:00
2007-08-07 18:31:46.037569000-07:00
Comment 10 Eduardo Habkost 2007-09-27 16:05:43 EDT
The FC-6 updates system haven't added an automatic note on this bug and I 
haven't noticed. A patch was included since 2.6.20-1.3001.fc6 that should help 
fix this issue.

Could you check what are the results when using 2.6.20-1.3001.fc6 or newer?

If you are using Fedora 7, the patch was added on 2.6.20-2930.fc7.
Comment 11 Chris Lalancette 2008-02-26 19:04:40 EST
This report targets FC6, which is now end-of-life.

Please re-test against Fedora 7 or later, and if the issue persists, open a new bug.

Thanks

Note You need to log in before you can comment on or make changes to this bug.