Created attachment 769171 [details] /var/log/messages (note the time change when chrony resets the time, near the end of the file) Description of problem: I have a dual-boot (WinXP/F19) machine with a clean F19 install. I used a registry hack to make Windows assume that the hardware clock is set to UTC instead of local time, so I can set the system clock to UTC in Fedora. My local time is currently 4 hours behind UTC. When I first boot, local time and UTC are both 4 hours fast, even though the RTC is correct. After about 30 seconds, chrony resets them to the correct values. For example (note the correct time was 04:56 EDT == 08:56 UTC): [root@dell-pc ~]# date Fri Jul 5 08:56:12 EDT 2013 [root@dell-pc ~]# timedatectl Local time: Fri 2013-07-05 08:56:13 EDT Universal time: Fri 2013-07-05 12:56:13 UTC RTC time: Fri 2013-07-05 08:56:13 Timezone: America/New_York (EDT, -0400) NTP enabled: yes NTP synchronized: yes RTC in local TZ: no DST active: yes Last DST change: DST began at Sun 2013-03-10 01:59:59 EST Sun 2013-03-10 03:00:00 EDT Next DST change: DST ends (the clock jumps one hour backwards) at Sun 2013-11-03 01:59:59 EDT Sun 2013-11-03 01:00:00 EST [root@dell-pc ~]# date ; timedatectl Fri Jul 5 04:56:24 EDT 2013 Local time: Fri 2013-07-05 04:56:24 EDT Universal time: Fri 2013-07-05 08:56:24 UTC RTC time: Fri 2013-07-05 08:56:23 Timezone: America/New_York (EDT, -0400) NTP enabled: yes NTP synchronized: no RTC in local TZ: no DST active: yes Last DST change: DST began at Sun 2013-03-10 01:59:59 EST Sun 2013-03-10 03:00:00 EDT Next DST change: DST ends (the clock jumps one hour backwards) at Sun 2013-11-03 01:59:59 EDT Sun 2013-11-03 01:00:00 EST [root@dell-pc ~]# If I set the system clock to local time instead of UTC, it works properly (even though timedatectl complains). Windows is not involved, since I can reboot Fedora and see the problem without Windows ever running. In any case, the RTC is always correct. I have another dual-boot (Windows Vista/F19) machine, again a clean F19 install, and that machine does NOT show this problem, even though it's configured exactly the same. I have no explanation. Version-Release number of selected component (if applicable): chrony-1.28-0.1.pre1.fc19 How reproducible: always on the one machine, never on a different one
I have no idea what the correct Component is, so please reassign as appropriate and sorry if this is the wrong one.
Created attachment 769239 [details] dmesg Nothing is added to dmesg when the clock is set back 4 hours. Note the entries [ 0.907737] Write protecting the kernel read-only data: 2276k [ 0.914338] systemd[1]: RTC configured in localtime, applying delta of -240 minutes to system time. [ 0.917759] systemd[1]: systemd 204 running in system mode. (+PAM +LIBWRAP +AUDIT +SELINUX +IMA +SYSVINIT +LIBCRYPTSETUP +GCRYPT +ACL +XZ) [ 0.918314] systemd[1]: Running in initial RAM disk. This is NOT true, the RTC is configured in UTC, as noted above. I'll change the component to systemd (still not sure that's correct, though).
I have a F19-only machine, again with system clock set to UTC. The time works fine on it. The dmesg output for it is [ 5.409921] Write protecting the kernel read-only data: 2248k [ 5.489993] systemd[1]: systemd 204 running in system mode. (+PAM +LIBWRAP +AUDIT +SELINUX +IMA +SYSVINIT +LIBCRYPTSETUP +GCRYPT +ACL +XZ) [ 5.491618] systemd[1]: Running in initial RAM disk. so on that machine, systemd correctly sees that the RTC is set to UTC. On both of my dual-boot machines, with system time set to UTC, on one of which the time is broken and one where it works correctly, systemd falsely claims "RTC configured in localtime".
RTC in UTC vs. LOCAL is defined by: /etc/adjtime What's in that file on both machines?
The third line in /etc/adjtime on both machines is "UTC", as expected.
Ok. What does: "systemd falsely claims "RTC configured in localtime" exactly mean than? What is "systemd" in that context?
(In reply to Andre Robatino from comment #3) On > both of my dual-boot machines, with system time set to UTC, on one of which > the time is broken and one where it works correctly, systemd falsely claims > "RTC configured in localtime". Hmm are you dual booting with windows on those machines? If so you might need to fix it ( via regedit hack ) on the windows side since windows is known to mishandle UTC and cause issues in dual booting setups. You should be able to confirm if that's the issue by opening Regedit, drill down to... HKLM\SYSTEM\CurrentControlSet\Control\TimeZoneInformation and create a new DWORD entry named “RealTimeIsUniversal”. Set the value to 1.
(In reply to Jóhann B. Guðmundsson from comment #7) > Hmm are you dual booting with windows on those machines? > > If so you might need to fix it ( via regedit hack ) on the windows side > since windows is known to mishandle UTC and cause issues in dual booting > setups. See comment 0. I already did that (and also told Windows not to set the time). In any case, it's irrelevant, since this problem happens on repeated reboots of Fedora, even when Windows never runs.
(In reply to Kay Sievers from comment #6) > What does: > "systemd falsely claims "RTC configured in localtime" > exactly mean than? > > What is "systemd" in that context? I don't understand the question. My RTC is configured in UTC (as confirmed by both /etc/adjtime and the timedatectl output in comment 0). The dmesg output on both dual-boot Windows/F19 machines says [ 0.914338] systemd[1]: RTC configured in localtime, applying delta of -240 minutes to system time.
What is the output of: $ sudo lsinitrd -f etc/adjtime
(In reply to Harald Hoyer from comment #10) > What is the output of: > > $ sudo lsinitrd -f etc/adjtime On both of my dual-boot Windows/F19 machines (both the one where time works properly, and the one where it is broken) the output is 0.0 0 0.0 0 LOCAL On my F19-only machine (where time works properly) the output is 0.0 0 0.0 0
Seems, the initramfs image still included the old config file. Running: dracut -f should update it.
(In reply to Kay Sievers from comment #12) > Seems, the initramfs image still included the old config file. > > Running: > dracut -f > should update it. Thanks, that works! I rebuilt the initramfs for all kernels on all 3 machines, and all of them work normally now. So where was the bug? Is it already fixed?
It's not really a "bug". There is currently no reasonable infrastructure to trigger a rebuild of the initrd when core config things change. It's a manual step.
OK, so I guess it's a side effect of the fact that I can't specify UTC vs. local time in the new installer, together with probably having applied the kernel updates before changing the system time to UTC (though I can't remember whether I did that). I'll close this as NOTABUG and keep an eye on the next kernel update, then. Thanks.
(In reply to Kay Sievers from comment #14) > It's not really a "bug". There is currently no reasonable infrastructure > to trigger a rebuild of the initrd when core config things change. It's a > manual step. Maybe there should be a bug for this problem. The initramfs contains executables, including systemd, as well as config files.[1] For example, the fix for this systemd bug did not take effect until the initramfs was rebuilt: Bug 870535 - Regression in handling of localtime RTC: timezone offset applied twice [1] $ sudo zcat /boot/initramfs-3.9.8-300.fc19.x86_64.img | cpio -tv
I'm not the only one seeing this - see http://forums.fedoraforum.org/showthread.php?p=1659967 . I hope the anaconda devs take this into account in deciding whether to put back the UTC/local time choice, or alternatively if there is another way to prevent this bug that would be okay as well. I don't mind having to use system-config-date to change the setting if it actually works.
Have you tried booting a rescue image? The rescue initramfs doesn't have /etc/adjtime or /etc/localtime: $ sudo zcat /boot/initramfs-0-rescue-*.img | cpio -tv --quiet | egrep 'adjtime|localtime'
Reopening, reassigning to dracut and changing the Summary in response to https://bugzilla.redhat.com/show_bug.cgi?id=981793#c5 .
If initramfs is rebuilt without /etc/adjtime, the system clock is correct on rebooting: 1. # lsinitrd -f /etc/adjtime # Verify contents of adjtime in initramfs. 2. # mv -i /etc/adjtime /etc/adjtime.BAK1 # Temporarily rename adjtime. 3. # dracut -f 4. # mv -i /etc/adjtime.BAK1 /etc/adjtime 5. # lsinitrd -f /etc/adjtime # Verify there is no adjtime in initramfs. 6. # reboot 7. # date # Verify the system clock is correct. Note that chronyd does not log any messages about stepping the system clock. Tested with a fresh, minimal install: $ qemu-img create f19-test-2.img 12G $ qemu-kvm -m 4096 -hda f19-test-2.img -cdrom ~/xfr/fedora/F19/Fedora-19-x86_64-DVD.iso -vga std -boot menu=on
This is also true if the RTC is on local time: 1. Change UTC to LOCAL in /etc/adjtime. 2. Reboot with the qemu option "-rtc base=localtime". 3. # date 4. # grep chronyd /var/log/messages If chronyd is disabled, in both cases (UTC or LOCAL), the system clock is also correct on booting: # systemctl disable chronyd Tested with: $ qemu-kvm -m 4096 -hda f19-test-2.img -cdrom ~/xfr/fedora/F19/Fedora-19-x86_64-DVD.iso -vga std -boot menu=on -rtc base=localtime
Harald Hoyer 2013-08-20 12:52:42 UTC Status: ASSIGNED → CLOSED Resolution: --- → RAWHIDE Last Closed: 2013-07-05 13:21:45 → 2013-08-20 08:52:42 I can't see a recent commit related to /etc/adjtime: http://git.kernel.org/cgit/boot/dracut/dracut.git/log/ The only commit related to /etc/adjtime is the one dated 2013-06-06 that installed /etc/adjtime and /etc/localtime to initramfs: http://git.kernel.org/cgit/boot/dracut/dracut.git/log/?qt=grep&q=%2Fetc%2Fadjtime
(In reply to Steve Tyler from comment #22) > Harald Hoyer 2013-08-20 12:52:42 UTC > Status: ASSIGNED → CLOSED > Resolution: --- → RAWHIDE > Last Closed: 2013-07-05 13:21:45 → 2013-08-20 08:52:42 > > I can't see a recent commit related to /etc/adjtime: > http://git.kernel.org/cgit/boot/dracut/dracut.git/log/ > > The only commit related to /etc/adjtime is the one dated 2013-06-06 that > installed /etc/adjtime and /etc/localtime to initramfs: > http://git.kernel.org/cgit/boot/dracut/dracut.git/log/ > ?qt=grep&q=%2Fetc%2Fadjtime commit d27cd4dfdd51c7f5178c5f4cb8f5bf4668228995 http://git.kernel.org/cgit/boot/dracut/dracut.git/commit/?id=d27cd4dfdd51c7f5178c5f4cb8f5bf4668228995
Thanks, Harald. Reverting the earlier commit is even better, but see below. I was searching commit messages for "/etc/adjtime", but a search for "adjtime" finds it: http://git.kernel.org/cgit/boot/dracut/dracut.git/log/?qt=grep&q=adjtime Andre: This removes /etc/adjtime _and_ /etc/localtime from initramfs.[1] This part of the commit may have some side-effects that we need to check: -# setup system time -if [ -f /etc/adjtime ]; then - if strstr "$(cat /etc/adjtime)" LOCAL; then - hwclock --hctosys --localtime - else - hwclock --hctosys --utc - fi -fi [1] Revert "base: setup correct system time and time zone in initrd" http://git.kernel.org/cgit/boot/dracut/dracut.git/commit/?id=d27cd4dfdd51c7f5178c5f4cb8f5bf4668228995
(In reply to Steve Tyler from comment #24) > This part of the commit may have some side-effects that we need to check: > > -# setup system time > -if [ -f /etc/adjtime ]; then > - if strstr "$(cat /etc/adjtime)" LOCAL; then > - hwclock --hctosys --localtime > - else > - hwclock --hctosys --utc > - fi > -fi Well, if no adjtime is in the initramfs that code fragment does not have any effect anyway.
Thanks for pointing that out: -if [ -f /etc/adjtime ]; then ^^ I saw "hwclock" and panicked ... :-)
Andre: This change is in dracut-032: http://git.kernel.org/cgit/boot/dracut/dracut.git/commit/?id=032