This service will be undergoing maintenance at 00:00 UTC, 2016-08-01. It is expected to last about 1 hours
Bug 985522 - RTC is not updated
RTC is not updated
Status: CLOSED ERRATA
Product: Fedora
Classification: Fedora
Component: kernel (Show other bugs)
19
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Kernel Maintainer List
Fedora Extras Quality Assurance
: Reopened
: 816752 (view as bug list)
Depends On:
Blocks: 1010351
  Show dependency treegraph
 
Reported: 2013-07-17 12:38 EDT by Chris Murphy
Modified: 2013-10-08 13:46 EDT (History)
9 users (show)

See Also:
Fixed In Version: kernel-3.11.2-301.fc20
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1010351 (view as bug list)
Environment:
Last Closed: 2013-09-30 21:57:38 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)
dmesg (127.69 KB, text/plain)
2013-07-17 12:40 EDT, Chris Murphy
no flags Details
[x1.strace] chronyd strace log (277.52 KB, text/plain)
2013-07-18 00:05 EDT, Steve Tyler
no flags Details
[ntpd-1.strace] ntpd strace log (172.26 KB, text/plain)
2013-07-18 03:16 EDT, Steve Tyler
no flags Details

  None (edit)
Description Chris Murphy 2013-07-17 12:38:13 EDT
Description of problem: The real time clock is not updated, even though the system clock is updated and correct.


Version-Release number of selected component (if applicable):
kernel-3.10.0-1.fc20.x86_64
chrony-1.28-0.1.pre1.fc19.x86_64

ntpdate and ntpd are not installed
Always when the RTC date is wrong by days/weeks/months.

Steps to Reproduce:
1. Set RTC to 2 months ago and reboot.
2. Wait 2 hours.

Actual results:
system clock is correctly updated by chrony very quickly (during startup, before the login prompt appears); the RTC is not ever updated. Not the date, nor the time, minute or seconds are updated.

As a consequence, a laptop with an incorrect RTC and intermittent internet connection will have variable date/time, due to the system clock being set to the RTC value absent an internet connection.

Expected results:
Expect the RTC to be updated relatively quickly.

Additional info:
[root@f19s ~]# uptime
 10:35:58 up  2:05,  1 user,  load average: 0.00, 0.01, 0.05

[root@f19s ~]# hwclock
Fri 17 May 2013 07:25:36 PM MDT  -0.032697 seconds

[root@f19s ~]# timedatectl
      Local time: Wed 2013-07-17 10:36:05 MDT
  Universal time: Wed 2013-07-17 16:36:05 UTC
        RTC time: Sat 2013-05-18 01:25:41
        Timezone: America/Denver (MDT, -0600)
     NTP enabled: yes
NTP synchronized: yes
 RTC in local TZ: no
      DST active: yes
 Last DST change: DST began at
                  Sun 2013-03-10 01:59:59 MST
                  Sun 2013-03-10 03:00:00 MDT
 Next DST change: DST ends (the clock jumps one hour backwards) at
                  Sun 2013-11-03 01:59:59 MDT
                  Sun 2013-11-03 01:00:00 MST
Comment 1 Chris Murphy 2013-07-17 12:40:04 EDT
Created attachment 774853 [details]
dmesg

DMI: Apple Inc. MacBookPro4,1
kernel 3.10.0-1

This hardware is booting in EFI mode (not with CSM), so this bug could be related to EFI.
Comment 2 Miroslav Lichvar 2013-07-17 13:06:52 EDT
I'm seeing this problem too.

Is it possible that the check in sync_cmos_clock() for the distance of the current time to the half of the second is too strict?

Would this help?

--- a/kernel/time/ntp.c
+++ b/kernel/time/ntp.c
@@ -485,7 +485,7 @@ static void sync_cmos_clock(struct work_struct *work)
        }
 
        getnstimeofday(&now);
-       if (abs(now.tv_nsec - (NSEC_PER_SEC / 2)) <= tick_nsec / 2) {
+       if (abs(now.tv_nsec - (NSEC_PER_SEC / 2)) <= 10000000) {
                struct timespec adjust = now;
 
                fail = -ENODEV;
Comment 3 Prarit Bhargava 2013-07-17 13:33:48 EDT
(In reply to Miroslav Lichvar from comment #2)
> I'm seeing this problem too.
> 
> Is it possible that the check in sync_cmos_clock() for the distance of the
> current time to the half of the second is too strict?
> 
> Would this help?
> 
> --- a/kernel/time/ntp.c
> +++ b/kernel/time/ntp.c
> @@ -485,7 +485,7 @@ static void sync_cmos_clock(struct work_struct *work)
>         }
>  
>         getnstimeofday(&now);
> -       if (abs(now.tv_nsec - (NSEC_PER_SEC / 2)) <= tick_nsec / 2) {
> +       if (abs(now.tv_nsec - (NSEC_PER_SEC / 2)) <= 10000000) {
>                 struct timespec adjust = now;
>  
>                 fail = -ENODEV;

3.11 will include a fix for this.  See upstream commit 3195ef59cb42cda3aeeb24a7fd2ba1b900c4a3cc

P.
Comment 4 Prarit Bhargava 2013-07-17 13:37:59 EDT
Actually, my previous comment is incorrect as

[prarit@prarit linux-2.6]$ git-describe 3195ef59cb42cda3aeeb24a7fd2ba1b900c4a3cc
v3.9-rc2-115-g3195ef5

This isn't a kernel issue however.  It works with (the now deprecated in favor of chrony) ntpdate.

Re-opening, assigning to chrony.

P.
Comment 5 Steve Tyler 2013-07-17 13:56:17 EDT
(In reply to Prarit Bhargava from comment #4)
...
> This isn't a kernel issue however.  It works with (the now deprecated in
> favor of chrony) ntpdate.
...

Oh, now I understand ... :-)

There are *two* packages -- ntp and ntpdate. They are different.

For testing, you need the ntp package or the chrony package, but not both.
Do not install ntpdate.

$ sudo repoquery ntp ntpdate chrony
chrony-0:1.28-0.1.pre1.fc19.x86_64
ntp-0:4.2.6p5-11.fc19.x86_64
ntpdate-0:4.2.6p5-11.fc19.x86_64
Comment 6 Steve Tyler 2013-07-17 14:32:00 EDT
I guess I should have checked this first, but installing ntp pulls in ntpdate. That may or may not be good packaging, but it is confusing for testing:

$ sudo repoquery --whatrequires ntpdate
anaconda-0:19.30.13-1.fc19.x86_64
nagios-plugins-ntp-perl-0:1.4.16-7.fc19.x86_64
ntp-0:4.2.6p5-11.fc19.x86_64

BTW: chrony replaced ntp in F16:

3.8.2. Chrony
Fedora 16 uses Chrony as the default Network Time Protocol (NTP) client.
http://docs.fedoraproject.org/en-US/Fedora/16/html/Release_Notes/sect-Release_Notes-Changes_for_Sysadmin.html
Comment 7 Chris Murphy 2013-07-17 14:49:41 EDT
http://chrony.tuxfamily.org/FAQ.html#question_9.3
http://chrony.tuxfamily.org/manual.html#rtcfile-directive

This implies that the current /etc/chrony.conf setting, rtcsync, is the problem. That directive tells chronyd that the kernel will set the RTC every 11 minutes. Yet this is clearly not happening with respect to minutes/seconds (let alone hours, or date).

Also, I don't know which kernel config option is referenced in 9.2 by "enhanced RTC support compiled into the kernel".

Further, http://chrony.tuxfamily.org/manual.html#rtconutc-directive indicates that by default, without the rtconoutc directive the RTC is assumed to be in local time. This is consistent with the UEFI spec, and Windows, but inconsistent with most all linux, unix, BSD and OS X.

So the proper handling of time is actually annoyingly difficult.
Comment 8 Steve Tyler 2013-07-17 15:05:28 EDT
Chris: Could you rerun your kernel 3.10.0-1 test with a smaller time difference between UTC and the RTC? (Say, +/-10 minutes)

If rtcsync is functioning, there should be an RTC update after 11 minutes, but it could be that the kernel is more cautious with extreme time differences. (See Bug 816752, Comment 79, which reports on a 20 minute difference and an adjustment to the nearest 30 minutes after 11 minutes.)
Comment 9 Steve Tyler 2013-07-17 15:43:37 EDT
(In reply to Steve Tyler from comment #6)
> ... installing ntp pulls in ntpdate. ...

To test with ntp, install the ntp package and enable the ntpd service, but not the ntpdate service. Remove the chrony package, or disable the chronyd service. Reboot.

# systemctl disable chronyd
# systemctl enable ntpd
# reboot
# systemctl list-unit-files | egrep 'ntp|chrony'
# ps -ef | egrep 'ntp|chrony'
Comment 10 Chris Murphy 2013-07-17 16:04:28 EDT
- Set RTC to be ~5m32s behind actual time.
- Reboot without an internet connection. Confirm in GRUB with date command clock is UTC minus 5m32s. Proceed with linux boot, confirm system clock and RTC are the same.
- Reconnect internet. About 8 minutes later, chronyd steps the system clock to the correct time. RTC is now 5m32s behind system clock.
- After 32 minutes uptime, RTC is still 5m32s behind the system clock.


[root@f19s ~]# uptime
 14:03:16 up 32 min,  2 users,  load average: 0.01, 0.02, 0.05

[root@f19s ~]# timedatectl
      Local time: Wed 2013-07-17 14:03:18 MDT
  Universal time: Wed 2013-07-17 20:03:18 UTC
        RTC time: Wed 2013-07-17 19:58:47
        Timezone: America/Denver (MDT, -0600)
     NTP enabled: yes
NTP synchronized: yes
 RTC in local TZ: no
--snip DST info--

[root@f19s ~]# journalctl -b -u chronyd
-- Logs begin at Sun 2013-06-30 14:26:46 MDT, end at Wed 2013-07-17 14:03:18 MDT. --
Jul 17 13:26:49 f19s.local chronyd[293]: chronyd version 1.28-pre1 starting
Jul 17 13:26:56 f19s.local chronyd[293]: Linux kernel major=3 minor=10 patch=0
Jul 17 13:26:56 f19s.local chronyd[293]: hz=100 shift_hz=7 freq_scale=1.00000000 nominal_tick=10000 slew_delta_tick=833 max_tick_bias=1000 shift_pll=2
Jul 17 13:26:56 f19s.local chronyd[293]: Frequency -17.882 +/- 0.918 ppm read from /var/lib/chrony/drift
Jul 17 13:26:57 f19s.local systemd[1]: Started NTP client/server.
Jul 17 13:30:20 f19s.local chronyd[293]: Selected source 169.229.70.183
Jul 17 13:30:20 f19s.local chronyd[293]: System clock wrong by 272.190594 seconds, adjustment started
Jul 17 13:34:52 f19s.local chronyd[293]: System clock was stepped by 272.191 seconds
Jul 17 13:34:54 f19s.local chronyd[293]: Selected source 198.55.111.50
Comment 11 Steve Tyler 2013-07-17 16:23:04 EDT
(In reply to Chris Murphy from comment #10)
...
> - After 32 minutes uptime, RTC is still 5m32s behind the system clock.
...

Thanks, Chris. This is starting to look like a kernel regression.
Can you reproduce your result with kernel-3.9.9-302.fc19.x86_64?

NB: I'm not being lazy about checking this myself, but since you are in the saddle and riding herd ... :-)
Comment 12 Chris Murphy 2013-07-17 16:50:19 EDT
With kernel-3.9.9-302.fc19.x86_64.rpm the RTC is not updated after 15 minutes.


[root@f19s ~]# uptime
 14:49:37 up 17 min,  2 users,  load average: 0.00, 0.03, 0.05

[root@f19s ~]# timedatectl
      Local time: Wed 2013-07-17 14:49:38 MDT
  Universal time: Wed 2013-07-17 20:49:38 UTC
        RTC time: Wed 2013-07-17 20:45:06
        Timezone: America/Denver (MDT, -0600)
     NTP enabled: yes
NTP synchronized: yes
 RTC in local TZ: no
--snipped DST info--
Comment 13 Steve Tyler 2013-07-17 17:15:17 EDT
Confirmed with kernel 3.9.9-302.fc19.i686.
After 24 minutes, the RTC is still about 4 minutes behind local time.

The kernel must have another RTC-sync threshold. Prarit?

[root@spruce ~]# uptime
 13:58:58 up 24 min,  4 users,  load average: 0.10, 0.08, 0.12
[root@spruce ~]# timedatectl
      Local time: Wed 2013-07-17 13:59:02 PDT
  Universal time: Wed 2013-07-17 20:59:02 UTC
        RTC time: Wed 2013-07-17 13:55:00
        Timezone: America/Los_Angeles (PDT, -0700)
     NTP enabled: yes
NTP synchronized: yes
 RTC in local TZ: yes
      DST active: yes
 Last DST change: DST began at
                  Sun 2013-03-10 01:59:59 PST
                  Sun 2013-03-10 03:00:00 PDT
 Next DST change: DST ends (the clock jumps one hour backwards) at
                  Sun 2013-11-03 01:59:59 PDT
                  Sun 2013-11-03 01:00:00 PST

Warning: The RTC is configured to maintain time in the local time zone. This
         mode is not fully supported and will create various problems with time
         zone changes and daylight saving adjustments. If at all possible use
         RTC in UTC, by calling 'timedatectl set-local-rtc 0'.
[root@spruce ~]# ps -ef | grep chrony
chrony     301     1  0 13:34 ?        00:00:00 /usr/sbin/chronyd -u chrony
root      1689  1572  0 13:59 pts/1    00:00:00 grep --color=auto chrony
[root@spruce ~]# uname -r
3.9.9-302.fc19.i686
[root@spruce ~]# rpm -q chrony
chrony-1.28-0.1.pre1.fc19.i686
[root@spruce ~]#
Comment 14 Steve Tyler 2013-07-17 18:13:03 EDT
With the RTC about 10 minutes behind local time, an RTC update occurs after about 11 minutes.

Tested with kernel 3.9.9-302.fc19.i686.
Shell script to test for RTC updates is below.

===
Wed Jul 17 14:38:32 PDT 2013
 14:38:32 up 11 min,  3 users,  load average: 0.05, 0.20, 0.23
      Local time: Wed 2013-07-17 14:38:32 PDT
  Universal time: Wed 2013-07-17 21:38:32 UTC
        RTC time: Wed 2013-07-17 14:28:26
===
Wed Jul 17 14:39:32 PDT 2013
 14:39:32 up 12 min,  3 users,  load average: 0.02, 0.16, 0.22
      Local time: Wed 2013-07-17 14:39:32 PDT
  Universal time: Wed 2013-07-17 21:39:32 UTC
        RTC time: Wed 2013-07-17 14:39:32
===

$ cat rtc-test-2.sh 
#!/bin/bash

sudo true

while true; do
  echo '==='
  date
  uptime
  #grep rtc /proc/driver/rtc
  sudo timedatectl | egrep '(Local|Universal|RTC) time'
  sleep 1m
done
Comment 15 Chris Murphy 2013-07-17 19:19:20 EDT
With 3.9.9-302.fc19.x86_64, after 2 hours, an RTC 5m32s behind real time is still not updated by the kernel. Is it worth testing 3.11.0-0.rc1.git1.1.fc20?
Comment 16 Prarit Bhargava 2013-07-17 19:41:39 EDT
(In reply to Steve Tyler from comment #13)
> Confirmed with kernel 3.9.9-302.fc19.i686.
> After 24 minutes, the RTC is still about 4 minutes behind local time.
> 
> The kernel must have another RTC-sync threshold. Prarit?

Nope, there's nothing in the kernel that prevents the time from being updated.  It might be interesting to instrument the kernel to see exactly what chrony is doing... I'll put it on my list of things to do ;)

> 
> [root@spruce ~]# uptime
>  13:58:58 up 24 min,  4 users,  load average: 0.10, 0.08, 0.12
> [root@spruce ~]# timedatectl
>       Local time: Wed 2013-07-17 13:59:02 PDT
>   Universal time: Wed 2013-07-17 20:59:02 UTC
>         RTC time: Wed 2013-07-17 13:55:00
>         Timezone: America/Los_Angeles (PDT, -0700)
>      NTP enabled: yes
> NTP synchronized: yes
>  RTC in local TZ: yes
>       DST active: yes
>  Last DST change: DST began at
>                   Sun 2013-03-10 01:59:59 PST
>                   Sun 2013-03-10 03:00:00 PDT
>  Next DST change: DST ends (the clock jumps one hour backwards) at
>                   Sun 2013-11-03 01:59:59 PDT
>                   Sun 2013-11-03 01:00:00 PST
> 
> Warning: The RTC is configured to maintain time in the local time zone. This
>          mode is not fully supported and will create various problems with
> time
>          zone changes and daylight saving adjustments. If at all possible use
>          RTC in UTC, by calling 'timedatectl set-local-rtc 0'.
> [root@spruce ~]# ps -ef | grep chrony
> chrony     301     1  0 13:34 ?        00:00:00 /usr/sbin/chronyd -u chrony
> root      1689  1572  0 13:59 pts/1    00:00:00 grep --color=auto chrony
> [root@spruce ~]# uname -r
> 3.9.9-302.fc19.i686
> [root@spruce ~]# rpm -q chrony
> chrony-1.28-0.1.pre1.fc19.i686
> [root@spruce ~]#
Comment 17 Steve Tyler 2013-07-17 22:01:54 EDT
(In reply to Chris Murphy from comment #15)
> With 3.9.9-302.fc19.x86_64, after 2 hours, an RTC 5m32s behind real time is
> still not updated by the kernel. Is it worth testing
> 3.11.0-0.rc1.git1.1.fc20?

Until we get a few convincing test cases with the 3.9.9-302 kernel, I would say not.

AIUI, we should be seeing RTC syncs if the RTC offset is within a +/-15 minute window. Maybe we should follow Prarit and try testing with ntp ...
Comment 18 Steve Tyler 2013-07-17 22:32:32 EDT
(In reply to Prarit Bhargava from comment #16)
> (In reply to Steve Tyler from comment #13)
...
> > The kernel must have another RTC-sync threshold. Prarit?
> 
> Nope, there's nothing in the kernel that prevents the time from being
> updated.  It might be interesting to instrument the kernel to see exactly
> what chrony is doing... I'll put it on my list of things to do ;)
...

Thanks, Prarit. That's a good suggestion.

strace would probably suffice for chrony.

I believe systemtap can be used on the kernel.
Is sync_cmos_clock() the function we should be looking at?

$ less -N linux-3.9.9/kernel/time/ntp.c
...
    492 static void sync_cmos_clock(struct work_struct *work)
    493 {
    494         struct timespec now, next;
    495         int fail = 1;
    496 
    497         /*
    498          * If we have an externally synchronized Linux clock, then update
    499          * CMOS clock accordingly every ~11 minutes. Set_rtc_mmss() has to be
    500          * called as close as possible to 500 ms before the new second starts.
    501          * This code is run on a timer.  If the clock is set, that timer
    502          * may not expire at the correct time.  Thus, we adjust...
    503          */
    504         if (!ntp_synced()) {
    505                 /*
    506                  * Not synced, exit, do not restart a timer (if one is
    507                  * running, let it run out).
    508                  */
    509                 return;
    510         }
...
Comment 19 Steve Tyler 2013-07-17 22:41:00 EDT
(In reply to Steve Tyler from comment #18)
...
> $ less -N linux-3.9.9/kernel/time/ntp.c
...

I'm not really qualified to be reading kernel code, but it looks like the branch at line 504 could bypass an RTC sync ...
Comment 20 Steve Tyler 2013-07-17 22:45:15 EDT
(In reply to Steve Tyler from comment #18)
...
> strace would probably suffice for chrony.
...

This could take a while ... :-)

$ sudo strace -f -ttt -p `pgrep chrony`
Process 443 attached
1374115401.977890 select(6, [1 2 3 5], NULL, NULL, {122, 593258}
Comment 21 Steve Tyler 2013-07-17 23:48:10 EDT
Here is a preliminary result:

$ sudo systemctl stop chronyd
$ sudo strace -f -ttt -o x1.strace /usr/sbin/chronyd -u chrony &

$ grep STA_UNSYNC x1.strace
13973 1374118138.520020 adjtimex({modes=0x100 /* ADJ_??? */, offset=0, freq=2386352, maxerror=16000000, esterror=16000000, status=STA_PLL|STA_UNSYNC|STA_FREQHOLD|STA_NANO, constant=0, precision=1, tolerance=32768000, time={1374118138, 520051619}, tick=10000, ppsfreq=0, jitter=0, shift=0, stabil=0, jitcnt=0, calcnt=0, errcnt=0, stbcnt=0}) = 5 (TIME_ERROR)

So STA_UNSYNC is set at chronyd startup. Is that expected?

$ grep -i rtc /etc/chrony.conf 
# Enable kernel RTC synchronization.
rtcsync

chrony-1.28-0.1.pre1.fc19.x86_64
kernel 3.9.9-302.fc19.x86_64
Comment 22 Steve Tyler 2013-07-18 00:05:46 EDT
Created attachment 775041 [details]
[x1.strace] chronyd strace log

$ sudo systemctl stop chronyd
$ sudo strace -f -ttt -o x1.strace /usr/sbin/chronyd -u chrony &

I manually killed chronyd after about 28 minutes:
$ sudo kill 13973
Comment 23 Steve Tyler 2013-07-18 00:14:31 EDT
(In reply to Steve Tyler from comment #21)
> Here is a preliminary result:
..
> So STA_UNSYNC is set at chronyd startup. Is that expected?
...

Clarification: In x1.strace, adjtimex() is called once with STA_UNSYNC set and 297 times with STA_UNSYNC cleared:

$ grep adjtimex x1.strace | grep -v STA_UNSYNC | wc -l
297
Comment 24 Steve Tyler 2013-07-18 02:07:16 EDT
(In reply to Chris Murphy from comment #10)
> - Set RTC to be ~5m32s behind actual time.
...

Here is a way to accurately set the RTC to the wrong time that I got from Miroslav (Bug 816752, Comment 74):

$ sudo hwclock --set --date '-1min'

For consistent testing, it might be a good idea to stop chronyd before and restart it after.
Comment 25 Miroslav Lichvar 2013-07-18 02:42:43 EDT
As you can see in the strace output (or with the ntptime tool), the adjtimex status has the STA_UNSYNC flag cleared, which tells the kernel to sync the RTC. This is a kernel problem, it's not in chronyd or ntpd.

The upstream commit 3195ef59cb42cda3aeeb24a7fd2ba1b900c4a3cc is about setting the RTC time fully, not just in the closest +/- 15 minute range. That was discussed in the bug #816752 and it seems to be fixed in the 3.10 kernel (when it actually does the sync).

This bug is about not setting the RTC at all. I suspect the kernel is trying to sync the RTC, but constantly failing the half second check.

Can someone please try the patch from the comment #2 and see if that is the case?
Comment 26 Steve Tyler 2013-07-18 03:16:49 EDT
Created attachment 775155 [details]
[ntpd-1.strace] ntpd strace log

With ntpd running instead of chronyd and with the RTC 1 minute behind local time, the RTC was eventually updated. The attached strace log shows that adjtimex() is called by ntpd, sometimes with STA_UNSYNC set and sometimes with STA_UNSYNC clear.

$ sudo strace -f -tt -o x3.strace -p `pgrep ntpd` &

ntp-4.2.6p5-11.fc19.i686
kernel 3.9.9-302.fc19.i686

NB: This log was captured on a different system from the previously attached x1.strace log.

I used the strace "-tt" option instead of "-ttt", because the timestamps are easier to correlate with wall time.
Comment 27 Steve Tyler 2013-07-18 03:33:13 EDT
Here are some differences in the two files.
Notably, chronyd always sets STA_FREQHOLD.

chronyd:

$ cat x1.strace | wc -l
2118

$ grep adjtimex x1.strace | wc -l
298

$ grep STA_FREQHOLD x1.strace | wc -l
298

ntpd:

$ cat ntpd-1.strace | wc -l
1521

$ grep adjtimex ntpd-1.strace | wc -l
9

$ grep STA_FREQHOLD ntpd-1.strace | wc -l
0
Comment 28 Miroslav Lichvar 2013-07-18 04:05:22 EDT
BTW, the STA_UNSYNC flag can be also unset with the ntptime tool from the ntp package like this:

ntptime -s 1 -o 0 -m 0
Comment 29 Steve Tyler 2013-07-18 08:25:59 EDT
(In reply to Miroslav Lichvar from comment #2)
> I'm seeing this problem too.
> 
> Is it possible that the check in sync_cmos_clock() for the distance of the
> current time to the half of the second is too strict?
...

This commit, dated 2013-01-16, changes sync_cmos_clock():

NTP: Add a CONFIG_RTC_SYSTOHC configuration
https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/commit/?id=023f333a99cee9b5cd3268ff87298eb01a31f78e

$ grep CONFIG_GENERIC_CMOS_UPDATE config-3.9.9-302.fc19.x86_64
CONFIG_GENERIC_CMOS_UPDATE=y
$ grep CONFIG_RTC_SYSTOHC config-3.9.9-302.fc19.x86_64
# CONFIG_RTC_SYSTOHC is not set

Chris: Do you have a kernel from before this change?
Comment 30 Steve Tyler 2013-07-18 09:09:56 EDT
(In reply to Steve Tyler from comment #29)
...
> This commit, dated 2013-01-16, changes sync_cmos_clock():
...
> Chris: Do you have a kernel from before this change?

3.6.10-4 is the F18 release kernel, dated 2012-12-11:
https://admin.fedoraproject.org/updates/FEDORA-2012-20200/kernel-3.6.10-4.fc18

$ sudo repoquery kernel --releasever=18 --show-dupes
kernel-0:3.6.10-4.fc18.x86_64
kernel-0:3.9.9-201.fc18.x86_64
Comment 31 Steve Tyler 2013-07-18 09:53:48 EDT
The RTC is not updated with F18 kernel 3.6.10-4 after ~60 minutes:

Procedure:

Minimal install from F18 DVD in a VM.[1]
# yum install chrony
# yum update
Reboot with kernel 3.6.10-4.

# hwclock --set --date '-1 min'
# watch -n 1 'uptime; timedatectl | grep time'

rtcsync is set in /etc/chrony.conf.

chrony-0:1.27-0.5.pre1.git1ca844.fc18.x86_64
kernel-0:3.6.10-4.fc18.x86_64

[1] VM commands:
$ qemu-img create f18-test-2.img 12G
$ qemu-kvm -m 4096 -hda f18-test-2.img -cdrom ~/xfr/fedora/F18/Fedora-18-x86_64-DVD.iso -vga std -boot menu=on
Comment 32 Chris Murphy 2013-07-18 10:10:24 EDT
I have only tested this on baremetal because I don't understand the possible interaction between host and guest in a VM context.

The 15 second sync was working with a prior kernel, I just don't remember which. Uncertain if it's OK to regress to an F18 3.7.x, 3.8.x series on an F19 install?
Comment 33 Steve Tyler 2013-07-18 11:20:41 EDT
(In reply to Chris Murphy from comment #32)
> I have only tested this on baremetal because I don't understand the possible
> interaction between host and guest in a VM context.

Good point.

The qemu man page describes the "-rtc" option, which allows the RTC to be initialized to any time ("utc", "localtime", other). I have verfied that "localtime" works as expected.  I have not tested the "clock=host|vm" suboption.

The hwclock command works as expected.

The VM RTC is preserved through a VM reboot, and the system time is set accordingly.

The main problem with a VM RTC is that it gets reset every time qemu is restarted (with the default options).

What interactions are you concerned about?

> The 15 second sync was working with a prior kernel, I just don't remember
> which. Uncertain if it's OK to regress to an F18 3.7.x, 3.8.x series on an
> F19 install?

I have been running an updated, minimal F17 install in a VM with kernel 3.3.4-5 for about 45 minutes, and the RTC has not been updated. chronyd is running and rtcsync is set in /etc/chrony.conf, so I believe the install is correctly configured. The RTC is 1 minute behind UTC.

Monitoring is being done with:
# watch -n 1 'uptime; date; date -u; grep rtc /proc/driver/rtc'
(timedatectl is not in F17.)
Comment 34 Steve Tyler 2013-07-18 11:30:03 EDT
(In reply to Steve Tyler from comment #33)
...
> I have been running an updated, minimal F17 install in a VM with kernel
> 3.3.4-5 for about 45 minutes, and the RTC has not been updated. chronyd is
> running and rtcsync is set in /etc/chrony.conf, so I believe the install is
> correctly configured. The RTC is 1 minute behind UTC.
...

chronyd slewed the system time after a VM reboot, and now the system time is correct, but the RTC is still 1 minute behind UTC.
Comment 35 Miroslav Lichvar 2013-07-18 11:39:09 EDT
This is probably a duplicate of the bug #753487.
Comment 36 Chris Murphy 2013-07-18 12:48:27 EDT
(In reply to Steve Tyler from comment #33)
> What interactions are you concerned about?
Reality conflicting with my possibly flawed expectations; user error due to additional complexity; bugs.

Chrony's plus over ntpd, is a more accurate system clock even when there isn't an internet connection. It seems to me it'd need some decently long period of time where the RTC isn't being sync'd to the system clock, in order for chronyd to develop an accurate drift calculation. I'd think it might even need a few days of this, potentially allowing a lot of drift to develop.

However, absent any drift data (new install), it seems initially chronyd should rather quickly sync a correct (from network time) system clock to the RTC. Then disable the syncing, and start tracking the drift. So far, either a 5 minute wrong or 2 month wrong RTC is not being set correctly.
Comment 37 Chris Murphy 2013-07-18 13:04:03 EDT
My wording in the last paragraph is confusing. I mean alter the RTC to match the (network time) corrected system clock, SYSTOHC. 

With enough drift data, and without an internet connection, I expect chronyd to take RTC + drift correction -> system clock within a minute of boot.
Comment 38 Miroslav Lichvar 2013-07-18 13:22:06 EDT
(In reply to Chris Murphy from comment #37)
> My wording in the last paragraph is confusing. I mean alter the RTC to match
> the (network time) corrected system clock, SYSTOHC. 

One problem with RTC is that its frequency can't be adjusted, so it's always drifting away.
 
> With enough drift data, and without an internet connection, I expect chronyd
> to take RTC + drift correction -> system clock within a minute of boot.

I think what you describe is the rtcfile option (incompatible with rtcsync), the -s option and the trimrtc command. In order for this feature to be reliable, nothing else must be touching the RTC. This makes it difficult to enable it by default.
Comment 39 Steve Tyler 2013-07-18 14:24:45 EDT
In a VM, the RTC may not drift at all, if the host is synced with a time server. The qemu man page says under the "-rtc" option:

"By default the RTC is driven by the host system time. This allows to use the RTC as accurate reference clock inside the guest, specifically if the host time is smoothly following an accurate external reference clock, e.g. via NTP. ..."
Comment 40 Chris Murphy 2013-07-18 18:19:48 EDT
Well I'm sufficiently confused by how it should work and probably need to read some more. The one thing I'm reasonably certain about is that a wrong RTC should be fixed by a new installation of Fedora once the computer has an internet connection - so that subsequent boots without an internet connection aren't wrong; and currently it isn't working that way.
Comment 41 Miroslav Lichvar 2013-08-02 03:32:29 EDT
The patch from comment #2 fixes the problem for me. An upstream submission is here: https://lkml.org/lkml/2013/8/1/416
Comment 42 Josh Boyer 2013-09-18 16:35:52 EDT
*********** MASS BUG UPDATE **************

We apologize for the inconvenience.  There is a large number of bugs to go through and several of them have gone stale.  Due to this, we are doing a mass bug update across all of the Fedora 19 kernel bugs.

Fedora 19 has now been rebased to 3.11.1-200.fc19.  Please test this kernel update and let us know if you issue has been resolved or if it is still present with the newer kernel.

If you experience different issues, please open a new bug report for those.
Comment 43 Miroslav Lichvar 2013-09-19 05:17:10 EDT
Not fixed yet, the patch might be included in 3.12.
Comment 44 Josh Boyer 2013-09-20 10:54:54 EDT
(In reply to Miroslav Lichvar from comment #43)
> Not fixed yet, the patch might be included in 3.12.

It's queued for 3.13 here:

http://git.linaro.org/gitweb?p=people/jstultz/linux.git;a=commit;h=a97ad0c4b447a132a322cedc3a5f7fa4cab4b304

I'll look at bringing it in shortly.
Comment 45 Fedora Update System 2013-09-27 17:42:54 EDT
kernel-3.11.2-201.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/kernel-3.11.2-201.fc19
Comment 46 Chris Murphy 2013-09-28 13:19:29 EDT
Works for me. 

Fedora 20
chrony-1.29-1.fc20.x86_64
kernel-3.11.2-301.fc20.x86_64

RTC set with YYMMDD HHMMSS all different from the system clock values. Within 10-15 minutes the RTC had been reset back to the correct time. Only tested with an internet connection.
Comment 47 Fedora Update System 2013-09-28 21:22:10 EDT
Package kernel-3.11.2-201.fc19:
* should fix your issue,
* was pushed to the Fedora 19 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing kernel-3.11.2-201.fc19'
as soon as you are able to, then reboot.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2013-17865/kernel-3.11.2-201.fc19
then log in and leave karma (feedback).
Comment 48 Fedora Update System 2013-09-30 12:28:25 EDT
kernel-3.11.2-301.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/kernel-3.11.2-301.fc20
Comment 49 Fedora Update System 2013-09-30 21:57:38 EDT
kernel-3.11.2-201.fc19 has been pushed to the Fedora 19 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 50 Fedora Update System 2013-10-02 02:36:30 EDT
kernel-3.11.2-301.fc20 has been pushed to the Fedora 20 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 51 Josh Boyer 2013-10-08 13:46:28 EDT
*** Bug 816752 has been marked as a duplicate of this bug. ***

Note You need to log in before you can comment on or make changes to this bug.