Bug 1074002 - Fedora ARM usually miss time (depending on the device)
Summary: Fedora ARM usually miss time (depending on the device)
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: kernel
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Kernel Maintainer List
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: ARMTracker
TreeView+ depends on / blocked
 
Reported: 2014-03-07 16:09 UTC by Nicolas Chauvet (kwizart)
Modified: 2022-12-08 15:22 UTC (History)
13 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-05-31 09:30:01 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Nicolas Chauvet (kwizart) 2014-03-07 16:09:22 UTC
Description of problem:
With the current F20 kernel/userland components, Fedora on ARM is missing the time on some devices.

The reason for this issue is that most RTC drivers are compiled as an external kernel modules in a Fedora kernel. This lead to messages such as (on wandboard):
-----
déc. 31 19:00:02 localhost kernel: drivers/rtc/hctosys.c: unable to open rtc device (rtc0)
...
déc. 31 19:00:09 localhost kernel: snvs_rtc 20cc034.snvs-rtc-lp: rtc core: registered 20cc034.snvs-rtc-lp as rtc0
-----
In this kernel message, the hardware clock is read by the kernel 7 seconds before the module is loaded and is able to operate. The kernel doesn't read the clock then.


Some devices are developers boards that only grab time from the network or do not even have a battery backed rtc. But even in this case the rtc could save time across reboot (time will be lost from a cold boot).

This is a problem when one is not relying on a network. Mobility devices such as smartbooks are affected or when doing early boot analysis.


This is also a trivial feature to be expected on any fully backed device on primary architectures.


Version-Release number of selected component (if applicable):
Fedora 20

How reproducible:
It depends of the device

Steps to Reproduce:
1. Boot on any ARM device disconnected from the internet
2. Verify that you are not in 1/1/1970
3.

Actual results:
In some time the clock cannot be read and the board is back into the 70's

Expected results:
The date should be more current

Additional info:

There are few way to fix the issue that I imagine:
- Having most RTC builtin the kernel. The current drivers are around 600ko when built as a module, maybe the kernel can grow less in size when builtin.
The problem is that some sub-system (such as I2C, PMIC depending on how the rtc is wired) will requires to be builtin also in order for the driver to operate in-time.
> This will lead for the kernel to grow needlessly over-time.

- Having most drivers to be provided sooner in the initramfs and using:
hwclock --htctosys /dev/rtc0 to manually update time.
The main problem here is to have such service to start appropriately (after kernel-module.load and after the rtc driver is fully loaded (which is a little bit after kernel-module ends according to my tests).
And before the network and ntp can be made available.
-> The initramfs will grow, but probably from an acceptable size.

- Change the kernel to allow to update time right after the rtc driver is able to operate.
I think some patches were made available to "delay" the read of the rtc clock from the kernel. I cannot find the refs at this points.


Advices welcomed either here of the the fedora Kernel/ARM mail list

Comment 1 Bill Nottingham 2014-03-07 16:14:42 UTC
Moving to kernel, cc'ing dracut maintainer.

Comment 2 Thomas Müller 2014-10-26 07:52:51 UTC
Is there any news here?

Comment 3 Nicolas Chauvet (kwizart) 2014-10-26 11:02:11 UTC
(In reply to Thomas Müller from comment #2)
> Is there any news here?
No, but can you describe your case (hardware, how rtc is wired).
As I tried to explain, there are multiple cases why time wouldn't be accurate.


For exemple, in my case (ac100) I currently need to switch
CONFIG_I2C_TEGRA=y
CONFIG_RTC_DRV_TPS6586X=y
CONFIG_REGULATOR_TPS6586X=y

Comment 4 Thomas Müller 2014-10-26 11:28:59 UTC
(In reply to Nicolas Chauvet (kwizart) from comment #3)
> (In reply to Thomas Müller from comment #2)
> > Is there any news here?
> No, but can you describe your case (hardware, how rtc is wired).
> As I tried to explain, there are multiple cases why time wouldn't be
> accurate.

I'm running a CompuLab Utilite [1] and the default fedora kernel config compiles the required rtc driver (snvs_rtc) as a module. It is not available in time for the kernel to initialize the correct time so it defaults to 1970.
Currently I resort to compiling my own kernel and compile-in the driver (CONFIG_RTC_DRV_SNVS=y), but I would welcome a more generic solution that would allow me to use the packaged kernel.

[1] http://compulab.co.il/utilite-computer/web/home

Comment 5 Jaroslav Reznik 2015-03-03 15:33:09 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 22 development cycle.
Changing version to '22'.

More information and reason for this action is here:
https://fedoraproject.org/wiki/Fedora_Program_Management/HouseKeeping/Fedora22

Comment 6 Justin M. Forbes 2015-10-20 19:26:27 UTC
*********** MASS BUG UPDATE **************

We apologize for the inconvenience.  There is a large number of bugs to go through and several of them have gone stale.  Due to this, we are doing a mass bug update across all of the Fedora 22 kernel bugs.

Fedora 22 has now been rebased to 4.2.3-200.fc22.  Please test this kernel update (or newer) and let us know if you issue has been resolved or if it is still present with the newer kernel.

If you have moved on to Fedora 23, and are still experiencing this issue, please change the version to Fedora 23.

If you experience different issues, please open a new bug report for those.

Comment 7 Peter Hjalmarsson 2017-12-30 15:18:46 UTC
Hitted this as well, using an rpi2 and rpi3 with a ds3231 added to the GPIO-pins.

I have also with fdtput altered the DT so the module is loaded correctly during initramfs-mode:
fdtput -c /boot/dtb/bcm2836-rpi-2-b.dtb /soc/i2c@7e804000/ds3231@68
fdtput -t x /boot/dtb/bcm2836-rpi-2-b.dtb /soc/i2c@7e804000/ds3231@68 reg 0x00000068
fdtput -t s /boot/dtb/bcm2836-rpi-2-b.dtb /soc/i2c@7e804000/ds3231@68 compatible "maxim,ds3231"

From logs:
jul 12 16:01:10 mcp.dodi.nu kernel: hctosys: unable to open rtc device (rtc0)
jul 12 16:01:10 mcp.dodi.nu systemd[1]: System time before build time, advancing clock.
jul 12 16:01:10 mcp.dodi.nu systemd[1]: systemd 234 running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD -IDN2 +IDN default-hierarchy=hybrid)
jul 12 16:01:10 mcp.dodi.nu systemd[1]: Detected architecture arm.
jul 12 16:01:10 mcp.dodi.nu systemd[1]: Running in initial RAM disk.
jul 12 16:01:12 mcp.dodi.nu systemd[1]: Started udev Coldplug all Devices.
jul 12 16:01:13 mcp.dodi.nu kernel: rtc-ds1307 1-0068: registered as rtc0
jul 12 16:01:55 mcp.dodi.nu chronyd[545]: System clock wrong by 14771915.725473 seconds, adjustment started
dec 30 14:20:31 mcp.dodi.nu chronyd[545]: System clock was stepped by 14771915.725473 seconds

[root@node1 ~]# hwclock
2017-12-30 14:39:43.649786+0100

So the ds3231 is found correctly, as proven by it being found by udev Coldplug, and working as intended as shown by hwclock.

I currently running a workaround which works pretty good:
[root@mcp ~]# cat /etc/dracut.conf.d/rtc.conf 
install_items+="/usr/sbin/hwclock /etc/udev/rules.d/95-rtc-hotplug.rules"
[root@mcp ~]# cat /etc/udev/rules.d/95-rtc-hotplug.rules 
ACTION=="add", SUBSYSTEM="rtc", ATTRS{hctosys}=="0", RUN+="/usr/sbin/hwclock -s --utc"
[root@mcp ~]# dracut -f

Comment 8 Nicolas Chauvet (kwizart) 2020-05-12 16:02:31 UTC
With commit f9b2a4d6a5f18e0aaf715206a056565c56889d9f it's now possible to have rtc modules to update system time at the end of their registration.

So this leave the issue with non-battery backed cases:

1/ on cold boot, we need to store any information on last time (such as root partition last mounted time ?).

2/ on warm reboot, we could re-use the htctosyc if systohtc where enabled in the kernel (so the rtc would store the right time across reboots).

Comment 9 Peter Robinson 2020-05-19 12:36:35 UTC
> So this leave the issue with non-battery backed cases:
> 
> 1/ on cold boot, we need to store any information on last time (such as root
> partition last mounted time ?).

Systemd I believe already had a mechanism to do this. If memory serves it restores the time of the last access of the rootfs (which is basically when it was shut down). A quick search didn't give me details thouhg

> 2/ on warm reboot, we could re-use the htctosyc if systohtc where enabled in
> the kernel (so the rtc would store the right time across reboots).

We can enable that, actually not sure why it's disabled actually as it's "default=y".

We can also likely use UEFI time services if there's a meanns for U-Boot to restore the time to something sane (aka close to the last boot) or retrieve it from the UEFI run time services (which is served by ATF on aarch64 if memory serves).

Comment 10 Peter Robinson 2020-05-19 12:51:49 UTC
The other thing I forgot to mention was a number of the Arm devices have two RTCs, the one built into the SoC is often the default but also not the one that is battery backed, and in sometimes doesn't even have a crystal attched to ensure accuracy. We should ensure those devices have appropriate aliases in the DT to ensure the best one is rtc0.

Comment 11 Peter Robinson 2020-12-02 22:54:59 UTC
So this PR enables CONFIG_RTC_SYSTOHC as arm doesn't have update_persistent_clock hence why it doesn't sync. This will at least ensure devices with RTCs will keep time over reboots, even if not over shut down.
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/781

and this one enables CONFIG_RTC_NVMEM for RTCs that have non volatile memory for saving state.
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/782

Once they land this is as much as we can do from a kernel PoV WRT to this issue but with these and the kernel change (f9b2a4d6a5f) it's a lot better than it was.

Comment 12 Dusty Mabe 2022-12-08 15:22:31 UTC
(In reply to Peter Robinson from comment #9)
> > So this leave the issue with non-battery backed cases:
> > 
> > 1/ on cold boot, we need to store any information on last time (such as root
> > partition last mounted time ?).
> 
> Systemd I believe already had a mechanism to do this. If memory serves it
> restores the time of the last access of the rootfs (which is basically when
> it was shut down). A quick search didn't give me details thouhg

For anyone finding this in the future there are two mechanisms systemd has for this:

- systemd-timesyncd will do it automatically for you if you have it enabled (versus chronyd)
    - https://github.com/systemd/systemd/pull/20142
- There is also /usr/lib/clock-epoch
    - https://lists.freedesktop.org/archives/systemd-devel/2020-November/045646.html
    - https://terinstock.com/post/2021/12/Systemds-clock-epoch-for-RTC-less-systems/


Note You need to log in before you can comment on or make changes to this bug.