Bug 2223383

Summary:	systemd-sleep clock error on resume
Product:	[Fedora] Fedora	Reporter:	Patrick O'Callaghan <poc>
Component:	systemd	Assignee:	systemd-maint
Status:	CLOSED NOTABUG	QA Contact:	Fedora Extras Quality Assurance <extras-qa>
Severity:	medium	Docs Contact:
Priority:	unspecified
Version:	38	CC:	dtardon, fedoraproject, filbranden, lnykryn, msekleta, ryncsn, systemd-maint, yuwatana, zbyszek
Target Milestone:	---
Target Release:	---
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2023-07-24 08:46:07 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Patrick O'Callaghan 2023-07-17 14:18:17 UTC

On resuming after hibernate, initial journal entries show the hardware clock time because the local timezone has not been set. This means that journal entry timestamps are not reliably monotonic in all cases, and for some ranges could show time going backwards.

Reproducible: Always

Steps to Reproduce:
1.systemctl hibernate
2.power cycle
3.check journal entries
Actual Results:  
Where local time and hardware clock time differ, initial journal entries can be off until the TZ is properly set.

Expected Results:  
Journal entry timestamps should always be monotonically increasing, or clear warnings given when this is not the case (e.g. by flagging entries created before the locale is set).

Comment 1 Zbigniew Jędrzejewski-Szmek 2023-07-17 17:06:02 UTC

Hmm, please describe the issue, not your interpretation of underlying causes.

The entries in the journal are stored in UTC, without a timezone. Any time
zone offsets are applied during display. Similarly, the hardware clock also
doesn't know anything about time zones.

If entries are non-monotonic, this is usually because the system clock jumps,
for example because it was initialized inaccurately, and then later is adjusted
using NTP.

Thus: please make sure that your hwclock is set, has a working battery, and
that the system is configured to interpret the hwclock as UTC
('timedatectl' shows 'RTC in local TZ: no'.)

Comment 2 Patrick O'Callaghan 2023-07-18 10:19:38 UTC

(In reply to Zbigniew Jędrzejewski-Szmek from comment #1)
> Hmm, please describe the issue, not your interpretation of underlying causes.
>

I believe I did describe the issue, though I agree that my interpretation was wrong.

> The entries in the journal are stored in UTC, without a timezone. Any time
> zone offsets are applied during display. Similarly, the hardware clock also
> doesn't know anything about time zones.
>

I understand that.

> If entries are non-monotonic, this is usually because the system clock jumps,
> for example because it was initialized inaccurately, and then later is
> adjusted
> using NTP.
> 
> Thus: please make sure that your hwclock is set, has a working battery, and
> that the system is configured to interpret the hwclock as UTC
> ('timedatectl' shows 'RTC in local TZ: no'.)

I checked the RTC and indeed the 'RTC in local TZ' field was set to 'yes'. I reset it:
$ timedatectl 
[...]
System clock synchronized: yes
              NTP service: active
          RTC in local TZ: no

However this didn't fix the problem:
Jul 18 09:01:11 Bree chronyd[1049]: Selected source 162.159.200.1 (2.fedora.pool.ntp.org)
Jul 18 08:01:12 Bree systemd-journald[643]: Time jumped backwards, rotating.
Jul 18 08:01:11 Bree systemd-resolved[920]: Clock change detected. Flushing caches.
Jul 18 09:01:11 Bree chronyd[1049]: System clock wrong by -3599.682211 seconds
Jul 18 08:01:11 Bree rsyslogd[953]: imjournal: journal files changed, reloading...  [v8.2306.0-1.fc38 try https://www.rsyslog.com/e/0 ]
Jul 18 08:01:11 Bree chronyd[1049]: System clock was stepped by -3599.682211 seconds
Jul 18 08:01:11 Bree chronyd[1049]: System clock TAI offset set to 37 seconds

Comment 3 Zbigniew Jędrzejewski-Szmek 2023-07-18 11:24:12 UTC

Please attach the output of 'timedatectl' and 'journalctl -b'.

Comment 4 Patrick O'Callaghan 2023-07-20 09:47:09 UTC

Apologies for the delay as I was dealing with other issues.

After trying this again a couple of times, it appears that the fix does in fact work, i.e. the clock is being correctly set on power-up. I'm 99% sure I did test this correctly the first time and it didn't work, but I have to assume I was mistaken.

Thanks for the help.

Comment 5 Zbigniew Jędrzejewski-Szmek 2023-07-24 12:12:56 UTC

Phew, problem solved ;) Feel free to reopen if the issue pops up again.