Bug 2013386

Summary: systemd-248.8-1.fc34 boot enters emergency mode
Product: [Fedora] Fedora Reporter: Sammy <sait.a.umar>
Component: systemdAssignee: systemd-maint
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 34CC: awilliam, fedoraproject, filbranden, flepied, kasong, kmansoft, lnykryn, msekleta, ssahani, s, systemd-maint, yuwatana, zbyszek
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-10-13 09:10:47 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
journalctl -b
none
boot.log none

Description Sammy 2021-10-12 17:53:58 UTC
Trying the systemd 248.8-1.fc34 from koji....the reboot enters the computer into the emergency mode but hitting ctrl-D results in normal boot. sddm comes up very quickly after ctrl-D so it must be failing somewhere before that. FYI.

Comment 1 Zbigniew Jędrzejewski-Szmek 2021-10-12 18:48:55 UTC
Do you have logs or other boot messages?

Comment 2 Sammy 2021-10-12 20:05:07 UTC
I did not see anything obvious....I can try again to see. What do you need?

Comment 3 Zbigniew Jędrzejewski-Szmek 2021-10-12 20:30:45 UTC
For start, 'journalctl -b' from a boot which entered emergency mode.

Comment 4 Sammy 2021-10-12 20:35:55 UTC
Created attachment 1832404 [details]
journalctl -b

Comment 5 Sammy 2021-10-12 20:36:37 UTC
Created attachment 1832405 [details]
boot.log

Comment 6 Sammy 2021-10-12 20:38:05 UTC
The boot.log from two different systemd (regular and emergency) did not show any difference.

Comment 7 Zbigniew Jędrzejewski-Szmek 2021-10-12 20:41:59 UTC
Hmm, maybe it's https://github.com/systemd/systemd-stable/commit/1f77dbfaae.
I'll look at this tomorrow.

Comment 8 Zbigniew Jędrzejewski-Szmek 2021-10-13 09:10:47 UTC
[  OK  ] Created slice Slice /system/configure-printer.
[FAILED] Failed to start Dispatch P…ts to Console Directory Watch.
See 'systemctl status systemd-ask-password-console.path' for details.
[  OK  ] Reached target Printer Support.
[FAILED] Failed to start Special ha… of early boot iSCSI sessions.
See 'systemctl status iscsi-onboot.service' for details.
[FAILED] Failed to start OSTree Remount OS/ Bind Mounts.
See 'systemctl status ostree-remount.service' for details.

So yeah, that commit seems to be the culprit. I thought the problem is because of one of the two first
units, but actually it's the third of those: ostree-remount.service has OnFailure=emergency.target,
so even though the unit would be silently skipped normally because of the missing start conditions,
we now enter emergency.target when it fails to start because of start limiting.

I'm not sure why we don't see the issue in the upstream. I installed a fresh F34 machine
to test this, and it doesn't reproduce. Anyway, I'll revert the patch in the v248-stable branch.

The update has already been obsoleted, so I'll close this.

Comment 9 Zbigniew Jędrzejewski-Szmek 2021-10-13 09:20:39 UTC
*** Bug 2013559 has been marked as a duplicate of this bug. ***

Comment 10 Kostya Vasilyev 2021-10-14 06:43:45 UTC
Updated to systemd-248.9-1.fc34.x86_64, seems OK, thanks!

Comment 11 Adam Williamson 2021-10-29 21:52:56 UTC
I backported the fix for Rawhide too, since it was also affected.