Bug 2061528

Summary: Fedora IoT - Failed to start systemd-userdbd.service - User Database Manager
Product: [Fedora] Fedora Reporter: Geoffrey Marr <gmarr>
Component: systemdAssignee: systemd-maint
Status: CLOSED INSUFFICIENT_DATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 36CC: dtardon, fedoraproject, filbranden, flepied, lnykryn, msekleta, robatino, ryncsn, ssahani, s, systemd-maint, yuwatana, zbyszek
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: RejectedBlocker AcceptedFreezeException
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-08-26 07:50:23 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1269538, 1953784    
Attachments:
Description Flags
output of 'journalctl -a | grep systemd-userdbd'
none
output of 'journalctl -a' none

Description Geoffrey Marr 2022-03-07 18:58:40 UTC
Created attachment 1864419 [details]
output of 'journalctl -a | grep systemd-userdbd'

Description of problem:
On the first boot of Fedora-IoT-36-20220306.0 (any arch), systemd-userdbd.service fails to start. On subsequent boots, the service starts and runs OK.


Version-Release number of selected component (if applicable):
Fedora-IoT-36-20220306.0

How reproducible:
Every time

Steps to Reproduce:
1. Install Fedora-IoT-36-20220306.0 any arch
2. 'systemctl --all --failed'
3. View failed systemd-userdbd.service

Additional info:
See attached logs.

Comment 1 Geoffrey Marr 2022-03-07 19:00:50 UTC
Created attachment 1864420 [details]
output of 'journalctl -a'

Comment 2 Fedora Blocker Bugs Application 2022-03-07 19:03:19 UTC
Proposed as a Blocker for 36-final by Fedora user coremodule using the blocker tracking app because:

 Proposing as a possible violation of the following F36 Final criterion:

"All system services present after installation with one of the release-blocking package sets must start properly, unless they require hardware which is not present." [0]

[0] https://fedoraproject.org/wiki/Fedora_36_Final_Release_Criteria#System_services

Comment 3 Geoffrey Marr 2022-03-07 22:44:35 UTC
Discussed during the 2022-03-07 blocker review meeting: [0]

The decision to classify this bug as a "RejectedBlocker (Final)" and an "AcceptedFreezeException (Beta)" was made as this seems to have no practical consequences and works on every boot after the first, so we decided it's not a sufficient violation of the criterion to block the release on. However it is confounding to see on first boot, so we grant an FE to fix that for Beta if possible.

[0] https://meetbot.fedoraproject.org/fedora-blocker-review/2022-03-07/f36-blocker-review.2022-03-07-17.01.txt

Comment 4 Zbigniew Jędrzejewski-Szmek 2022-03-23 12:22:43 UTC
Mar 07 11:18:32 fedora audit: BPF prog-id=61 op=LOAD
Mar 07 11:18:32 fedora systemd[1]: Starting systemd-userdbd.service - User Database Manager...
Mar 07 11:18:32 fedora systemd[668]: systemd-userdbd.service: Failed to set up mount namespacing: /run/systemd/unit-root/dev: Read-only file system
Mar 07 11:18:32 fedora systemd[668]: systemd-userdbd.service: Failed at step NAMESPACE spawning /usr/lib/systemd/systemd-userdbd: Read-only file system
Mar 07 11:18:32 fedora systemd[1]: systemd-userdbd.service: Main process exited, code=exited, status=226/NAMESPACE
Mar 07 11:18:32 fedora systemd[1]: systemd-userdbd.service: Failed with result 'exit-code'.
Mar 07 11:18:32 fedora systemd[1]: Failed to start systemd-userdbd.service - User Database Manager.

Looks similar to #1663040.

If you boot with selinux in permissive mode, does it work?

Comment 5 Geoffrey Marr 2022-03-23 14:32:11 UTC
Zbigniew, just tested on Fedora-IoT-IoT-ostree-x86_64-36-20220318.0.iso. If I boot with SELinux in 'permissive' mode, the problem goes away. Running 'systemctl --all --failed' shows nothing when in 'permissive' mode.

Comment 6 Zbigniew Jędrzejewski-Szmek 2022-03-23 16:22:11 UTC
It sounds like this is some variant of rhbz#1663040. It would make sense to check the selinux labels
on the image, see the extensive discussion there.

Comment 7 Zbigniew Jędrzejewski-Szmek 2022-04-28 17:53:00 UTC
Somebody needs to do an analysis like in https://bugzilla.redhat.com/show_bug.cgi?id=1663040#c24