This bug was initially created as a copy of Bug #2391342 I am copying this bug because: I am copying this bug because the issue may be in the base image or in the systemd unit definitions: Description of problem: In the https://github.com/freeipa/freeipa-container project we build Fedora-based FreeIPA server images in https://quay.io/repository/freeipa/freeipa-server?tab=tags. The FreeIPA containers are systemd-based and we depend on systemd and dbus working. In latest registry.fedoraproject.org/fedora:rawhide with systemd installed, we observe that dbus-broker fails to start. Version-Release number of selected component (if applicable): REPOSITORY TAG IMAGE ID CREATED SIZE registry.fedoraproject.org/fedora rawhide ac69f932f370 2 days ago 184 MB with systemd 258~rc3-2.fc44 installed. How reproducible: Deterministic. Steps to Reproduce: 1. Have Dockerfile FROM specify-image-with-from-parameter RUN dnf install --setopt=install_weak_deps=False -y systemd ENTRYPOINT [ "/usr/sbin/init" ] STOPSIGNAL RTMIN+3 2. Build Fedora 42 and Fedora rawhide images with rootless podman: $ podman build --from registry.fedoraproject.org/fedora:42 -t localhost/fedora-init:42 . $ podman build --from registry.fedoraproject.org/fedora:rawhide -t localhost/fedora-init:rawhide . 3. Run Fedora 42 systemd container: $ podman run --rm -ti --name fedora-init-42 localhost/fedora-init:42 4. From another terminal check that dbus-broker service is running: $ podman exec -ti fedora-init-42 systemctl status dbus-broker ● dbus-broker.service - D-Bus System Message Bus Loaded: loaded (/usr/lib/systemd/system/dbus-broker.service; enabled; preset: enabled) Drop-In: /usr/lib/systemd/system/service.d └─10-timeout-abort.conf Active: active (running) since Wed 2025-08-27 19:26:07 UTC; 6s ago Invocation: a11ab904f5544a9ca1f57daaf01f364d TriggeredBy: ● dbus.socket Docs: man:dbus-broker-launch(1) Main PID: 35 (dbus-broker-lau) Tasks: 2 (limit: 307) Memory: 1.1M (peak: 2.1M) CPU: 9ms CGroup: /system.slice/dbus-broker.service ├─35 /usr/bin/dbus-broker-launch --scope system --audit └─37 dbus-broker --log 4 --controller 9 --machine-id dd4eb2cf674143ef88b5f6162aad6450 --max-bytes 536870912 --max-fds 4096 --max-matches 16384 --audit Aug 27 19:26:07 2f01e1f9eac7 systemd[1]: Starting dbus-broker.service - D-Bus System Message Bus... Aug 27 19:26:07 2f01e1f9eac7 systemd[1]: Started dbus-broker.service - D-Bus System Message Bus. Aug 27 19:26:07 2f01e1f9eac7 dbus-broker-launch[35]: Ready 5. From that second terminal, $ podman rm -f fedora-init-42 6. Now repeat with the rawhide image: $ podman run --rm -ti --name fedora-init-rawhide localhost/fedora-init:rawhide 7. From another terminal, check $ podman exec -ti fedora-init-rawhide systemctl status dbus-broker Actual results: ○ dbus-broker.service - D-Bus System Message Bus Loaded: loaded (/usr/lib/systemd/system/dbus-broker.service; enabled; preset: enabled) Drop-In: /usr/lib/systemd/system/service.d └─10-timeout-abort.conf Active: inactive (dead) TriggeredBy: ● dbus.socket Docs: man:dbus-broker-launch(1) Expected results: dbus-broker.service active / running. Additional info: Attempt to start the service manually fails: $ podman exec -ti fedora-init-rawhide systemctl start dbus-broker $ podman exec -ti fedora-init-rawhide journalctl -l | grep dbus Aug 27 19:27:54 dd57db4d12f4 systemd[1]: Listening on dbus.socket - D-Bus System Message Bus Socket. Aug 27 19:29:02 dd57db4d12f4 systemd[1]: Starting dbus-broker.service - D-Bus System Message Bus... Aug 27 19:29:02 dd57db4d12f4 systemd[1]: Started dbus-broker.service - D-Bus System Message Bus. Aug 27 19:29:02 dd57db4d12f4 dbus-broker-launch[104]: ERROR util_audit_drop_permissions @ ../src/util/audit.c +81: Operation not permitted Aug 27 19:29:02 dd57db4d12f4 dbus-broker-launch[102]: ERROR service_add @ ../src/launch/service.c +1011: Transport endpoint is not connected Aug 27 19:29:02 dd57db4d12f4 dbus-broker-launch[102]: launcher_add_services @ ../src/launch/launcher.c +823 Aug 27 19:29:02 dd57db4d12f4 dbus-broker-launch[102]: launcher_run @ ../src/launch/launcher.c +1439 Aug 27 19:29:02 dd57db4d12f4 dbus-broker-launch[102]: run @ ../src/launch/main.c +152 Aug 27 19:29:02 dd57db4d12f4 dbus-broker-launch[102]: main @ ../src/launch/main.c +178 Aug 27 19:29:02 dd57db4d12f4 dbus-broker-launch[102]: Exiting due to fatal error: -107 $ podman exec -ti fedora-init-rawhide systemctl status dbus-broker ● dbus-broker.service - D-Bus System Message Bus Loaded: loaded (/usr/lib/systemd/system/dbus-broker.service; enabled; preset: enabled) Drop-In: /usr/lib/systemd/system/service.d └─10-timeout-abort.conf Active: activating (start) since Wed 2025-08-27 19:29:48 UTC; 9ms ago Job: 410 Invocation: 40e7f7bd649649cfaa72f9ce69c03b16 TriggeredBy: ● dbus.socket Docs: man:dbus-broker-launch(1) Main PID: 112 ((r-launch)) Tasks: 1 (limit: 307) Memory: 1M (peak: 1M) CPU: 6ms CGroup: /system.slice/dbus-broker.service └─112 "(r-launch)" Aug 27 19:29:48 dd57db4d12f4 systemd[1]: Starting dbus-broker.service - D-Bus System Message Bus...
Problem is still present with systemd-258-1.fc44.x86_64 in Fedora rawhide.
Problem is still present with systemd-258.1-2.fc44.x86_64 in Fedora rawhide.
The bug is now present on now-released Fedora 43 as well: $ podman exec -ti fedora-init-43 systemctl status dbus-broker ○ dbus-broker.service - D-Bus System Message Bus Loaded: loaded (/usr/lib/systemd/system/dbus-broker.service; enabled; preset: enabled) Drop-In: /usr/lib/systemd/system/service.d └─10-timeout-abort.conf Active: inactive (dead) TriggeredBy: ● dbus.socket Docs: man:dbus-broker-launch(1) $ podman exec -ti fedora-init-43 systemctl start dbus-broker $ podman exec -ti fedora-init-43 journalctl -l | grep dbus Oct 30 05:49:18 2762b2e4ed99 systemd[1]: Listening on dbus.socket - D-Bus System Message Bus Socket. Oct 30 05:49:46 2762b2e4ed99 systemd[1]: Starting dbus-broker.service - D-Bus System Message Bus... Oct 30 05:49:46 2762b2e4ed99 systemd[1]: Started dbus-broker.service - D-Bus System Message Bus. Oct 30 05:49:46 2762b2e4ed99 dbus-broker-launch[106]: ERROR util_audit_drop_permissions @ ../src/util/audit.c +81: Operation not permitted Oct 30 05:49:46 2762b2e4ed99 dbus-broker-launch[104]: ERROR service_add @ ../src/launch/service.c +1011: Transport endpoint is not connected Oct 30 05:49:46 2762b2e4ed99 dbus-broker-launch[104]: launcher_add_services @ ../src/launch/launcher.c +823 Oct 30 05:49:46 2762b2e4ed99 dbus-broker-launch[104]: launcher_run @ ../src/launch/launcher.c +1439 Oct 30 05:49:46 2762b2e4ed99 dbus-broker-launch[104]: run @ ../src/launch/main.c +152 Oct 30 05:49:46 2762b2e4ed99 dbus-broker-launch[104]: main @ ../src/launch/main.c +178 Oct 30 05:49:46 2762b2e4ed99 dbus-broker-launch[104]: Exiting due to fatal error: -107
I might add that the issue is not present when this rootless systemd-based container is podman-run as --privileged. But that is obviously not a way we'd recommend users to run the containers, and it was not necessary for many years now.
Hmm, maybe I'm missing something, but this seems to be a problem with dbus-broker-launch, not systemd directly. I'm seeing the same failure: Nov 04 13:38:34 76393db05c54 systemd[1]: Started dbus-broker.service - D-Bus System Message Bus. Nov 04 13:38:34 76393db05c54 dbus-broker-launch[45]: ERROR util_audit_drop_permissions @ ../src/util/audit.c +81: Operation not permitted Nov 04 13:38:34 76393db05c54 dbus-broker-launch[43]: ERROR service_add @ ../src/launch/service.c +1011: Transport endpoint is not connected Nov 04 13:38:34 76393db05c54 dbus-broker-launch[43]: launcher_add_services @ ../src/launch/launcher.c +823 Nov 04 13:38:34 76393db05c54 dbus-broker-launch[43]: launcher_run @ ../src/launch/launcher.c +1439 Nov 04 13:38:34 76393db05c54 dbus-broker-launch[43]: run @ ../src/launch/main.c +152 Nov 04 13:38:34 76393db05c54 dbus-broker-launch[43]: main @ ../src/launch/main.c +178 Nov 04 13:38:34 76393db05c54 dbus-broker-launch[43]: Exiting due to fatal error: -107 Nov 04 13:39:19 76393db05c54 systemd[1]: dbus-broker.service: Main process exited, code=exited, status=1/FAILURE Nov 04 13:39:19 76393db05c54 systemd[1]: dbus-broker.service: Failed with result 'exit-code'. I'm also seeing the following: Nov 04 13:38:33 76393db05c54 systemd-update-utmp[28]: Failed to connect to audit log, ignoring: Invalid argument This suggests that the audit log socket is set up strangely. Maybe this is related to the failure in dbus-broker-launch. I'll reassign this to dbus-broker for comments.
*** Bug 2391237 has been marked as a duplicate of this bug. ***
Currently, on Fedora 42 (where things work) the version of dbus-broker is $ podman run --rm --entrypoint rpm localhost/fedora-init:42 -q dbus-broker dbus-broker-36-6.fc42.x86_64 Even if I force the same version of dbus-broker into the Fedora 42 container with FROM registry.fedoraproject.org/fedora:43 RUN dnf install --setopt=install_weak_deps=False -y https://kojipkgs.fedoraproject.org//packages/dbus-broker/36/6.fc43/x86_64/dbus-broker-36-6.fc43.x86_64.rpm systemd ENTRYPOINT [ "/usr/sbin/init" ] STOPSIGNAL RTMIN+3 I still get the failure described. So I suspect that something else has changed that causes the problem, that's why I filed the bugzilla gainst systemd because I feel systemd dictates the overall system behaviour. Of course, there might be an actual problem with dbus-broker that it did not make appropriate compatibility change to go with whatever has changed between the Fedora 42's systemd systemd-257.10-1.fc42.x86_64 and Fedora 43's systemd-258.1-1.fc43.x86_64.
To further bisect the component causing the change in behaviour, I tried to build the Fedora 43-based container image with Fedora 42's systemd: FROM registry.fedoraproject.org/fedora:43 RUN dnf install --setopt=install_weak_deps=False -y https://kojipkgs.fedoraproject.org//packages/systemd/257.10/1.fc42/x86_64/systemd-257.10-1.fc42.x86_64.rpm https://kojipkgs.fedoraproject.org//packages/systemd/257.10/1.fc42/x86_64/systemd-shared-257.10-1.fc42.x86_64.rpm https://kojipkgs.fedoraproject.org//packages/systemd/257.10/1.fc42/x86_64/systemd-pam-257.10-1.fc42.x86_64.rpm https://kojipkgs.fedoraproject.org//packages/systemd/257.10/1.fc42/x86_64/systemd-libs-257.10-1.fc42.x86_64.rpm ENTRYPOINT [ "/usr/sbin/init" ] STOPSIGNAL RTMIN+3 This gave me Package Arch Version Repository Size Downgrading: systemd-libs x86_64 257.10-1.fc42 @commandline 2.2 MiB replacing systemd-libs x86_64 258.1-1.fc43 595dad8e40c0452fbc9f056f963b8c82 2.3 MiB Installing: systemd x86_64 257.10-1.fc42 @commandline 12.1 MiB systemd-pam x86_64 257.10-1.fc42 @commandline 1.1 MiB systemd-shared x86_64 257.10-1.fc42 @commandline 4.6 MiB Installing dependencies: dbus x86_64 1:1.16.0-4.fc43 fedora 0.0 B dbus-broker x86_64 37-2.fc43 fedora 387.2 KiB dbus-common noarch 1:1.16.0-4.fc43 fedora 11.2 KiB expat x86_64 2.7.2-1.fc43 fedora 298.6 KiB libfdisk x86_64 2.41.1-17.fc43 fedora 380.4 KiB libseccomp x86_64 2.6.0-2.fc43 fedora 224.9 KiB and podman run --rm -ti --name fedora-init-43 localhost/fedora-init:43 with this image works. So it really seems like systemd is the component that needs to be investigated.
Amending typo in comment 7: > Even if I force the same version of dbus-broker into the Fedora 42 container with Even if I force the same (Fedora 42's) version of dbus-broker into the Fedora *43* container with And to be extra sure, I actually tried that with the exact fc42 build FROM registry.fedoraproject.org/fedora:43 RUN dnf install --setopt=install_weak_deps=False -y https://kojipkgs.fedoraproject.org//packages/dbus-broker/36/6.fc42/x86_64/dbus-broker-36-6.fc42.x86_64.rpm systemd ENTRYPOINT [ "/usr/sbin/init" ] STOPSIGNAL RTMIN+3 not just with the fc43 build of the same NVR. This combination fails as well.
[pid 1744163] setresgid(81, 81, 81) = -1 EINVAL (Invalid argument) setresgid.2 says: > EINVAL One or more of the target user or group IDs is not valid in this user namespace.
When I add util-linux to the RUN dnf install --setopt=install_weak_deps=False -y systemd util-linux command, I can then do $ podman run --rm --entrypoint runuser localhost/fedora-init:43 -u dbus -- id uid=81(dbus) gid=81(dbus) groups=81(dbus) fine.
Upstream issue to discuss this: https://github.com/systemd/systemd/issues/39563
Alexander mentioned in https://github.com/freeipa/freeipa-container/issues/690#issuecomment-3495892952 that you were hoping to do a backport Fedora builds yesterday. For our planning and testing purposes, what is the current timeline of getting the fix to Fedora 43+? We have a workaround with the use of Fedora 42's systemd but we wouldn't be able to publish those images, so if updated Fedora 43 and rawhide builds might appear shortly, we might just prefer to go with the proper fix than the workaround.
The CI was failing and we ended up splitting the series into batches to figure out what was wrong. This is still going through the steps. Probably it should be finished today, but unfortunately some more hiccups are possible.
FEDORA-2025-0c5b003313 (systemd-258.2-1.fc44) has been submitted as an update to Fedora 44. https://bodhi.fedoraproject.org/updates/FEDORA-2025-0c5b003313
I tested the build with FROM registry.fedoraproject.org/fedora:rawhide RUN dnf install --setopt=install_weak_deps=False -y \ https://kojipkgs.fedoraproject.org//packages/systemd/258.2/1.fc44/x86_64/systemd-258.2-1.fc44.x86_64.rpm \ https://kojipkgs.fedoraproject.org//packages/systemd/258.2/1.fc44/x86_64/systemd-libs-258.2-1.fc44.x86_64.rpm \ https://kojipkgs.fedoraproject.org//packages/systemd/258.2/1.fc44/x86_64/systemd-pam-258.2-1.fc44.x86_64.rpm \ https://kojipkgs.fedoraproject.org//packages/systemd/258.2/1.fc44/x86_64/systemd-shared-258.2-1.fc44.x86_64.rpm ENTRYPOINT [ "/usr/sbin/init" ] STOPSIGNAL RTMIN+3 and confirm that the systemd-based container works again well -- dbus-service is active (running) and no failed services (reported in https://bugzilla.redhat.com/show_bug.cgi?id=2391237).
FEDORA-2025-0c5b003313 (systemd-258.2-1.fc44) has been pushed to the Fedora 44 stable repository. If problem still persists, please make note of it in this bug report.
The systemd-258.2-1.fc44 works fine, thanks. Will you be able to do a Fedora 43 shortly, or should I clone this bugzilla for Fedora 43 to track that work?
Let's just reuse this bug.
FEDORA-2025-5c83c2df70 (systemd-258.2-1.fc43) has been submitted as an update to Fedora 43. https://bodhi.fedoraproject.org/updates/FEDORA-2025-5c83c2df70
Thanks. Karma given to the Fedora 43 errata.
FEDORA-2025-5c83c2df70 has been pushed to the Fedora 43 testing repository. Soon you'll be able to install the update with the following command: `sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2025-5c83c2df70` You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2025-5c83c2df70 See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
FEDORA-2025-5c83c2df70 (systemd-258.2-1.fc43) has been pushed to the Fedora 43 stable repository. If problem still persists, please make note of it in this bug report.