Bug 2390898 - auditd and sometimes other services do not run on current cloud images due to ordering cycle
Summary: auditd and sometimes other services do not run on current cloud images due to...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: cloud-init
Version: 43
Hardware: All
OS: Linux
unspecified
high
Target Milestone: ---
Assignee: Jeremy Cline
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard: AcceptedFreezeException
Depends On:
Blocks: F43BetaFreezeException F43FinalBlocker
TreeView+ depends on / blocked
 
Reported: 2025-08-25 19:20 UTC by Adam Williamson
Modified: 2025-08-28 19:23 UTC (History)
8 users (show)

Fixed In Version: cloud-init-25.2-5.fc43
Clone Of:
Environment:
Last Closed: 2025-08-28 19:23:56 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Adam Williamson 2025-08-25 19:20:31 UTC
Starting with Fedora-Rawhide-20250815.n.0, we got a lot of cases of services being deleted due to an "ordering cycle" with cloud-init-local.service:

https://openqa.fedoraproject.org/tests/3612516#step/base_services_start/7

that means those services will not start. In 20250816.n.0 most of them seemed to go away, but a couple remained, now claiming the cycle is with sysinit.target/start:

https://openqa.fedoraproject.org/tests/3630900#step/base_services_start/7

and on 20250817.n.0, only one remained, auditd:

https://openqa.fedoraproject.org/tests/3636315#step/base_services_start/7

On 20250818.n.0 we were back to a lot:

https://openqa.fedoraproject.org/tests/3640628#step/base_services_start/7

It's varied a bit day-to-day since, but *at least* auditd *always* seems to fail. All of this seems to stem back to 20250815.n.0 - and most likely to cloud-init-25.2 , which was introduced in that compose, both for Rawhide and F43.

Comment 1 Adam Williamson 2025-08-25 19:21:55 UTC
Proposing as a Final blocker - "All system services present after installation with one of the release-blocking package sets must start properly, unless they require hardware which is not present", https://fedoraproject.org/wiki/Fedora_43_Final_Release_Criteria#System_services . Also proposing as a Beta FE, I'd say we should fix this for Beta if we can.

Comment 2 Jeremy Cline 2025-08-25 20:03:29 UTC
Okay, so it looks like Fedora's systemd unit doesn't include DefaultDependencies=no[0] but it _does_ include a Before=auditd.service which itself does specify DefaultDependencies=no. So, based on my reading of the documentation, systemd adds a dependency to cloud-init-local.service of type Requires= and After= on sysinit.target, but auditd includes Before=sysinit.target.

Before=auditd.service got added in https://github.com/canonical/cloud-init/commit/cc8d1b4c464de6883bca50c0822c44fd3ba46735. I'll prep a patch to ensure Fedora has DefaultDependencies=no.

[0] https://github.com/canonical/cloud-init/blob/890873f50d7f14dc235f5eaf2820aae3288a98b1/systemd/cloud-init-local.service.tmpl#L6

Comment 3 Jeremy Cline 2025-08-25 21:39:27 UTC
Okay, I submitted https://github.com/canonical/cloud-init/pull/6423 and did a Rawhide build with it in https://bodhi.fedoraproject.org/updates/FEDORA-2025-1426ef0ad4, so we'll see how the tests do tomorrow and go from there.

Comment 4 Adam Williamson 2025-08-26 17:04:02 UTC
On a personal note, I am very happy that the check I added for this in 2018 finally proved useful. :P

https://pagure.io/fedora-qa/os-autoinst-distri-fedora/c/b4cd1b4a9e4fb25f1238843ebbb1e0f02088a2fa

Comment 5 Fedora Update System 2025-08-26 17:56:23 UTC
FEDORA-2025-39306e86fd (cloud-init-25.2-4.fc43) has been submitted as an update to Fedora 43.
https://bodhi.fedoraproject.org/updates/FEDORA-2025-39306e86fd

Comment 6 Adam Williamson 2025-08-26 23:20:37 UTC
Unfortunately the fix does not work.

https://openqa.fedoraproject.org/tests/3668307#step/base_services_start/7

Comment 7 Fedora Update System 2025-08-27 01:00:38 UTC
FEDORA-2025-39306e86fd has been pushed to the Fedora 43 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2025-39306e86fd`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2025-39306e86fd

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 8 Jeremy Cline 2025-08-27 13:39:51 UTC
Last night I made a tweak to the proposed patch to drop the `After=dbus.socket` requirement from `cloud-init-main.service` since that pushed the service back to `After=sysinit.target`: https://bodhi.fedoraproject.org/updates/FEDORA-2025-7acc66c0d1. Since it seemed to work (thanks Adam for checking) I've updated the proposed F43 update to 25.2-5 as well.

Comment 9 Fedora Update System 2025-08-28 01:34:49 UTC
FEDORA-2025-39306e86fd has been pushed to the Fedora 43 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2025-39306e86fd`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2025-39306e86fd

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 10 Adam Williamson 2025-08-28 15:39:31 UTC
+4 in https://pagure.io/fedora-qa/blocker-review/issue/1879 , marking accepted.

Comment 11 Fedora Update System 2025-08-28 19:23:56 UTC
FEDORA-2025-39306e86fd (cloud-init-25.2-5.fc43) has been pushed to the Fedora 43 stable repository.
If problem still persists, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.