Bug 1696796 - Boot doesn't fail if requirement of initrd.target fails
Summary: Boot doesn't fail if requirement of initrd.target fails
Keywords:
Status: CLOSED EOL
Alias: None
Product: Fedora
Classification: Fedora
Component: systemd
Version: 29
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: systemd-maint
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-04-05 16:35 UTC by Jonathan Lebon
Modified: 2019-11-27 23:25 UTC (History)
6 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2019-11-27 23:25:11 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Jonathan Lebon 2019-04-05 16:35:10 UTC
Description of problem:

If a hard requirement of initrd.target fails, systemd just keeps looping in the initrd instead of going to the emergency console.

Version-Release number of selected component (if applicable):

systemd-239-12.git8bca462.fc29.x86_64
Note that this doesn't reproduce on f30 (systemd-241-4.gitcbf14c9.fc30.x86_64).

How reproducible:

Always

Steps to Reproduce:
1. Create a new e.g. "80foobar" dracut module with:

# foobar.service
[Unit]
Description=Foobar
DefaultDependencies=false
Before=initrd.target
ConditionKernelCommandLine=foobar.fail=1

[Service]
Type=oneshot
ExecStart=/usr/bin/false

# module-setup.sh
#!/bin/bash

install() {
        inst_multiple true false

        inst_simple "$moddir/foobar.service" \
                "$systemdsystemunitdir/foobar.service"
        mkdir -p "$initdir/$systemdsystemunitdir/initrd.target.requires"
        ln -s "../foobar.service" "$initdir/$systemdsystemunitdir/initrd.target.requires/foobar.service"
}

2. dracut -f -a foobar

3. Boot system with foobar.fail=1

Actual results:

The foobar.service fails, but systemd keeps switching between starting emergency.target and initrd-cleanup.service and never actually brings up the emergency prompt.

Expected results:

systemd goes to emergency.target and stays there.

Additional info:

Interestingly, adding `systemd.log_level=debug systemd.log_target=console` will eventually stop the loop, so I suspect it's sensitive to timing/ordering of jobs.

From analyzing those logs in the loops before it stops, ISTM like the issue is that `emergency.target` is being overriden by the `systemctl isolate initrd-switch-root.target` call from `initrd-cleanup.service`:

```
[FAILED] Failed to start Foobar.
See 'systemctl status foobar.service' for details.
initrd.target: Job initrd.target/start finished, result=dependency
[DEPEND] Dependency failed for Initrd Default Target.
initrd.target: Job initrd.target/start failed with result 'dependency'.
initrd.target: Triggering OnFailure= dependencies.
emergency.target: Trying to enqueue job emergency.target/start/replace-irreversibly
emergency.target: Installed new job emergency.target/start as 50
emergency.service: Installed new job emergency.service/start as 51
emergency.target: Enqueued job emergency.target/start as 50
foobar.service: Unit entered failed state.
...
emergency.service: Passing 0 fds to service
emergency.service: About to execute: /bin/dracut-emergency
emergency.service: Forked /bin/dracut-emergency as 225
emergency.service: Changed dead -> running
emergency.service: Job emergency.service/start finished, result=done
[  OK  ] Started Emergency Shell.
emergency.target changed dead -> active
emergency.target: Job emergency.target/start finished, result=done
...
emergency.service: Changed dead -> running
...
emergency.target changed dead -> active
```

But then a bit after:

```
initrd-cleanup.service: ConditionPathExists=/etc/initrd-release succeeded.
initrd-cleanup.service: Passing 0 fds to service
initrd-cleanup.service: About to execute: /usr/bin/systemctl --no-block isolate initrd-switch-root.target
initrd-cleanup.service: Forked /usr/bin/systemctl as 288
initrd-cleanup.service: Changed dead -> start
         Starting Cleaning Up and Shutting Down Daemons...
initrd-cleanup.service: Executing: /usr/bin/systemctl --no-block isolate initrd-switch-root.target
initrd-switch-root.target: Trying to enqueue job initrd-switch-root.target/start/isolate
initrd-switch-root.target: Installed new job initrd-switch-root.target/start as 91
emergency.service: Installed new job emergency.service/stop as 123
emergency.target: Installed new job emergency.target/stop as 124
...
emergency.target changed active -> dead
emergency.target: Job emergency.target/stop finished, result=done
[  OK  ] Stopped target Emergency Mode.
```

Note: we've been working around this for now by adding `OnFailure=emergency.target` & `OnFailureJobMode=isolate` to our units. I suspect switching `initrd.target` to `OnFailureJobMode=isolate` as well would achieve the same thing, though I haven't tested it.

Comment 1 Ben Cotton 2019-10-31 18:50:41 UTC
This message is a reminder that Fedora 29 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 29 on 2019-11-26.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '29'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 29 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 2 Ben Cotton 2019-11-27 23:25:11 UTC
Fedora 29 changed to end-of-life (EOL) status on 2019-11-26. Fedora 29 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.


Note You need to log in before you can comment on or make changes to this bug.