Created attachment 1941036 [details]
Description of problem:
Boot Fedora-Server-dvd-x86_64-Rawhide-20230126.n.0.iso/Fedora-Server-netinst-x86_64-Rawhide-20230126.n.0.iso with inst.text command line added,system hangs at Finished plymouth-quit.service
Version-Release number of selected component (if applicable):
Steps to Reproduce:
Proposed as a Blocker for 38-beta by Fedora user lnie using the blocker tracking app because:
When using a dedicated installer image, the installer must be able to complete an installation using the text, graphical and VNC installation interfaces.
This has been failing in openQA since Fedora-Rawhide-20230125.n.0 (I just didn't get around to reporting it yet, last week was kinda hectic). systemd 253~rc1 got tagged on 0124, so I'm betting this is a systemd 253 bug. Re-assigning.
Rescue mode boot fails the same way.
I can reproduce this locally. The journal does show anaconda running, but it's not actually visible on any VT, it should be on VT1. VT2, VT3, VT5 and VT6 show root consoles (this is normal, except for VT2, which should show a custom tmux setup). VT4 shows the system logs (this is normal).
To reproduce: download https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-Rawhide-20230130.n.0/compose/Everything/x86_64/iso/Fedora-Everything-netinst-x86_64-Rawhide-20230130.n.0.iso , boot it with `inst.text`.
I do see a message:
/usr/lib/systemd/system-generators/anaconda-generator failed with exit status 1.
which may be related. It's very likely why VT2 doesn't have the custom tmux session that it ought to have. Running that script directly after system boot does not seem to fail, it exits 0. Before I run that script manually, /usr/lib/systemd/system/anaconda.target.wants does not exist. After running that script, it exists and contains several services.
Oh, yes, this cannot work. As documented in systemd.generator(7):
> Each generator is called with three directory paths that are to be used for generator output.
Generators must not write outside of those paths.
I see that anaconda-generator does
> mkdir -p $target_dir
> ln -sf $systemd_dir/anaconda-nm-config.service $target_dir/anaconda-nm-config.service
> ln -sf $systemd_dir/anaconda-nm-disable-autocons.service $target_dir/anaconda-nm-disable-autocons.service
> ln -sf $systemd_dir/anaconda-pre.service $target_dir/anaconda-pre.service
This is broken on many levels:
1. Generators are called early, potentially before / has been remounted rw.
2. Generators are not supposed to create permanent configuration, their purpose is to translate foreign config into units on the fly.
3. Generator output is tied to the lifetime of systemd daemon-reload, i.e. potentially at any time a reload may happen, and if the generator is replaced or masked, its effects are supposed to vanish.
In systemd-253-rc1 we added a "sandbox" that provides a writable /tmp for
generators (so that here-docs work in bash even if /tmp is not available yet),
and makes most of the file system read-only. This obviously interferes with
the stuff that anaconda-generator was doing.
Please make the generator write in the expected locations.
Zbigniew, does https://github.com/rhinstaller/anaconda/pull/4534 look good to you? It seems to work for me.
Setting needinfo for comment 6.
The PR is merged now, setting MODIFIED.
The PR LGTM.
This bug appears to have been reported against 'rawhide' during the Fedora Linux 38 development cycle.
Changing version to 38.
For the reference, the issue made "dracut-visible-warnings" kickstart test fail (https://github.com/rhinstaller/kickstart-tests/issues/875).
It is passing now.
oh, yeah, this is confirmed fixed, openQA tests are passing now. Sorry, I forgot to update the bug.
No problem @awilliam and thank you for the fix!