Bug 1329023 - Rawhide non-graphical installs often do not start a tty on vt1
Summary: Rawhide non-graphical installs often do not start a tty on vt1
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: systemd
Version: rawhide
Hardware: All
OS: All
unspecified
medium
Target Milestone: ---
Assignee: systemd-maint
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-04-21 00:11 UTC by Adam Williamson
Modified: 2016-04-22 20:15 UTC (History)
7 users (show)

Fixed In Version: systemd-229-14.fc25
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-04-22 20:15:17 UTC


Attachments (Terms of Use)

Description Adam Williamson 2016-04-21 00:11:11 UTC
Seemingly starting with 2016-04-20, Rawhide installs without a graphical desktop often don't start a vt on tty1 as they should. See e.g. the openQA results for 2016-04-20:

https://openqa.fedoraproject.org/tests/overview?distri=fedora&version=Rawhide&build=Fedora-Rawhide-20160420.n.0&groupid=1

many of the failures (most of the _console_wait_login fails) are of this type. But it doesn't *always* fail, as some tests did get to a tty.

I reproduced this with a local manual install, and found that ttys start up just fine on vt2 and vt3 if you switch to them; it's just tty1 which doesn't get one for some reason.

It doesn't seem to be just a delay (or at least it's a really *long* one) - I've had my test VM running 20 minutes now and still vt1 is showing only a blinking cursor.

Filing on systemd for now, but that's just a guess, and it didn't change in the 201604020.n.0 compose. Nothing *obviously* related did; the closest guesses I have are firewalld and fedora-release-notes.

Proposing as an F25 Beta blocker: "A system installed without a graphical package set must boot to a working login prompt without any unintended user intervention, and all virtual consoles intended to provide a working login prompt must do so."

Affected image download: https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-Rawhide-20160420.n.0/compose/Server/x86_64/iso/Fedora-Server-netinst-x86_64-Rawhide-20160420.n.0.iso

Comment 1 Zbigniew Jędrzejewski-Szmek 2016-04-21 00:28:42 UTC
I'm not aware of any changes from systemd side in that area... If it's systemd, my guess would be that systemd thinks something is hogging tty1. Whether that something exists in reality, or it's just a figment of systemd's imagination, I have no idea.

Comment 2 Adam Williamson 2016-04-21 00:47:06 UTC
So I remembered a wrinkle here: because openQA tests fire off Bodhi fedmsgs, network install tests often wind up pulling in the *previous* day's packages because the mirror they pull from (dl.fp.o basically, but using an infra internal hostname) is not synced when the tests run. (I should probably actually look at tweaking things so they only run after the mirror is synced...hmm). The Server DVD install test for 20160419.n.0 hit this bug, while the netinst tests didn't. So that suggests it was actually introduced by a change in 20160419.n.0 rather than one in 20160420.n.0.

Looking at the 20160419.n.0 changelog, I want to blame...well, I want to blame Docker, but that's entirely out of spite. No. We could look at kernel (which went from kernel-4.6.0-0.rc3.git2.1.fc25 to kernel-4.6.0-0.rc4.git0.1.fc25), ipset (ipset-6.27-2.fc24 to ipset-6.29-1.fc25), selinux-policy (3.13.1-182 to 3.13.1-183), or indeed systemd (229-9 to 229-13).

the systemd package changes seem mostly related to subpackage splitting, so...the packages installed in a minimal Rawhide are systemd, systemd-udev , systemd-libs and systemd-pam . Other subpackages are *not* installed, so if anything significant got moved into one of those, that could be the problem?

I'll play with it a bit.

Comment 3 Adam Williamson 2016-04-21 00:59:38 UTC
hmm, ausearch does show a denial, but neither 'enforcing=0' nor 'selinux=0' causes the bug to go away, so maybe it's coincidental? The denial is:

type=AVC msg=audit(1461199708.094:278): avc: denied { create } for  pid=1115 comm="systemd-user-se" name=".#nologinDUdQgP" scontext=system_u:system_r:init_t:s0 tcontext=system_u:object_r:systemd_login_var_run_t:s0 tclass=file permissive=0

which looks like it's related to handling of 'nologin', which I've had...fun...with before.

with 'enforcing=0' I get subsequent similar denials for the same file and pid for write, setattr and rename.

Comment 4 Zbigniew Jędrzejewski-Szmek 2016-04-21 12:48:54 UTC
rpm -qpl shows that {systemd,systemd-udev,systemd-libs}-229-9-*.rpm and {systemd,systemd-udev,systemd-libs,systemd-pam}-229-13-*.rpm have the same set of files (apart from man pages, a license file, shell completion scripts), with one change: systemd-nspawn@.service was moved out into systemd-container.rpm.

systemd-udev %post/%preun,%pre scriptlets are also slightly modified, but not in a way that should matter.

(In reply to Adam Williamson from comment #3)
> type=AVC msg=audit(1461199708.094:278): avc: denied { create } for  pid=1115
> comm="systemd-user-se" name=".#nologinDUdQgP"
That'd be systemd-user-sessions.service trying to forbid user logins. If that fails, this shouldn't cause any issues. OTOH, I'm a bit surprised to see this. At what point during boot does this happen?

Comment 5 Zbigniew Jędrzejewski-Szmek 2016-04-21 21:17:09 UTC
This appears to be a result of getty@tty1.service not being enabled properly. It looks like either 'systemct preset' was not called properly or it failed somehow... For example it refuses to enable console-getty.service. I need to debug this some more.

Comment 6 Zbigniew Jędrzejewski-Szmek 2016-04-22 02:39:41 UTC
False positive with the console-getty.service. systemctl was stupidly giving a warning about missing [Install] section, which is something to fix upstream, but not relevant to getty@tty1.

So it seems that the systemd %post scriptlet failed for some reason. There's nothing in the logs though. Might be selinux-related. I'll try to redo the installation tomorrow with selinux in permissive mode.

Comment 7 Adam Williamson 2016-04-22 04:03:04 UTC
ah, that would explain why downgrading systemd didn't help when I tried that.

Comment 8 Zbigniew Jędrzejewski-Szmek 2016-04-22 20:15:17 UTC
This was a fallout from the recent changes to move more files to systemd-udev (229-10). Should be fixed now.


Note You need to log in before you can comment on or make changes to this bug.