Bug 683729

Please retry with plymouth and systemd from koji, as this bug should be fixed in more recent plymouth versions.

Which versions should I take?

systemd 20 and plymouth 0.20110304.1.fc15

b20 + 0.20110304 seem to solve the problem.

Thanks.

- Gilboa

Spoke too soon.
I've added vga=0x317 to the command line to get a graphical boot process and now the password prompt appearance is random.
Out of 10 reboots I only seen it once; in the rest of the boot attempts it simply timed out and trashed left me with a dead GDM.

- Gilboa

Please boot with "systemd.log_level=debug" and "systemd.log_target=kmsg", try to reproduce the issue and paste the last log output this generated here before the password stuff timed out.

Please do you testing with the current systemd version in F15

Started Wait for storage scan.
Starting Initialize storage subsystems (RAID, LVM, etc.)...
[   19.060530] systemd[1]: Forked /lib/systemd/fedora-storage-init as 611
[   19.083575] systemd[1]: fedora-storage-init.service changed dead -> start
[   19.117292] systemd[1]: Accepted connection on private bus.
[   19.117452] systemd[1]: Accepted connection on private bus.
[   19.117534] systemd[1]: Accepted connection on private bus.
[   19.118419] systemd[1]: Got D-Bus request: org.freedesktop.systemd1.Agent.Released() on /org/freedesktop/systemd1/agent
[   19.120448] systemd[1]: plymouth-start.service: cgroup is empty
[   19.120830] systemd[1]: Got D-Bus request: org.freedesktop.systemd1.Agent.Released() on /org/freedesktop/systemd1/agent
[   19.120875] systemd[1]: fedora-wait-storage.service: cgroup is empty
[   19.120924] systemd[1]: Got D-Bus request: org.freedesktop.DBus.Local.Disconnected() on /org/freedesktop/DBus/Local
[   19.120970] systemd[1]: Got D-Bus request: org.freedesktop.systemd1.Agent.Released() on /org/freedesktop/systemd1/agent
[   19.120998] systemd[1]: fedora-wait-storage.service: cgroup is empty
[   19.121063] systemd[1]: Got D-Bus request: org.freedesktop.DBus.Local.Disconnected() on /org/freedesktop/DBus/Local
[   19.121101] systemd[1]: Got D-Bus request: org.freedesktop.DBus.Local.Disconnected() on /org/freedesktop/DBus/Local
[   19.121430] systemd[1]: Got D-Bus request: org.freedesktop.systemd1.Agent.Released() on /org/freedesktop/systemd1/agent
[   19.121464] systemd[1]: fedora-wait-storage.service: cgroup is empty
[   19.121507] systemd[1]: Got D-Bus request: org.freedesktop.DBus.Local.Disconnected() on /org/freedesktop/DBus/Local
[   19.124733] systemd[1]: Got D-Bus request: org.freedesktop.systemd1.Agent.Released() on /org/freedesktop/systemd1/agent
[   19.124797] systemd[1]: udev-settle.service: cgroup is empty
[   19.124856] systemd[1]: Got D-Bus request: org.freedesktop.DBus.Local.Disconnected() on /org/freedesktop/DBus/Local
[   20.114391] systemd[1]: dev-disk-by\x2duuid-567ea53b\x2dd9b5\x2d43bd\x2d9beb\x2d030347e5a7b8.device changed dead -> plugged
[   20.134317] systemd[1]: dev-disk-by\x2did-dm\x2duuid\x2dLVM\x2dbFhsOpK47rjCiBZ0RladmTiu2TJtJhMveZ7o8u8Mg1ZQvnR9cFryE53apC9MsKVk.device changed dead -> plugged
[   20.173653] systemd[1]: dev-disk-by\x2did-dm\x2dname\x2dVolRoot\x2dLogHome.device changed dead -> plugged
[   20.200853] systemd[1]: dev-VolRoot-LogHome.device changed dead -> plugged
[   20.221448] systemd[1]: dev-mapper-VolRoot\x2dLogHome.device changed dead -> plugged
[   20.236274] systemd[1]: dev-dm\x2d2.device changed dead -> plugged
[   20.247301] systemd[1]: sys-devices-virtual-block-dm\x2d2.device changed dead -> plugged
[   20.268852] systemd[1]: Received SIGCHLD from PID 611 (fedora-storage-).
[   20.279779] systemd[1]: Got SIGCHLD for process 611 (fedora-storage-)
[   20.279921] systemd[1]: Child 611 died (code=exited, status=0/SUCCESS)
[   20.279927] systemd[1]: Child 611 belongs to fedora-storage-init.service
[   20.279940] systemd[1]: fedora-storage-init.service: main process exited, code=exited, status=0
[   20.386689] systemd[1]: fedora-storage-init.service changed start -> exited
[   20.401671] systemd[1]: Job fedora-storage-init.service/start finished, result=done
[   20.462590] systemd[1]: Accepted connection on private bus.
[1]: Got D-Bus request: org.freedesktop.systemd1.Agent.Released() on /org/freedesktop/systemd1/agent

[   20.504137] systemd[1]: fedora-storage-init.service: cgroup is empty
[   20.519265] systemd[1]: Got D-Bus request: org.freedesktop.DBus.Local.Disconnected() on /org/freedesktop/DBus/Local


... At this point it simply hangs there (till the timeout ends). Nothing on the console, nothing on the QXL spicec viewer. Nada.


Now, -far- worse, when the timeout passes:
[  190.900437] systemd[1]: Got D-Bus request: org.freedesktop.DBus.Local.Disconnected() on /org/freedesktop/DBus/Local
Welcome to emergency mode. Use "systemctl default" or ^D to activate default mode.
[  191.306957] systemd[1]: Received SIGCHLD from PID 631 (echo).
[  191.315167] systemd[1]: Got SIGCHLD for process 631 (echo)
[  191.323767] systemd[1]: Child 631 died (code=exited, status=0/SUCCESS)
[  191.334062] systemd[1]: Child 631 belongs to emergency.service
[  191.348671] systemd[1]: emergency.service: control process exited, code=exited status=0
[  191.359529] systemd[1]: emergency.service got final SIGCHLD for state start-pre
[  191.370636] systemd[1]: About to execute: /sbin/sulogin
[  191.399176] systemd[1]: Forked /sbin/sulogin as 633
[  191.401185] systemd[1]: emergency.service changed start-pre -> running
[  191.401206] systemd[1]: Job emergency.service/start finished, result=done
[  191.401386] systemd[1]: emergency.target changed dead -> active
[  191.401394] systemd[1]: Job emergency.target/start finished, result=done
[  191.401478] systemd[1]: Startup finished in 1s 866ms 679us (kernel) + 4s 69ms 753us (initrd) + 3min 5s 464ms 966us (userspace) = 3min 11s 401ms 398us.
[  191.401799] systemd[1]: Accepted connection on private bus.
[  191.403117] systemd[1]: Got D-Bus request: org.freedesktop.systemd1.Agent.Released() on /org/freedesktop/systemd1/agent
[  191.403237] systemd[1]: Got D-Bus request: org.freedesktop.DBus.Local.Disconnected() on /org/freedesktop/DBus/Local
Give root password for maintenance
(or type Control-D to continue): 
Login incorrect.
Give root password for maintenance
(or type Control-D to continue): 

... And no matter what I do, I cannot type the password. In essence, I have a dead VM.

- Gilboa

Strike that, the VM is complete dead.
Even if I disable serial console al-together -and- remove the vga option, I'm still getting a long timeout and the systemd emergency console.
The systemd doesn't report any errors and/or reason for dropping me into an emergency console (should I file this as a separate bug?) so I have no idea what's broken.

... Time to give up and reinstall?

I should add that the VM was fully upgraded a couple of hours ago.

- Gilboa

I removed quiet from the command line, and now I do see an error code:
Starting Show Playmouth Boot Screen failed, see 'systemctl status plymouth-start.service' for details.

Here's the weird thing: I -do- see the slowly filling "F" boot screen and can switch back and forth from the log view and the graphical view...

Trashed VM?

- Gilboa

Finally got the error code:

systemd[1]: Job dev-mapper-luks\x2dLogHome.device/start timed out.
systemd[1]: Job cryptsetup.target/start failed with result 'dependency'.
systemd[1]: Job cryptsetup@luks\x2dLogHome.servuce/start failed with result 'dependency'.

And the list continues (I'm copying this by hand as serial console no long works)

- Gilboa

hmm, seems the password prompt doesn't work. 

Can you boot with systemd.log_level=debug and systemd.log_target=kmsg, then please capture the full output on the console and attach it here?

Probably some borkage in ply.

Created attachment 496190 [details]
F15 boot output

Job dev-mapper-luks\x2dLogHome.device/start timed out.

It seems the LVM device your encrypted home directory is on never shows up. You are using LUKS on top of LVM, right?

The boot seems to put you into emergency mode. Can you check whether the LogHome LV shows up in /dev/mapper? If it shows up there, it should normally be a symlink to /dev/md4711 or a similar device. Please attach here the output of "udevadm info -p /sys/class/block/md4711 -qall" here (replacing the 4711 of course). This should tell us why systemd never picks up the device.

You might just have an invalid line in /etc/crypttab, that's all.

Created attachment 496425 [details]
/etc/fstab, /etc/crypttab and udevadm output.

There seem to be two issues:
1. I cannot login into Emergency console over a serial console, when I have "console=tty console=ttyS0,57600n8" - each character is followed by NL.

2. Somehow systemd (?) fails to mount my encrypted home (no password prompt), dropping me into an emergency console loop (which I cannot login into - see item 1).

The logical question would be:
Do you want me to open a separate bug for the second issue?

Per you question:
I can see both /dev/mapper/VolRoot-LogHome and /dev/VolRoot/LogHome; both symlink to /dev/dm-2.
I can manually mount it using:
$ /sbin/cryptsetup luksOpen /dev/mapper/VolRoot/LogHome luksHome
$ mount /home

The UUID in /etc/cryttab and the one in returned by luksUUID and the one that appears in udevadm info's ID_FS_UUID_ENC.

(In reply to comment #14)
> 1. I cannot login into Emergency console over a serial console, when I have
> "console=tty console=ttyS0,57600n8" - each character is followed by NL.

That looks like bug 691374.

Oh, OK. my mistake.
How can I debug the second issue?

Beyond that, how I can force the system to start, even if it fails to mount encrypted partition? (A non issue with F14)

There was a missing U in my crypttab (UID instead of UUID).
I'll open up a separate RFE for not being able to bypass a broken crytab entry.

Thanks for your time!

- Gilboa

Forgot to add: Please close this bug. (DUPLICATE of 691374).

*** This bug has been marked as a duplicate of bug 691374 ***