Bug 683729

Summary: F15 Beta doesn't seem to play nice with serial console + VGA. (console=tty console=ttyS0 vga=0x37f)
Product: [Fedora] Fedora Reporter: Gilboa Davara <gilboad>
Component: systemdAssignee: Lennart Poettering <lpoetter>
Status: CLOSED DUPLICATE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: unspecified    
Version: 15CC: iarlyy, johannbg, jonathan, lpoetter, metherid, mschmidt, notting, plautrba
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-05-09 15:02:28 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
F15 boot output
none
/etc/fstab, /etc/crypttab and udevadm output. none

Description Gilboa Davara 2011-03-10 07:59:17 UTC
Description of problem:
As the title suggests: I've installed F15 Alpha using the x86_64 XFCE LiveCD and encrypted my /home directory.
During first and all subsequent boots, the /home unlock password was clearly visible. 

Given the possible security implication of this bug, I believe this bug is a major show-stopper. (Hence the high-severity)

- Gilboa

Comment 1 Lennart Poettering 2011-03-11 23:57:06 UTC
Please retry with plymouth and systemd from koji, as this bug should be fixed in more recent plymouth versions.

Comment 2 Gilboa Davara 2011-03-12 07:42:52 UTC
Which versions should I take?

Comment 3 Lennart Poettering 2011-03-14 00:12:28 UTC
systemd 20 and plymouth 0.20110304.1.fc15

Comment 4 Gilboa Davara 2011-03-22 14:10:14 UTC
b20 + 0.20110304 seem to solve the problem.

Thanks.

- Gilboa

Comment 5 Gilboa Davara 2011-03-28 08:51:48 UTC
Spoke too soon.
I've added vga=0x317 to the command line to get a graphical boot process and now the password prompt appearance is random.
Out of 10 reboots I only seen it once; in the rest of the boot attempts it simply timed out and trashed left me with a dead GDM.

- Gilboa

Comment 6 Lennart Poettering 2011-04-06 01:45:02 UTC
Please boot with "systemd.log_level=debug" and "systemd.log_target=kmsg", try to reproduce the issue and paste the last log output this generated here before the password stuff timed out.

Please do you testing with the current systemd version in F15

Comment 7 Gilboa Davara 2011-04-21 14:24:01 UTC
Started Wait for storage scan.
Starting Initialize storage subsystems (RAID, LVM, etc.)...
[   19.060530] systemd[1]: Forked /lib/systemd/fedora-storage-init as 611
[   19.083575] systemd[1]: fedora-storage-init.service changed dead -> start
[   19.117292] systemd[1]: Accepted connection on private bus.
[   19.117452] systemd[1]: Accepted connection on private bus.
[   19.117534] systemd[1]: Accepted connection on private bus.
[   19.118419] systemd[1]: Got D-Bus request: org.freedesktop.systemd1.Agent.Released() on /org/freedesktop/systemd1/agent
[   19.120448] systemd[1]: plymouth-start.service: cgroup is empty
[   19.120830] systemd[1]: Got D-Bus request: org.freedesktop.systemd1.Agent.Released() on /org/freedesktop/systemd1/agent
[   19.120875] systemd[1]: fedora-wait-storage.service: cgroup is empty
[   19.120924] systemd[1]: Got D-Bus request: org.freedesktop.DBus.Local.Disconnected() on /org/freedesktop/DBus/Local
[   19.120970] systemd[1]: Got D-Bus request: org.freedesktop.systemd1.Agent.Released() on /org/freedesktop/systemd1/agent
[   19.120998] systemd[1]: fedora-wait-storage.service: cgroup is empty
[   19.121063] systemd[1]: Got D-Bus request: org.freedesktop.DBus.Local.Disconnected() on /org/freedesktop/DBus/Local
[   19.121101] systemd[1]: Got D-Bus request: org.freedesktop.DBus.Local.Disconnected() on /org/freedesktop/DBus/Local
[   19.121430] systemd[1]: Got D-Bus request: org.freedesktop.systemd1.Agent.Released() on /org/freedesktop/systemd1/agent
[   19.121464] systemd[1]: fedora-wait-storage.service: cgroup is empty
[   19.121507] systemd[1]: Got D-Bus request: org.freedesktop.DBus.Local.Disconnected() on /org/freedesktop/DBus/Local
[   19.124733] systemd[1]: Got D-Bus request: org.freedesktop.systemd1.Agent.Released() on /org/freedesktop/systemd1/agent
[   19.124797] systemd[1]: udev-settle.service: cgroup is empty
[   19.124856] systemd[1]: Got D-Bus request: org.freedesktop.DBus.Local.Disconnected() on /org/freedesktop/DBus/Local
[   20.114391] systemd[1]: dev-disk-by\x2duuid-567ea53b\x2dd9b5\x2d43bd\x2d9beb\x2d030347e5a7b8.device changed dead -> plugged
[   20.134317] systemd[1]: dev-disk-by\x2did-dm\x2duuid\x2dLVM\x2dbFhsOpK47rjCiBZ0RladmTiu2TJtJhMveZ7o8u8Mg1ZQvnR9cFryE53apC9MsKVk.device changed dead -> plugged
[   20.173653] systemd[1]: dev-disk-by\x2did-dm\x2dname\x2dVolRoot\x2dLogHome.device changed dead -> plugged
[   20.200853] systemd[1]: dev-VolRoot-LogHome.device changed dead -> plugged
[   20.221448] systemd[1]: dev-mapper-VolRoot\x2dLogHome.device changed dead -> plugged
[   20.236274] systemd[1]: dev-dm\x2d2.device changed dead -> plugged
[   20.247301] systemd[1]: sys-devices-virtual-block-dm\x2d2.device changed dead -> plugged
[   20.268852] systemd[1]: Received SIGCHLD from PID 611 (fedora-storage-).
[   20.279779] systemd[1]: Got SIGCHLD for process 611 (fedora-storage-)
[   20.279921] systemd[1]: Child 611 died (code=exited, status=0/SUCCESS)
[   20.279927] systemd[1]: Child 611 belongs to fedora-storage-init.service
[   20.279940] systemd[1]: fedora-storage-init.service: main process exited, code=exited, status=0
[   20.386689] systemd[1]: fedora-storage-init.service changed start -> exited
[   20.401671] systemd[1]: Job fedora-storage-init.service/start finished, result=done
[   20.462590] systemd[1]: Accepted connection on private bus.
[1]: Got D-Bus request: org.freedesktop.systemd1.Agent.Released() on /org/freedesktop/systemd1/agent

[   20.504137] systemd[1]: fedora-storage-init.service: cgroup is empty
[   20.519265] systemd[1]: Got D-Bus request: org.freedesktop.DBus.Local.Disconnected() on /org/freedesktop/DBus/Local


... At this point it simply hangs there (till the timeout ends). Nothing on the console, nothing on the QXL spicec viewer. Nada.


Now, -far- worse, when the timeout passes:
[  190.900437] systemd[1]: Got D-Bus request: org.freedesktop.DBus.Local.Disconnected() on /org/freedesktop/DBus/Local
Welcome to emergency mode. Use "systemctl default" or ^D to activate default mode.
[  191.306957] systemd[1]: Received SIGCHLD from PID 631 (echo).
[  191.315167] systemd[1]: Got SIGCHLD for process 631 (echo)
[  191.323767] systemd[1]: Child 631 died (code=exited, status=0/SUCCESS)
[  191.334062] systemd[1]: Child 631 belongs to emergency.service
[  191.348671] systemd[1]: emergency.service: control process exited, code=exited status=0
[  191.359529] systemd[1]: emergency.service got final SIGCHLD for state start-pre
[  191.370636] systemd[1]: About to execute: /sbin/sulogin
[  191.399176] systemd[1]: Forked /sbin/sulogin as 633
[  191.401185] systemd[1]: emergency.service changed start-pre -> running
[  191.401206] systemd[1]: Job emergency.service/start finished, result=done
[  191.401386] systemd[1]: emergency.target changed dead -> active
[  191.401394] systemd[1]: Job emergency.target/start finished, result=done
[  191.401478] systemd[1]: Startup finished in 1s 866ms 679us (kernel) + 4s 69ms 753us (initrd) + 3min 5s 464ms 966us (userspace) = 3min 11s 401ms 398us.
[  191.401799] systemd[1]: Accepted connection on private bus.
[  191.403117] systemd[1]: Got D-Bus request: org.freedesktop.systemd1.Agent.Released() on /org/freedesktop/systemd1/agent
[  191.403237] systemd[1]: Got D-Bus request: org.freedesktop.DBus.Local.Disconnected() on /org/freedesktop/DBus/Local
Give root password for maintenance
(or type Control-D to continue): 
Login incorrect.
Give root password for maintenance
(or type Control-D to continue): 

... And no matter what I do, I cannot type the password. In essence, I have a dead VM.

- Gilboa

Comment 8 Gilboa Davara 2011-04-21 14:37:38 UTC
Strike that, the VM is complete dead.
Even if I disable serial console al-together -and- remove the vga option, I'm still getting a long timeout and the systemd emergency console.
The systemd doesn't report any errors and/or reason for dropping me into an emergency console (should I file this as a separate bug?) so I have no idea what's broken.

... Time to give up and reinstall?

I should add that the VM was fully upgraded a couple of hours ago.

- Gilboa

Comment 9 Gilboa Davara 2011-04-21 14:41:16 UTC
I removed quiet from the command line, and now I do see an error code:
Starting Show Playmouth Boot Screen failed, see 'systemctl status plymouth-start.service' for details.

Here's the weird thing: I -do- see the slowly filling "F" boot screen and can switch back and forth from the log view and the graphical view...

Trashed VM?

- Gilboa

Comment 10 Gilboa Davara 2011-04-21 14:44:32 UTC
Finally got the error code:

systemd[1]: Job dev-mapper-luks\x2dLogHome.device/start timed out.
systemd[1]: Job cryptsetup.target/start failed with result 'dependency'.
systemd[1]: Job cryptsetup@luks\x2dLogHome.servuce/start failed with result 'dependency'.

And the list continues (I'm copying this by hand as serial console no long works)

- Gilboa

Comment 11 Lennart Poettering 2011-04-28 17:57:48 UTC
hmm, seems the password prompt doesn't work. 

Can you boot with systemd.log_level=debug and systemd.log_target=kmsg, then please capture the full output on the console and attach it here?

Probably some borkage in ply.

Comment 12 Gilboa Davara 2011-05-02 08:30:38 UTC
Created attachment 496190 [details]
F15 boot output

Comment 13 Lennart Poettering 2011-05-02 16:52:32 UTC
Job dev-mapper-luks\x2dLogHome.device/start timed out.

It seems the LVM device your encrypted home directory is on never shows up. You are using LUKS on top of LVM, right?

The boot seems to put you into emergency mode. Can you check whether the LogHome LV shows up in /dev/mapper? If it shows up there, it should normally be a symlink to /dev/md4711 or a similar device. Please attach here the output of "udevadm info -p /sys/class/block/md4711 -qall" here (replacing the 4711 of course). This should tell us why systemd never picks up the device.

You might just have an invalid line in /etc/crypttab, that's all.

Comment 14 Gilboa Davara 2011-05-03 04:36:14 UTC
Created attachment 496425 [details]
/etc/fstab, /etc/crypttab and udevadm output.

There seem to be two issues:
1. I cannot login into Emergency console over a serial console, when I have "console=tty console=ttyS0,57600n8" - each character is followed by NL.

2. Somehow systemd (?) fails to mount my encrypted home (no password prompt), dropping me into an emergency console loop (which I cannot login into - see item 1).

The logical question would be:
Do you want me to open a separate bug for the second issue?

Per you question:
I can see both /dev/mapper/VolRoot-LogHome and /dev/VolRoot/LogHome; both symlink to /dev/dm-2.
I can manually mount it using:
$ /sbin/cryptsetup luksOpen /dev/mapper/VolRoot/LogHome luksHome
$ mount /home

The UUID in /etc/cryttab and the one in returned by luksUUID and the one that appears in udevadm info's ID_FS_UUID_ENC.

Comment 15 Michal Schmidt 2011-05-03 10:51:43 UTC
(In reply to comment #14)
> 1. I cannot login into Emergency console over a serial console, when I have
> "console=tty console=ttyS0,57600n8" - each character is followed by NL.

That looks like bug 691374.

Comment 16 Gilboa Davara 2011-05-06 10:13:28 UTC
Oh, OK. my mistake.
How can I debug the second issue?

Beyond that, how I can force the system to start, even if it fails to mount encrypted partition? (A non issue with F14)

Comment 17 Gilboa Davara 2011-05-09 14:50:00 UTC
There was a missing U in my crypttab (UID instead of UUID).
I'll open up a separate RFE for not being able to bypass a broken crytab entry.

Thanks for your time!

- Gilboa

Comment 18 Gilboa Davara 2011-05-09 14:52:15 UTC
Forgot to add: Please close this bug. (DUPLICATE of 691374).

Comment 19 Michal Schmidt 2011-05-09 15:02:28 UTC

*** This bug has been marked as a duplicate of bug 691374 ***