Bug 1593028
| Summary: | No boot messages / plymouth / decryption prompt shown on display of aarch64 VM | ||||||
|---|---|---|---|---|---|---|---|
| Product: | [Fedora] Fedora | Reporter: | Adam Williamson <awilliam> | ||||
| Component: | dracut | Assignee: | dracut-maint-list | ||||
| Status: | CLOSED RAWHIDE | QA Contact: | Fedora Extras Quality Assurance <extras-qa> | ||||
| Severity: | high | Docs Contact: | |||||
| Priority: | unspecified | ||||||
| Version: | 29 | CC: | airlied, berrange, bskeggs, crobinso, dracut-maint-list, ewk, hdegoede, ichavero, itamar, jarodwilson, jglisse, john.j5live, jonathan, josef, kernel-maint, kraxel, linville, mchehab, mjg59, pbonzini, pbrobinson, pjones, pwhalen, rstrode, steved, zbyszek | ||||
| Target Milestone: | --- | ||||||
| Target Release: | --- | ||||||
| Hardware: | aarch64 | ||||||
| OS: | Linux | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2018-12-17 22:25:21 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Attachments: |
|
||||||
|
Description
Adam Williamson
2018-06-19 21:57:20 UTC
pwhalen, can you confirm or deny that an aarch64 VM run manually behaves the same? I don't have a local aarch64 box handy to test myself. Created attachment 1453045 [details]
log with plymouth, kernel and drm messages as verbose as i have
Maybe we simply have an initialization order issue here? I see ... Started Forward Password Requests to Plymouth Directory Watch. ... comes before fbcon setup ... Console: switching to colour frame buffer device 128x48 virtio_gpu virtio0: fb0: virtiodrmfb frame buffer device ... in the log. Try build the aarch64 kernel with CONFIG_DRM_VIRTIO_GPU=y (which is probably a good idea anyway given we have no firmware framebuffer to use) so the framebuffer is already there when plymouth starts. > <airlied> it's likely the boot_vga detection stuff wonn't work, but there
> should only be one framebuffer device in the system
Note that virtio-gpu-pci is PCI_CLASS_DISPLAY_OTHER not PCI_CLASS_DISPLAY_VGA. Possibly this is the reason plymouth doesn't pick up the device?
> Just passing "-device virtio-gpu-pci" (with no "-vga std") results in
> exactly the behaviour described in this bug, i.e. dropping "-vga std" makes
> no difference at all.
You should use "-vga none -device virtio-gpu-pci". The -vga switch is ignored on qemu-system-aarch64 but when trying to reproduce this on x86_64 with ovmf you need it.
> <pjones> c64688f36a8b3 (<kraxel> 2018-06-13 178) > INF OvmfPkg/QemuRamfbDxe/QemuRamfbDxe.inf If you wanna play with that you need cutting edge edk2 and qemu (git master as of today) and a kernel with https://www.spinics.net/lists/linux-efi/msg14247.html Then use "qemu -vga none -device ramfb". So...IIUC you're basically saying the problem and the difference with x86_64 here could be that there's *no* firmware framebuffer on aarch64 (well, maybe there is as of like today with all bleeding-edge bits) but there is on x86_64, with edk2? Will try with a CONFIG_DRM_VIRTIO_GPU=y kernel, thanks. (In reply to Adam Williamson from comment #1) > pwhalen, can you confirm or deny that an aarch64 VM run manually behaves the > same? I don't have a local aarch64 box handy to test myself. Confirm, we use the serial console (ttyAMA0) for output. (In reply to Adam Williamson from comment #7) > So...IIUC you're basically saying the problem and the difference with x86_64 > here could be that there's *no* firmware framebuffer on aarch64 (well, maybe > there is as of like today with all bleeding-edge bits) but there is on > x86_64, with edk2? To be exact, there is a firmware framebuffer with "virtio-vga" (which uses the vga compatibility bits of the device), but there isn't one with "virtio-gpu-pci" (which is the same device without vga compatibility). virtio-vga is not available on aarch64. x86_64 should behave like aarch64 when using virtio-gpu-pci (see comment 5). OK, so I've been testing this today, and I'm pretty sure you're right. I built a scratch kernel with CONFIG_DRM_VIRTIO_GPU=y - https://koji.fedoraproject.org/koji/taskinfo?taskID=27769483 - and hacked up the openQA test so it installs that kernel in the installed system chroot after install is complete, before rebooting. With that change, we *do* get messages during boot, including the decryption prompt. So, re-assigning to kernel. I guess the recommendation is to make this change in the official kernel builds at least until we have the firmware framebuffer working on aarch64 with packaged edk2, kernel and qemu? Thanks a lot! > So, re-assigning to kernel. I guess the recommendation is to make this
> change in the official kernel builds at least until we have the firmware
> framebuffer working on aarch64 with packaged edk2, kernel and qemu?
What change is being recommended here?
Changing CONFIG_DRM_VIRTIO_GPU=m to CONFIG_DRM_VIRTIO_GPU=y for aarch64, as there is currently no good way to get a firmware framebuffer in aarch64 VMs, so this is the only way to get early boot messages (inc. decryption prompts and the like) to show up on a VT. Note this requires some other changes, the diff I wound up with is: --- a/kernel-aarch64.config +++ b/kernel-aarch64.config @@ -1377,6 +1377,7 @@ CONFIG_DRM_AST=m CONFIG_DRM_BOCHS=m # CONFIG_DRM_CDNS_DSI is not set CONFIG_DRM_CIRRUS_QEMU=m +# CONFIG_DRM_DEBUG_MM is not set # CONFIG_DRM_DEBUG_MM_SELFTEST is not set # CONFIG_DRM_DEBUG_SELFTEST is not set CONFIG_DRM_DP_AUX_CHARDEV=y @@ -1402,7 +1403,7 @@ CONFIG_DRM_I2C_SIL164=m # CONFIG_DRM_LEGACY is not set CONFIG_DRM_LOAD_EDID_FIRMWARE=y CONFIG_DRM_LVDS_ENCODER=m -CONFIG_DRM=m +CONFIG_DRM=y CONFIG_DRM_MALI_DISPLAY=m # CONFIG_DRM_MEGACHIPS_STDPXXXX_GE_B850V3_FW is not set CONFIG_DRM_MESON_DW_HDMI=m @@ -1480,7 +1481,7 @@ CONFIG_DRM_VC4_HDMI_CEC=y CONFIG_DRM_VC4=m CONFIG_DRM_VGEM=m CONFIG_DRM_VIA=m -CONFIG_DRM_VIRTIO_GPU=m +CONFIG_DRM_VIRTIO_GPU=y # CONFIG_DRM_XEN is not set # CONFIG_DS1682 is not set # CONFIG_DS1803 is not set @@ -2166,7 +2167,7 @@ CONFIG_HZ_100=y # CONFIG_HZ_300 is not set # CONFIG_HZ_500 is not set # CONFIG_HZ_PERIODIC is not set -CONFIG_I2C_ALGOBIT=m +CONFIG_I2C_ALGOBIT=y CONFIG_I2C_ALGOPCA=m CONFIG_I2C_ALGOPCF=m # CONFIG_I2C_ALI1535 is not set I suppose another possibility might be to keep building it as a module, but get dracut to load it during the initramfs phase...haven't tried that yet. Update: I think I've found a further wrinkle here, investigating the dracut angle.
I think dracut actually already tries to pull DRM modules into the initramfs. In generic mode it pulls in the whole of drivers/gpu/drm , which should ensure virtio-gpu is included...and I think it actually is, and is loaded during the initramfs phase of boot, because when booting an installer image (which has a generic initramfs) we *do* get messages from about halfway through the initramfs phase.
However, I think there may be a bug in what it does in hostonly mode. It does this:
for i in /sys/bus/{pci/devices,soc/devices/soc?}/*/modalias; do
[[ -e $i ]] || continue
if hostonly="" dracut_instmods --silent -s "drm_crtc_init" -S "iw_handler_get_spy" $(<$i); then
if strstr "$(modinfo -F filename $(<$i) 2>/dev/null)" radeon.ko; then
hostonly='' instmods amdkfd
fi
fi
done
I think the bug is that we're missing a bit in that 'for' loop. *It doesn't look in /sys/bus/virtio/devices*...which is exactly where the entry for virtio-gpu will be.
So, I'm currently testing a patch to add virtio/devices to that bit. If I'm right, that should cause virtio-gpu to be pulled into the host-only initramfs generated during install, and then on the installed system, we should get a framebuffer (and hence boot messages) from about halfway through the initramfs phase...which should be early enough for us to see decryption prompts.
If that's the case, we can fix this just by fixing dracut, without touching the kernel.
So my test looked to confirm that fixes things, so I've gone ahead and submitted the patch upstream - https://github.com/dracutdevs/dracut/pull/418 - and done a build for Rawhide with the patch included - https://koji.fedoraproject.org/koji/taskinfo?taskID=27791964 . Should hopefully be fixed with the next compose that includes that build. This bug appears to have been reported against 'rawhide' during the Fedora 29 development cycle. Changing version to '29'. This ought to be fixed by now. This seems to be back in current Rawhide; encrypted install tests almost always fail on the decrypt prompt not showing up. Oddly it seems to work just very occasionally, but almost always fails. Am looking into it now. I'm suspecting plymouth for the new issue; this seems to have been working reliably up till about late October then broken some time between then and mid-November (hard to be more precise than that), in that timeframe I do see a new dracut, but without any obviously incriminating changes. There was also, however, a new plymouth - 0.9.4 - which comes with some very related changes (to the drm code), so that looks like a good suspect. Will file a new bug. There's a general bug/problem that prompts from systemd-ask-password are not visible on the console because they get overwritten by kernel messages. A user can always press ^L to reprint the prompt, and that works fine. Unfortunately we don't have an easy way to say "keep this line at the bottom". It's a race condition, so it might be broken by code changes even if they are not "wrong". Nah, that's not the problem: it's not that the prompt gets scrolled off the screen by other content, *nothing* is displayed. I think it's plymouth DRM mode init stuff. See https://bugzilla.redhat.com/show_bug.cgi?id=1661288 . (Note: I think the problem you mention doesn't happen when Plymouth is active, as plymouth really *does* display password prompts 'over the top of' boot messages. I think that problem only happens when Plymouth is disabled. IMBW though!) |