Bug 2218719 - Cloud vm guest hangs after installing new kernel
Summary: Cloud vm guest hangs after installing new kernel
Keywords:
Status: CLOSED EOL
Alias: None
Product: Fedora
Classification: Fedora
Component: qemu
Version: 38
Hardware: x86_64
OS: Linux
unspecified
urgent
Target Milestone: ---
Assignee: Fedora Virtualization Maintainers
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2023-06-30 02:02 UTC by Josef Bacik
Modified: 2024-05-28 13:17 UTC (History)
8 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2024-05-28 13:17:06 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)

Description Josef Bacik 2023-06-30 02:02:37 UTC
If I build a new virt guest based off of a f38 cloud image (I assume other ones as well, this just happens to be how I'm doing it) and I re-install the kernel I can no longer boot the guest, it simply hangs.  There's no console information or anything.

Reproducible: Always

Steps to Reproduce:
1. Download a qcow2 image of fedora 38
2. Use virt-customize to inject your ssh key or set a root password.
3. Do an installer with
virt-install --memory 4096 --vcpus 2 --name testvm \
        --import --disk $VM_IMAGE,format=qcow2,bus=virtio \
        --os-variant fedora38 \
        --network bridge=virbr0,model=virtio \
        --graphics none \
        --noautoconsole
3. Log into the vm, run dnf reinstall kernel-core, reboot
Actual Results:  
The vm no longer boots, it just hangs.

Expected Results:  
I would like it to actually boot.

I have this same workflow working on an older fedora machine and it's working fine.

Comment 1 Richard W.M. Jones 2023-06-30 07:46:43 UTC
You'll need to add a serial console to the guest, then interrupt grub and
edit the kernel command line.  Remove rhgb quiet.  Add console=ttyS0.  Hit
Ctrl+x to boot and see where its hanging.

Comment 2 Josef Bacik 2023-06-30 13:40:44 UTC
That makes it work, which is maddening.  This has gotten a bit confusing,

WORKING CASE 1:
1) create guest in the above described way.
2) grubby --remove-args="rhgb quiet" --args=console=ttyS0,115200 --update-kernel=DEFAULT
3) dnf reinstall -y kernel-core
4) reboot

WORKING CASE 2:
1) add '--boot uefi' to the above virt-install command.
2) dnf reinstall -y kernel-core
3) reboot

BROKEN CASE:
1) create guest in the above described way
2) dnf reinstall -y kernel-core (or even make install from a kernel source which is what I was trying to do originally)
3) reboot

The image is setup to be UEFI booted, it's got a UEFI partition and everything.  With the original virt-install command it gets booted in bios mode.  If I mess with the grubby options then something changes with how we do the grub install to make it boot properly.  In case #1 we're still booted with bios mode when I reboot.  In the broken case it appears to break the bios booting, I'm not entirely sure how.  With case #2 we're just always UEFI and everything is happy.

Comment 3 Richard W.M. Jones 2023-06-30 13:51:54 UTC
What's the full output from the kernel when it hangs?

Comment 4 Josef Bacik 2023-06-30 14:00:17 UTC
There's no output, it just sits there and spins.

Comment 5 Josef Bacik 2023-06-30 14:00:54 UTC
I don't think it's even getting to grub, the VM just spins on a cpu.

Comment 7 Josef Bacik 2023-06-30 16:00:06 UTC
I have the corresponding fix in my devel tree that I was having the same issue with yesterday, it's how I noticed the problem, I did the dnf reinstall thing to make sure it wasn't an upstream bug.

Comment 8 Aoife Moloney 2024-05-28 13:17:06 UTC
Fedora Linux 38 entered end-of-life (EOL) status on 2024-05-21.

Fedora Linux 38 is no longer maintained, which means that it
will not receive any further security or bug fix updates. As a result we
are closing this bug.

If you can reproduce this bug against a currently maintained version of Fedora Linux
please feel free to reopen this bug against that version. Note that the version
field may be hidden. Click the "Show advanced fields" button if you do not see
the version field.

If you are unable to reopen this bug, please file a new report against an
active release.

Thank you for reporting this bug and we are sorry it could not be fixed.


Note You need to log in before you can comment on or make changes to this bug.