RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 2007257 - Can't get PXE boot logs from libvirt guest on a serial console
Summary: Can't get PXE boot logs from libvirt guest on a serial console
Keywords:
Status: CLOSED DUPLICATE of bug 2003092
Alias: None
Product: Red Hat Enterprise Linux 9
Classification: Red Hat
Component: qemu-kvm
Version: 9.0
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: rc
: ---
Assignee: Klaus Heinrich Kiwi
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-09-23 12:26 UTC by Matej Marušák
Modified: 2021-10-08 02:30 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-10-07 20:02:23 UTC
Type: Bug
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHELPLAN-98051 0 None None None 2021-09-23 12:26:52 UTC

Description Matej Marušák 2021-09-23 12:26:10 UTC 
In Cockpit after updating testing VM one test started to fail. From updated packages qemu-kvm seem the most suspicious. With previous version `/tmp/serial.txt` contained the output.
Affected version 6.1.0-2.el9
Previous working version: 6.0.0-12.el9

Steps to reproduce:

1. Create a libvirt Virtual Network with tftp server configured. Example XML:

# virsh net-dumpxml pxe-nat
<network>
  <name>pxe-nat</name>
  <forward mode='nat'>
    <nat>
      <port start='1024' end='65535'/>
    </nat>
  </forward>
  <bridge name='virbr0' stp='on' delay='0'/>
  <mac address='52:54:00:53:7d:8e'/>
  <ip address='192.168.122.1' netmask='255.255.255.0'>
    <tftp root='/var/lib/libvirt/pxe-config'/>
    <dhcp>
      <range start='192.168.122.2' end='192.168.122.254'/>
      <bootp file='pxe.cfg'/>
    </dhcp>
  </ip>
</network>

2. Create the IPXE configuration file with some dummy message to
expect in the serial console.
# cat /var/lib/libvirt/pxe-config/pxe.cfg
#!ipxe

echo Rebooting in 60 seconds
sleep 60
reboot

3. Create a guest OS with no disks, configured to have network as boot
device. The following virt-install command can be used:

virt-install --name test-1 --pxe --network network=pxe-nat --boot
useserial=yes --console log.file=/tmp/serial.txt --disk none --memory
500

Expected results:
The /tmp/serial.txt file should be populated with the ipxe logs.

Additional info:
journalctl -u libvirtd doesn't have any errors/warnings.
The host is actually a VM, with kvm module not loaded, thus I
am using qemu emulation for the nested guest.

Note tha the issue is not related to the serial console to file
redirection. If we don't speficy file type for console and just run
the command like this:
virt-install --name test-1 --pxe --network network=pxe-nat --boot
useserial=yes --nographics --disk none --memory 500

we still will not see the PXE logs in the terminal.

Let me know if I can provide with additional files for debugging.
Comment 1 yalzhang@redhat.com 2021-09-24 06:41:19 UTC 
I also hit this bug on qemu-kvm-6.1.0-2.el9.x86_64, it is a regression and not happened on qemu-kvm-6.0.0-30.module+el8.5.0+12586+476da3e1.x86_64.

Detail steps as below:
1. configure a vm as below, set it boot from network, and enable Serial Graphics Adapter which allows users to see BIOS messages on a serial port:

 <os>
    <type arch='x86_64' machine='pc-q35-rhel8.5.0'>hvm</type>
    <boot dev='network'/>
    <**bios useserial='yes'** rebootTimeout='-1'/>
  </os>
...
 <interface type='bridge'>
      <mac address='52:54:00:bf:59:6f'/>
      <source bridge='br0'/>
      <target dev='vnet3'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>
    </interface>

2. start the vm and connect to the serial console, there are no outputs from console:
# virsh start vm2 --console
Domain 'vm2' started
Connected to domain 'vm2'
Escape character is ^] (Ctrl + ])

3. but check the vm status from virt-viewer, there are boot messages about pxe like:
SeaBIOS (version 1.14.0-7.el9)
Machine UUID xxxxxx

iPXE (http://ipxe.org) 07:00.0 CA00 PCI2.10 PnP .......
Booting from ROM...
Comment 2 John Ferlan 2021-10-01 19:53:00 UTC 
Assigned to Klaus for initial triage per bz process and age of bug created or assigned to virt-maint without triage.

Unclear if this is a graphics bug or a bios bug - please change the subcomponent when assigning!
Comment 3 Klaus Heinrich Kiwi 2021-10-04 18:49:09 UTC 
I *believe* this is a symptom/duplicate with us removing the SGA device without libvirt being prepared for it (yet) - see Bug 2002993

If you can get it back to work with the version specified in 'fixed in version' in that bug (or higher), please close this as duplicate for that bug. Otherwise, I'll ask for some more investigation.
Comment 4 Matej Marušák 2021-10-05 05:30:09 UTC 
> I *believe* this is a symptom/duplicate with us removing the SGA device without libvirt being prepared for it

I am not entirely sure. Our CI hit the problem I reported originally about a week ago. Then yesterday we got yet another rhel-9 update and now we fail to `virsh create` if our xml contains `<bios useserial='yes' />`. It now fails with `qemu does not support SGA`.

> If you can get it back to work with the version specified in 'fixed in version' in that bug (or higher)...

It says it is fixed in version 6.0.0, but in both cases I described we had version 6.1.0
Comment 5 Klaus Heinrich Kiwi 2021-10-05 12:20:44 UTC 
(In reply to Matej Marušák from comment #4)
> > I *believe* this is a symptom/duplicate with us removing the SGA device without libvirt being prepared for it
> 
> I am not entirely sure. Our CI hit the problem I reported originally about a
> week ago. Then yesterday we got yet another rhel-9 update and now we fail to
> `virsh create` if our xml contains `<bios useserial='yes' />`. It now fails
> with `qemu does not support SGA`.
> 
> > If you can get it back to work with the version specified in 'fixed in version' in that bug (or higher)...
> 
> It says it is fixed in version 6.0.0, but in both cases I described we had
> version 6.1.0

It's complicated (apologies for that)..

RHEL9-Beta disabled the SGA device without the changes in libvirt and guestfs that would prevent us from having regressions...

So this downstream-only change was reverted through Bug 2002993 in the package qemu-kvm-6.0.0-13.el9_b.4 (which is available only in the RHEL9-Beta stream, through Errata https://errata.devel.redhat.com/advisory/70225

RHEL9-GA (post-beta stream, which is currently using the qemu-6.1 rebase) doesn't have this BZ reverted, because libvirt and guestfs are in the process of being fixed in that release - but those might not be merged in the product yet.

So to check if this a duplicate, please make sure you are running RHEL9-Beta fully updated (that means your qemu package should be qemu-kvm-6.0.0-13.el9_b.4 or higher, but still in the qemu-6.0.0 base.. 

Thanks,

 -Klaus
Comment 6 Matej Marušák 2021-10-07 08:24:37 UTC 
Yes, we see this in 9-GA. Beta worked fine for us.
Comment 7 Klaus Heinrich Kiwi 2021-10-07 20:02:23 UTC 
> Yes, we see this in 9-GA. Beta worked fine for us.

Thanks. So marking this as a Dupe for Bug 2003092 for now, but feel free to reopen if there's some aspect that is not addressed.

 -Klaus

*** This bug has been marked as a duplicate of bug 2003092 ***
Note You need to log in before you can comment on or make changes to this bug.