Libguestfs no longer works with seabios 1.17.0 (worked fine with 1.16.3-4.fc42). Specifically virtio-pci seems to be broken, so no virtio-scsi etc devices can be seen. Although this affects libguestfs, it seems to be a general problem. With 1.16 (working) we see virtio-pci messages like this: [ 3.349205] ACPI: \_SB_.LNKS: No IRQ available. Try pci=noacpi or acpi=off [ 3.349791] pcieport 0000:00:01.0: PCI INT A: no GSI [ 3.370839] ACPI: \_SB_.LNKS: No IRQ available. Try pci=noacpi or acpi=off [ 3.371148] pcieport 0000:00:01.1: PCI INT A: no GSI [ 3.377850] ACPI: \_SB_.LNKS: No IRQ available. Try pci=noacpi or acpi=off [ 3.378147] pcieport 0000:00:01.2: PCI INT A: no GSI [ 3.385052] ACPI: \_SB_.LNKS: No IRQ available. Try pci=noacpi or acpi=off [ 3.385384] pcieport 0000:00:01.3: PCI INT A: no GSI [ 3.395282] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.406833] ACPI: button: Power Button [PWRF] [ 3.415676] ACPI: \_SB_.LNKS: No IRQ available. Try pci=noacpi or acpi=off [ 3.415963] virtio-pci 0000:01:00.0: PCI INT A: no GSI [ 3.421829] ACPI: \_SB_.LNKS: No IRQ available. Try pci=noacpi or acpi=off [ 3.422098] virtio-pci 0000:02:00.0: PCI INT A: no GSI [ 3.425255] ACPI: \_SB_.LNKS: No IRQ available. Try pci=noacpi or acpi=off [ 3.425550] virtio-pci 0000:03:00.0: PCI INT A: no GSI With 1.17 (broken): [ 3.284196] pcieport 0000:00:01.0: PCI->APIC IRQ transform: INT A -> IRQ 10 [ 3.302387] pcieport 0000:00:01.0: PME: Signaling with IRQ 24 [ 3.306012] pcieport 0000:00:01.0: pciehp: Slot #0 AttnBtn+ PwrCtrl+ MRL- AttnInd+ PwrInd+ HotPlug+ Surprise+ Interlock+ NoCompl- IbPresDis- LLActRep+ [ 3.314950] pcieport 0000:00:01.0: pciehp: Slot(0): Card not present [ 3.315310] pcieport 0000:00:01.0: pciehp: Slot(0): Already disabled [ 3.319718] pcieport 0000:00:01.1: PCI->APIC IRQ transform: INT A -> IRQ 10 [ 3.323070] pcieport 0000:00:01.1: PME: Signaling with IRQ 25 [ 3.323966] pcieport 0000:00:01.1: pciehp: Slot #0 AttnBtn+ PwrCtrl+ MRL- AttnInd+ PwrInd+ HotPlug+ Surprise+ Interlock+ NoCompl- IbPresDis- LLActRep+ [ 3.326756] pcieport 0000:00:01.1: pciehp: Slot(0-1): Card not present [ 3.327103] pcieport 0000:00:01.1: pciehp: Slot(0-1): Already disabled [ 3.328802] pcieport 0000:00:01.2: PCI->APIC IRQ transform: INT A -> IRQ 10 [ 3.331523] pcieport 0000:00:01.2: PME: Signaling with IRQ 26 [ 3.332491] pcieport 0000:00:01.2: pciehp: Slot #0 AttnBtn+ PwrCtrl+ MRL- AttnInd+ PwrInd+ HotPlug+ Surprise+ Interlock+ NoCompl- IbPresDis- LLActRep+ [ 3.335062] pcieport 0000:00:01.2: pciehp: Slot(0-2): Card not present [ 3.335294] pcieport 0000:00:01.2: pciehp: Slot(0-2): Already disabled [ 3.336583] pcieport 0000:00:01.3: PCI->APIC IRQ transform: INT A -> IRQ 10 [ 3.339542] pcieport 0000:00:01.3: PME: Signaling with IRQ 27 [ 3.340379] pcieport 0000:00:01.3: pciehp: Slot #0 AttnBtn+ PwrCtrl+ MRL- AttnInd+ PwrInd+ HotPlug+ Surprise+ Interlock+ NoCompl- IbPresDis- LLActRep+ [ 3.357059] virtio-pci 0000:01:00.0: Unable to change power state from D3cold to D0, device inaccessible [ 3.359371] virtio-pci 0000:01:00.0: using bridge 0000:00:01.0 INT C to get IRQ 10 [ 3.359677] virtio-pci 0000:01:00.0: PCI->APIC IRQ transform: INT C -> IRQ 10 [ 3.360359] virtio-pci 0000:01:00.0: virtio_pci: leaving for legacy driver [ 3.362412] virtio-pci 0000:02:00.0: Unable to change power state from D3cold to D0, device inaccessible [ 3.362726] virtio-pci 0000:02:00.0: using bridge 0000:00:01.1 INT C to get IRQ 10 [ 3.363083] virtio-pci 0000:02:00.0: PCI->APIC IRQ transform: INT C -> IRQ 10 [ 3.363299] virtio-pci 0000:02:00.0: virtio_pci: leaving for legacy driver [ 3.364328] virtio-pci 0000:03:00.0: Unable to change power state from D3cold to D0, device inaccessible [ 3.364603] virtio-pci 0000:03:00.0: using bridge 0000:00:01.2 INT C to get IRQ 10 [ 3.364983] virtio-pci 0000:03:00.0: PCI->APIC IRQ transform: INT C -> IRQ 10 [ 3.365211] virtio-pci 0000:03:00.0: virtio_pci: leaving for legacy driver Then later, loading the virtio-scsi module shows no virtio-scsi devices, where we expect to see two. Reproducible: Always Steps to Reproduce: 1. Install seabios-bin 1.17.0 in Fedora 2. Run 'libguestfs-test-tool'
Created attachment 2093692 [details] build.log build.log from broken libguestfs build in Koji.
Created attachment 2093693 [details] libguestfs-test-tool output
Hmm. With seabios 1.17.0 installed qemu ends up being started with '-machine acpi=off' when running libguestfs-test-tool. Use 'ps | grep acpi' to see that. Not obvious why. When searching the logs the first time 'acpi' shows up is in the kernel boot log: [ 0.045524] ACPI: Early table checksum verification disabled [ 0.045749] ACPI BIOS Error (bug): A valid RSDP was not found (20240827/tbxfroot-222) Specifically there is *nothing* in the xml passed to libvirt by libguestfs. So apparently libvirt figures by itself it should turn off acpi for some reason?
Apparently you have to add <features> <acpi/> </features> else libvirt does this. Adding that fixes the problem. How did it work before?
libguestfs fix: https://github.com/libguestfs/libguestfs/commit/7cf0ed750ef7119503fc72cb19c29a269a893e8e
(In reply to Richard W.M. Jones from comment #4) > Apparently you have to add <features> <acpi/> </features> else libvirt does > this. > > Adding that fixes the problem. > > How did it work before? Well. After downgrading seabios to 1.16.3 libguestfs-test-tool works fine again, and I can see qemu being started with "acpi=on". Libvirt seems to pick different defaults depending on the seabios version installed (if the acpi feature is not explicitly specified in xml).
(In reply to Richard W.M. Jones from comment #5) > libguestfs fix: > https://github.com/libguestfs/libguestfs/commit/ > 7cf0ed750ef7119503fc72cb19c29a269a893e8e Explicitly asking for acpi being enables surely doesn't hurt. Nevertheless libvirt should not turn off acpi on its own. Reassigning to libvirt for investigation what is going on here.
(In reply to Richard W.M. Jones from comment #5) > libguestfs fix: > https://github.com/libguestfs/libguestfs/commit/ > 7cf0ed750ef7119503fc72cb19c29a269a893e8e Except that doesn't work because you can only use ACPI on some architectures, sigh. It failed to build on ppc64 & s390x: https://koji.fedoraproject.org/koji/taskinfo?taskID=133842269 As a temporary fix I made it only use the feature flag on x86 & Arm: https://github.com/libguestfs/libguestfs/commit/f6fe0611a8980694b8e4e33b9c824805d353679f
(In reply to Gerd Hoffmann from comment #6) > (In reply to Richard W.M. Jones from comment #4) > > Apparently you have to add <features> <acpi/> </features> else libvirt does > > this. > > > > Adding that fixes the problem. > > > > How did it work before? > > Well. After downgrading seabios to 1.16.3 libguestfs-test-tool works fine > again, and I can see qemu being started with "acpi=on". Libvirt seems to > pick different defaults depending on the seabios version installed (if the > acpi feature is not explicitly specified in xml). I don't see that behaviour & it doesn't match what libvirt code does either. In my testing libvirt is using acpi=off with *both* old and new seabios for libguestfs' XML config. This is what I'd expect as libvirt doesn't consider BIOS when deciding whether to set acpi=off. It merely asks QEMU if the machine has an 'acpi' property, and if so, sets it based on existence (or lack) of the <acpi> XML feature. Bisecting the seabios changes, the problem is caused by this change: commit 35aa9a72c279fb2ed3a4917eef8b7155bb7de84c Author: Gerd Hoffmann <kraxel> Date: Wed Nov 20 12:42:13 2024 +0100 drop obsolete acpi table code It's there for backward compatibility with qemu 1.6 and older. This release is older than a decade. Even qemu itself has removed backward compatibility support (i.e. machine types) for qemu versions that old. It should be safe to remove this code now. The old SeaBIOS code would unconditionally add ACPI tables for the VM regardless of whether QEMU had acpi=off or acpi=on. With that seabios code removed, the guest now lacks ACPI tables when acpi=off. So this is both a regression (because it breaks libguestfs) and a bug fix (because we correctly honour acpi=off now) at the same time. I'd say this is ok for rawhide, but we should not push this to stable releases.
(In reply to Richard W.M. Jones from comment #8) > (In reply to Richard W.M. Jones from comment #5) > > libguestfs fix: > > https://github.com/libguestfs/libguestfs/commit/ > > 7cf0ed750ef7119503fc72cb19c29a269a893e8e > > Except that doesn't work because you can only use ACPI on some > architectures, sigh. > It failed to build on ppc64 & s390x: > > https://koji.fedoraproject.org/koji/taskinfo?taskID=133842269 > > As a temporary fix I made it only use the feature flag on x86 & Arm: > > https://github.com/libguestfs/libguestfs/commit/ > f6fe0611a8980694b8e4e33b9c824805d353679f You can query whether a given QEMU supports ACPI from the capabilities $ virsh capabilities --xpath //features/acpi <acpi default="on" toggle="yes"/> <acpi default="on" toggle="yes"/> <acpi default="on" toggle="yes"/> <acpi default="on" toggle="yes"/> If it reports toggle=yes, then it is valid to set the <acpi/> feature. Ignore the 'default' value, that reflects QEMU's built-in default, not libvirt's XML defaults $ virsh capabilities --xpath '//guest[count(features/acpi) = 1]/arch/@name' name="aarch64" name="i686" name="loongarch64" name="x86_64"
> > Well. After downgrading seabios to 1.16.3 libguestfs-test-tool works fine > > again, and I can see qemu being started with "acpi=on". Libvirt seems to > > pick different defaults depending on the seabios version installed (if the > > acpi feature is not explicitly specified in xml). > > I don't see that behaviour & it doesn't match what libvirt code does either. Double-checked. You are correct. > Bisecting the seabios changes, the problem is caused by this change: > > commit 35aa9a72c279fb2ed3a4917eef8b7155bb7de84c > Author: Gerd Hoffmann <kraxel> > Date: Wed Nov 20 12:42:13 2024 +0100 > > drop obsolete acpi table code > > It's there for backward compatibility with qemu 1.6 and older. This > release is older than a decade. Even qemu itself has removed backward > compatibility support (i.e. machine types) for qemu versions that old. > > It should be safe to remove this code now. > > The old SeaBIOS code would unconditionally add ACPI tables for the VM > regardless of whether QEMU had acpi=off or acpi=on. > > With that seabios code removed, the guest now lacks ACPI tables when > acpi=off. seabios had static acpi tables for the 'pc' machine type compiled in as fallback for old qemu versions. I wasn't aware that these have been used with 'acpi=off' too. It's kind of amazing the linux kernel could actually boot with that with the 'q35' machine type being used, given that IRQ routing information (and probably more) in the fallback acpi tables is totally bonkers then. > So this is both a regression (because it breaks libguestfs) and a bug fix > (because we correctly honour acpi=off now) at the same time. > > I'd say this is ok for rawhide, but we should not push this to stable > releases. Lets see if there is more fallout. I somehow doubt there will be much given how many features depend on proper ACPI tables these days.
Hi, > https://github.com/libguestfs/libguestfs/commit/ > f6fe0611a8980694b8e4e33b9c824805d353679f What is the status / what are the plans for rawhide / f42 / f41? I think it makes sense for the new seabios to conflict with old libguestfs versions.
I backported this commit (and a follow-on) to F41, F42 & Rawhide already. However I can't recall if I built it in F41 & F42 or not. If not then it'll get fixed fairly soon anyway as there are some more upstream fixes (unrelated) that need to go into those branches.
(In reply to Richard W.M. Jones from comment #13) > I backported this commit (and a follow-on) to F41, F42 & Rawhide already. > However > I can't recall if I built it in F41 & F42 or not. If not then it'll get > fixed > fairly soon anyway as there are some more upstream fixes (unrelated) that > need to > go into those branches. Ok, can you add the version numbers here once the next build is complete, so I can tweak the seabios updates accordingly?
It'll be fixed in libguestfs 1.56.1. I wouldn't bother waiting, if there's some breakage we can deal with it.
(In reply to Richard W.M. Jones from comment #15) > It'll be fixed in libguestfs 1.56.1. I wouldn't bother waiting, if there's > some breakage we can deal with it. Yep, a version number is good enough, I don't need an actual build. I'll go add this to seabios: Conflicts: libguestfs < 1.56.1
Conflicts: libguestfs < 1:1.56.1 since we're on epoch = 1
FEDORA-2025-1e1f6594c6 (seabios-1.17.0-4.fc42) has been submitted as an update to Fedora 42. https://bodhi.fedoraproject.org/updates/FEDORA-2025-1e1f6594c6
FEDORA-2025-dde8d7958b (seabios-1.17.0-4.fc41) has been submitted as an update to Fedora 41. https://bodhi.fedoraproject.org/updates/FEDORA-2025-dde8d7958b
FEDORA-2025-1e1f6594c6 has been pushed to the Fedora 42 testing repository. Soon you'll be able to install the update with the following command: `sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2025-1e1f6594c6` You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2025-1e1f6594c6 See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
FEDORA-2025-dde8d7958b has been pushed to the Fedora 41 testing repository. Soon you'll be able to install the update with the following command: `sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2025-dde8d7958b` You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2025-dde8d7958b See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.