Bug 2087155

Summary: Guest will get stuck at "Reached target Basic System" if insert the virtio-iommu device in pcie-root-port
Product: Red Hat Enterprise Linux 9 Reporter: Yihuang Yu <yihyu>
Component: qemu-kvmAssignee: Eric Auger <eric.auger>
qemu-kvm sub component: Devices QA Contact: Yihuang Yu <yihyu>
Status: CLOSED ERRATA Docs Contact:
Severity: medium    
Priority: low CC: abologna, coli, eric.auger, gshan, jinzhao, juzhang, lijin, mrezanin, virt-maint, zhenyzha
Version: 9.1Keywords: Triaged
Target Milestone: rc   
Target Release: 9.2   
Hardware: aarch64   
OS: Linux   
Whiteboard:
Fixed In Version: qemu-kvm-7.2.0-1.el9 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-05-09 07:19:33 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1924294    

Description Yihuang Yu 2022-05-17 13:36:13 UTC
Description of problem:
If insert the virtio-iommu device in a pcie-root-port, the guest will get stuck at "Reached target Basic System" and finally enter the dracut system.

Version-Release number of selected component (if applicable):
Host kernel: 5.14.0-90.el9.aarch64
Guest kernel: 5.14.0-86.el9.aarch64
qemu version: qemu-kvm-7.0.0-3.el9.aarch64
edk2 version: edk2-aarch64-20220221gitb24306f15d-1.el9.noarch

How reproducible:
Always

Steps to Reproduce:
1. Launch a guest with virtio-iommu which insert in the pcie-root-port
MALLOC_PERTURB_=1  /usr/libexec/qemu-kvm \
    -name 'avocado-vt-vm1'  \
    -sandbox on  \
    -blockdev node-name=file_aavmf_code,driver=file,filename=/usr/share/edk2/aarch64/QEMU_EFI-silent-pflash.raw,auto-read-only=on,discard=unmap \
    -blockdev node-name=drive_aavmf_code,driver=raw,read-only=on,file=file_aavmf_code \
    -blockdev node-name=file_aavmf_vars,driver=file,filename=/home/kvm_autotest_root/images/avocado-vt-vm1_rhel910-aarch64-virtio.qcow2_VARS.fd,auto-read-only=on,discard=unmap \
    -blockdev node-name=drive_aavmf_vars,driver=raw,read-only=off,file=file_aavmf_vars \
    -machine virt,gic-version=host,memory-backend=mem-machine_mem,pflash0=drive_aavmf_code,pflash1=drive_aavmf_vars \
    -device pcie-root-port,id=pcie-root-port-0,multifunction=on,bus=pcie.0,addr=0x1,chassis=1 \
    -device pcie-pci-bridge,id=pcie-pci-bridge-0,addr=0x0,bus=pcie-root-port-0  \
    -nodefaults \
    -device pcie-root-port,id=pcie-root-port-1,port=0x1,addr=0x1.0x1,bus=pcie.0,chassis=2 \
    -device virtio-iommu-pci,bus=pcie-root-port-1,addr=0x0 \
    -device pcie-root-port,id=pcie-root-port-2,port=0x2,addr=0x1.0x2,bus=pcie.0,chassis=3 \
    -device virtio-gpu-pci,bus=pcie-root-port-2,addr=0x0 \
    -m 8192 \
    -object memory-backend-ram,size=8192M,id=mem-machine_mem  \
    -smp 4,maxcpus=4,cores=2,threads=1,sockets=2  \
    -cpu 'host' \
    -serial unix:'/tmp/serial-serial0',server=on,wait=off \
    -device pcie-root-port,id=pcie-root-port-3,port=0x3,addr=0x1.0x3,bus=pcie.0,chassis=4 \
    -device qemu-xhci,id=usb1,bus=pcie-root-port-3,addr=0x0 \
    -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \
    -blockdev node-name=file_image1,driver=file,auto-read-only=on,discard=unmap,aio=threads,filename=/home/kvm_autotest_root/images/rhel910-aarch64-virtio.qcow2,cache.direct=on,cache.no-flush=off \
    -blockdev node-name=drive_image1,driver=qcow2,read-only=off,cache.direct=on,cache.no-flush=off,file=file_image1 \
    -device pcie-root-port,id=pcie-root-port-4,port=0x4,addr=0x1.0x4,bus=pcie.0,chassis=5 \
    -device virtio-blk-pci,id=image1,drive=drive_image1,bootindex=0,write-cache=on,bus=pcie-root-port-4,addr=0x0 \
    -device pcie-root-port,id=pcie-root-port-5,port=0x5,addr=0x1.0x5,bus=pcie.0,chassis=6 \
    -device virtio-net-pci,mac=9a:8e:da:06:5e:22,rombar=0,id=id3Znk3z,netdev=idjOsJKR,bus=pcie-root-port-5,addr=0x0  \
    -netdev tap,id=idjOsJKR,vhost=on  \
    -vnc :0  \
    -rtc base=utc,clock=host,driftfix=slew \
    -enable-kvm \
    -monitor stdio

2. Check the status of the guest
# nc -U /tmp/serial-serial0

3.

Actual results:
......
......
[  OK  ] Reached target Local File Systems.
[  OK  ] Reached target System Initialization.
[  OK  ] Reached target Basic System.
[  157.476029] dracut-initqueue[465]: Warning: dracut-initqueue: timeout, still waiting for following initqueue hooks:
[  157.476798] dracut-initqueue[465]: Warning: /lib/dracut/hooks/initqueue/finished/devexists-\x2fdev\x2fmapper\x2frhel_dhcp156--104-root.sh: "if ! grep -q After=remote-fs-pre.target /run/systemd/generator/systemd-cryptsetup@*.service 2>/dev/null; then
......
......
         Starting Dracut Emergency Shell...
Warning: /dev/mapper/rhel_dhcp156--104-root does not exist
Warning: /dev/rhel_dhcp156-104/root does not exist
Warning: /dev/rhel_dhcp156-104/swap does not exist

Generating "/run/initramfs/rdsosreport.txt"

Expected results:
The guest should boot up successfully.

Additional info:

Comment 1 Eric Auger 2022-05-17 14:05:52 UTC
According to Andrea this is not a topology that is used by libvirt so it is low priority and does not block the feature. However it should not prevent the guest from booting so that's worth debugging.

Comment 5 Eric Auger 2022-11-14 15:20:13 UTC
Fix: e72cfabf4e  hw/virtio/virtio-iommu-pci: Enforce the device is plugged on the root bus has landed upstream (v7.2.0-rc1). Move to POST.

Comment 9 Yihuang Yu 2022-12-20 03:39:46 UTC
Verify with qemu-kvm-7.2.0-1.el9.aarch64

# /usr/libexec/qemu-kvm -version
QEMU emulator version 7.2.0 (qemu-kvm-7.2.0-1.el9)
Copyright (c) 2003-2022 Fabrice Bellard and the QEMU Project developers

# /usr/libexec/qemu-kvm -M virt -accel kvm -cpu host -device pcie-root-port,id=pcie-root-port-0,multifunction=on,bus=pcie.0,addr=0x1,chassis=1 -device virtio-iommu-pci,addr=0x0,bus=pcie-root-port-0 -nodefaults
qemu-kvm: -device virtio-iommu-pci,addr=0x0,bus=pcie-root-port-0: virtio-iommu-pci must be plugged on the root bus

Comment 13 Yihuang Yu 2022-12-24 04:30:27 UTC
Move to "VERIFIED" based on comment 9

Comment 15 errata-xmlrpc 2023-05-09 07:19:33 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: qemu-kvm security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2023:2162