+++ This bug was initially created as a clone of Bug #1738440 +++ Description of problem: (1)Booting guest with device assignment and vIOMMU without "caching-mode=on", qemu can work well. (2)However when booting guest with vIOMMU without "caching-mode=on", then hot plug a vfio device, qemu will quit. So qemu shows conflict behaviors. If caching-mode=on is a must for intel-iommu to enable device assignment with IOMMU protection, qemu should reject to boot in scenario (1). If caching-mode=on is not a must, qemu should allow hot plug a vfio device. Version-Release number of selected component (if applicable): 4.18.0-128.el8.x86_64 qemu-kvm-2.12.0-83.module+el8.1.0+3852+0ba8aef0.x86_64 How reproducible: 100% Steps to Reproduce: 1. Boot qemu with vIOMMU without "caching-mode=on", full cmd refer to [1] -device intel-iommu,intremap=on,device-iotlb=on \ 2. Hot plug a vfio device, qemu quit. (qemu) device_add vfio-pci,host=0000:5e:00.0,bus=root.3 We need to set caching-mode=1 for intel-iommu to enable device assignment with IOMMU protection. # 3. Boot qemu with vfio device and vIOMMU without "caching-mode=on", qemu works well. -device intel-iommu,intremap=on,device-iotlb=on \ -device vfio-pci,host=0000:5e:00.0,bus=root.3 \ Actual results: qemu shows conflict behaviors when boot/hot-plug vfio with vIOMMU but without caching-mode=on. Expected results: qemu should show consistent behavior when boot/hot-plug vfio with vIOMMU but without caching-mode=on. Additional info: 1. This bug has been closed in rhel7. Bug 1441605 - qemu crash when attach a hostdev device to the guest with intel-iommu device enabled 2. Bug 1622209 - libvirt should notify the user if caching-mode isn't specified for iommu emulation in certain cases Reference: [1] /usr/libexec/qemu-kvm \ -name rhel8.1 \ -M q35,kernel-irqchip=split \ -cpu Haswell-noTSX \ -m 3G \ -device intel-iommu,intremap=on,device-iotlb=on \ -smp 4,sockets=1,cores=4,threads=1 \ -device pcie-root-port,id=root.1,chassis=1 \ -device pcie-root-port,id=root.2,chassis=2 \ -device pcie-root-port,id=root.3,chassis=3 \ -blockdev driver=file,cache.direct=off,cache.no-flush=on,filename=/home/images_nfv-virt-rt-kvm//rhel8.0.qcow2,node-name=my_file \ -blockdev driver=qcow2,node-name=my,file=my_file \ -device virtio-blk-pci,drive=my,id=virtio-blk0,bus=root.1 \ -vnc :1 \ -monitor stdio \ -netdev tap,id=hostnet0 \ -device virtio-net-pci,netdev=hostnet0,id=net0,mac=88:66:da:5f:dd:01,bus=root.2 \
Versions: 4.18.0-128.el8.x86_64 qemu v4.1.0-rc3
Verified this bz with qemu-kvm-4.2.0-1.module+el8.2.0+4793+b09dd2fb.x86_64: Steps following Description. After step 2, qemu prompts warning info and works well. This is expected behavior. (qemu) device_add vfio-pci,host=0000:5e:00.0,bus=root.3 Error: Device assignment is not allowed without enabling caching-mode=on for Intel IOMMU. After step 3, qemu fails reboot without "caching-mode=on". This is expected behavior. -device intel-iommu,intremap=on,device-iotlb=on \ -device vfio-pci,host=0000:5e:00.0,bus=root.3 \ (qemu) qemu-kvm: -device vfio-pci,host=0000:5e:00.0,bus=root.3: warning: vfio 0000:5e:00.0: failed to setup resample irqfd: Invalid argument qemu-kvm: We need to set caching-mode=on for intel-iommu to enable device assignment with IOMMU protection. So this bug has been fixed very well. Move to 'Verified'.
QEMU has been recently split into sub-components and as a one-time operation to avoid breakage of tools, we are setting the QEMU sub-component of this BZ to "General". Please review and change the sub-component if necessary the next time you review this BZ. Thanks
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:2017