Bug 1738450 - For intel-iommu, qemu shows conflict behaviors between booting a guest with vfio and hot plugging vfio device
Summary: For intel-iommu, qemu shows conflict behaviors between booting a guest with v...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux Advanced Virtualization
Classification: Red Hat
Component: qemu-kvm
Version: 8.1
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: rc
: ---
Assignee: Peter Xu
QA Contact: Pei Zhang
URL:
Whiteboard:
Depends On: 1738440
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-08-07 08:16 UTC by Pei Zhang
Modified: 2020-12-20 09:21 UTC (History)
11 users (show)

Fixed In Version: qemu-kvm-4.2.0-1.module+el8.2.0+4793+b09dd2fb
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1738440
Environment:
Last Closed: 2020-05-05 09:47:43 UTC
Type: Bug
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Pei Zhang 2019-08-07 08:16:39 UTC
+++ This bug was initially created as a clone of Bug #1738440 +++

Description of problem:

(1)Booting guest with device assignment and vIOMMU without "caching-mode=on", qemu can work well. 

(2)However when booting guest with vIOMMU without "caching-mode=on", then hot plug a vfio device, qemu will quit.

So qemu shows conflict behaviors. If caching-mode=on is a must for intel-iommu to enable device assignment with IOMMU protection, qemu should reject to boot in scenario (1). If caching-mode=on is not a must, qemu should allow hot plug a vfio device.


Version-Release number of selected component (if applicable):
4.18.0-128.el8.x86_64
qemu-kvm-2.12.0-83.module+el8.1.0+3852+0ba8aef0.x86_64


How reproducible:
100%

Steps to Reproduce:
1. Boot qemu with vIOMMU without "caching-mode=on", full cmd refer to [1]

-device intel-iommu,intremap=on,device-iotlb=on \

2. Hot plug a vfio device, qemu quit.

(qemu) device_add vfio-pci,host=0000:5e:00.0,bus=root.3 
We need to set caching-mode=1 for intel-iommu to enable device assignment with IOMMU protection.
# 

3. Boot qemu with vfio device and vIOMMU without "caching-mode=on", qemu works well.

-device intel-iommu,intremap=on,device-iotlb=on \
-device vfio-pci,host=0000:5e:00.0,bus=root.3 \


Actual results:
qemu shows conflict behaviors when boot/hot-plug vfio with vIOMMU but without caching-mode=on.

Expected results:
qemu should show consistent behavior when boot/hot-plug vfio with vIOMMU but without caching-mode=on.

Additional info:
1. This bug has been closed in rhel7.
Bug 1441605 - qemu crash when attach a hostdev device to the guest with intel-iommu device enabled

2. Bug 1622209 - libvirt should notify the user if caching-mode isn't specified for iommu emulation in certain cases 


Reference:
[1]
/usr/libexec/qemu-kvm \
-name rhel8.1 \
-M q35,kernel-irqchip=split \
-cpu Haswell-noTSX \
-m 3G \
-device intel-iommu,intremap=on,device-iotlb=on \
-smp 4,sockets=1,cores=4,threads=1 \
-device pcie-root-port,id=root.1,chassis=1 \
-device pcie-root-port,id=root.2,chassis=2 \
-device pcie-root-port,id=root.3,chassis=3 \
-blockdev driver=file,cache.direct=off,cache.no-flush=on,filename=/home/images_nfv-virt-rt-kvm//rhel8.0.qcow2,node-name=my_file \
-blockdev driver=qcow2,node-name=my,file=my_file \
-device virtio-blk-pci,drive=my,id=virtio-blk0,bus=root.1 \
-vnc :1 \
-monitor stdio \
-netdev tap,id=hostnet0 \
-device virtio-net-pci,netdev=hostnet0,id=net0,mac=88:66:da:5f:dd:01,bus=root.2 \

Comment 1 Pei Zhang 2019-08-07 08:18:52 UTC
Versions:
4.18.0-128.el8.x86_64
qemu v4.1.0-rc3

Comment 6 Pei Zhang 2019-12-06 09:16:31 UTC
Verified this bz with qemu-kvm-4.2.0-1.module+el8.2.0+4793+b09dd2fb.x86_64:

Steps following Description.

After step 2, qemu prompts warning info and works well. This is expected behavior.

(qemu) device_add vfio-pci,host=0000:5e:00.0,bus=root.3 
Error: Device assignment is not allowed without enabling caching-mode=on for Intel IOMMU.


After step 3, qemu fails reboot without "caching-mode=on". This is expected behavior.

-device intel-iommu,intremap=on,device-iotlb=on \
-device vfio-pci,host=0000:5e:00.0,bus=root.3 \

(qemu) qemu-kvm: -device vfio-pci,host=0000:5e:00.0,bus=root.3: warning: vfio 0000:5e:00.0: failed to setup resample irqfd: Invalid argument
qemu-kvm: We need to set caching-mode=on for intel-iommu to enable device assignment with IOMMU protection.


So this bug has been fixed very well. Move to 'Verified'.

Comment 7 Ademar Reis 2020-02-05 23:02:11 UTC
QEMU has been recently split into sub-components and as a one-time operation to avoid breakage of tools, we are setting the QEMU sub-component of this BZ to "General". Please review and change the sub-component if necessary the next time you review this BZ. Thanks

Comment 9 errata-xmlrpc 2020-05-05 09:47:43 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:2017


Note You need to log in before you can comment on or make changes to this bug.