Bug 2094252
Summary: | Compile the virtio-iommu device on x86_64 | ||
---|---|---|---|
Product: | Red Hat Enterprise Linux 9 | Reporter: | Eric Auger <eric.auger> |
Component: | qemu-kvm | Assignee: | Eric Auger <eric.auger> |
qemu-kvm sub component: | General | QA Contact: | jinl |
Status: | CLOSED ERRATA | Docs Contact: | |
Severity: | medium | ||
Priority: | medium | CC: | berrange, coli, jinl, jinzhao, juzhang, mst, virt-maint |
Version: | 9.0 | Keywords: | RFE, Triaged |
Target Milestone: | rc | ||
Target Release: | --- | ||
Hardware: | x86_64 | ||
OS: | Linux | ||
Whiteboard: | |||
Fixed In Version: | qemu-kvm-7.0.0-7.el9 | Doc Type: | If docs needed, set a value |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | 2022-11-15 09:54:42 UTC | Type: | Feature Request |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: |
Description
Eric Auger
2022-06-07 09:40:11 UTC
IIUC, on x86_64 we already have the intel-iommu device available. Is there some benefit from also having the virtio-iommu on x86_64 as well ? Presumably the intel-iommu is more portable for guest OS since it is not reliant on the virtio transport. So if we have 2 options on x86_674, what criteria should apps use to decide which to use ? Hi Daniel. In the current state of the driver and qemu device I don't see any great added value compared to the intel-iommu as the perf will be worse than with the intel-iommu. See this old presentation, slide 22: http://events17.linuxfoundation.org/sites/events/files/slides/viommu_arm.pdf So at the moment apps should stick to the intel-iommu. However, Jerry, our kernel IOMMU maintainer said he has some long standing requests to enable the feature for customer and mostly for Intel to start "playing" with it. I think potential customer are interested in next "promised" virtualization related optimizations that would be more likely implemented in the virtio-iommu driver rather than in a native iommu driver. Today most people now look at looming virtual shared virtual memory support (ie. PASID). However PASID may be enabled along with intel-iommu first. This vSVM work is in progress and also depends on /dev/iommu. QE bot(pre verify): Set 'Verified:Tested,SanityOnly' as gating/tier1 test pass. Verify with qemu-kvm-7.0.0-7.el9.src.rpm guest kernel: kernel-5.14.0-104.mr955_220602_1540.el9.src.rpm 1)with intel host: qemu command line: /usr/libexec/qemu-kvm \ -name guest=v1,debug-threads=on \ -S \ -object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/var/lib/libvirt/qemu/domain-5-v1/master-key.aes"}' \ -blockdev '{"driver":"file","filename":"/usr/share/edk2/ovmf/OVMF_CODE.secboot.fd","node-name":"libvirt-pflash0-storage","auto-read-only":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-pflash0-format","read-only":true,"driver":"raw","file":"libvirt-pflash0-storage"}' \ -blockdev '{"driver":"file","filename":"/var/lib/libvirt/qemu/nvram/v1_VARS.fd","node-name":"libvirt-pflash1-storage","auto-read-only":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-pflash1-format","read-only":false,"driver":"raw","file":"libvirt-pflash1-storage"}' \ -machine pc-q35-rhel9.0.0,usb=off,smm=on,dump-guest-core=off,kernel_irqchip=split,pflash0=libvirt-pflash0-format,pflash1=libvirt-pflash1-format,memory-backend=pc.ram \ -accel kvm \ -cpu Broadwell-IBRS,vme=on,ss=on,vmx=on,pdcm=on,f16c=on,rdrand=on,hypervisor=on,arat=on,tsc-adjust=on,umip=on,md-clear=on,stibp=on,arch-capabilities=on,ssbd=on,xsaveopt=on,pdpe1gb=on,abm=on,ibpb=on,ibrs=on,amd-stibp=on,amd-ssbd=on,skip-l1dfl-vmentry=on,pschange-mc-no=on \ -global driver=cfi.pflash01,property=secure,value=on \ -m 8192 \ -object '{"qom-type":"memory-backend-ram","id":"pc.ram","size":8589934592}' \ -overcommit mem-lock=off \ -smp 4,sockets=1,dies=1,cores=4,threads=1 \ -uuid ffe7972d-b950-4e96-9a27-a7414786848d \ -no-user-config \ -nodefaults \ -chardev socket,id=charmonitor,fd=21,server=on,wait=off \ -mon chardev=charmonitor,id=monitor,mode=control \ -rtc base=utc,driftfix=slew \ -global kvm-pit.lost_tick_policy=delay \ -no-hpet \ -no-shutdown \ -global ICH9-LPC.disable_s3=1 \ -global ICH9-LPC.disable_s4=1 \ -boot strict=on \ -device '{"driver":"virtio-iommu","bus":"pcie.0","addr":"0x3"}' \ -device '{"driver":"pcie-root-port","port":16,"chassis":1,"id":"pci.1","bus":"pcie.0","multifunction":true,"addr":"0x2"}' \ -device '{"driver":"pcie-root-port","port":17,"chassis":2,"id":"pci.2","bus":"pcie.0","addr":"0x2.0x1"}' \ -device '{"driver":"pcie-root-port","port":18,"chassis":3,"id":"pci.3","bus":"pcie.0","addr":"0x2.0x2"}' \ -device '{"driver":"pcie-root-port","port":19,"chassis":4,"id":"pci.4","bus":"pcie.0","addr":"0x2.0x3"}' \ -device '{"driver":"ich9-usb-ehci1","id":"usb","bus":"pcie.0","addr":"0x1d.0x7"}' \ -device '{"driver":"ich9-usb-uhci1","masterbus":"usb.0","firstport":0,"bus":"pcie.0","multifunction":true,"addr":"0x1d"}' \ -device '{"driver":"ich9-usb-uhci2","masterbus":"usb.0","firstport":2,"bus":"pcie.0","addr":"0x1d.0x1"}' \ -device '{"driver":"ich9-usb-uhci3","masterbus":"usb.0","firstport":4,"bus":"pcie.0","addr":"0x1d.0x2"}' \ -device '{"driver":"virtio-scsi-pci","iommu_platform":true,"ats":true,"id":"scsi0","bus":"pci.2","addr":"0x0"}' \ -blockdev '{"driver":"file","filename":"/home/rhel8.qcow2","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-1-format","read-only":false,"driver":"qcow2","file":"libvirt-1-storage","backing":null}' \ -device '{"driver":"scsi-hd","bus":"scsi0.0","channel":0,"scsi-id":0,"lun":0,"device_id":"drive-scsi0-0-0-0","drive":"libvirt-1-format","id":"scsi0-0-0-0","bootindex":1}' \ -netdev tap,fd=24,vhost=on,vhostfd=28,id=hostnet0 \ -device '{"driver":"virtio-net-pci","iommu_platform":true,"ats":true,"netdev":"hostnet0","id":"net0","mac":"52:54:00:e8:32:0e","bus":"pci.1","addr":"0x0"}' \ -chardev pty,id=charserial0 \ -device '{"driver":"isa-serial","chardev":"charserial0","id":"serial0","index":0}' \ -device '{"driver":"usb-tablet","id":"input0","bus":"usb.0","port":"1"}' \ -audiodev '{"id":"audio1","driver":"none"}' \ -vnc 0.0.0.0:0,audiodev=audio1 \ -device '{"driver":"bochs-display","id":"video0","vgamem":16777216,"bus":"pcie.0","addr":"0x1"}' \ -device '{"driver":"virtio-balloon-pci","id":"balloon0","bus":"pci.3","addr":"0x0"}' \ -global virtio-net-pci.aer=on \ -global virtio-scsi-pci.aer=on \ -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \ -msg timestamp=on check dmesg: [root@bootp-73-199-57 ~]# dmesg | grep iommu [ 0.388509] iommu: Default domain type: Translated [ 0.389109] iommu: DMA domain TLB invalidation policy: lazy mode [ 0.583798] virtio_iommu virtio0: input address: 64 bits [ 0.584329] virtio_iommu virtio0: page mask: 0xfffffffffffff000 [ 0.588694] ehci-pci 0000:00:1d.7: Adding to iommu group 0 [ 0.589248] iommu: Failed to allocate default IOMMU domain of type 11 for group (null) - Falling back to IOMMU_DOMAIN_DMA [ 0.606243] uhci_hcd 0000:00:1d.0: Adding to iommu group 0 [ 0.614426] uhci_hcd 0000:00:1d.1: Adding to iommu group 0 [ 0.622630] uhci_hcd 0000:00:1d.2: Adding to iommu group 0 [ 1.032554] pcieport 0000:00:02.0: Adding to iommu group 1 [ 1.033088] iommu: Failed to allocate default IOMMU domain of type 11 for group (null) - Falling back to IOMMU_DOMAIN_DMA [ 1.037331] pcieport 0000:00:02.1: Adding to iommu group 1 [ 1.040324] pcieport 0000:00:02.2: Adding to iommu group 1 [ 1.043271] pcieport 0000:00:02.3: Adding to iommu group 1 [ 1.046349] virtio-pci 0000:01:00.0: Adding to iommu group 1 [ 1.047832] virtio-pci 0000:02:00.0: Adding to iommu group 1 [ 1.049222] virtio-pci 0000:03:00.0: Adding to iommu group 1 [ 1.530908] ahci 0000:00:1f.2: Adding to iommu group 2 [ 1.531691] iommu: Failed to allocate default IOMMU domain of type 11 for group (null) - Falling back to IOMMU_DOMAIN_DMA [ 3.296466] lpc_ich 0000:00:1f.0: Adding to iommu group 2 [ 3.298084] i801_smbus 0000:00:1f.3: Adding to iommu group 2 [ 3.393899] bochs-drm 0000:00:01.0: Adding to iommu group 3 [ 3.394717] iommu: Failed to allocate default IOMMU domain of type 11 for group (null) - Falling back to IOMMU_DOMAIN_DMA check iommu group: [root@bootp-73-199-57 ~]# find /sys/kernel/iommu_groups/ -type l /sys/kernel/iommu_groups/3/devices/0000:00:01.0 /sys/kernel/iommu_groups/1/devices/0000:03:00.0 /sys/kernel/iommu_groups/1/devices/0000:02:00.0 /sys/kernel/iommu_groups/1/devices/0000:00:02.3 /sys/kernel/iommu_groups/1/devices/0000:01:00.0 /sys/kernel/iommu_groups/1/devices/0000:00:02.1 /sys/kernel/iommu_groups/1/devices/0000:00:02.2 /sys/kernel/iommu_groups/1/devices/0000:00:02.0 /sys/kernel/iommu_groups/2/devices/0000:00:1f.2 /sys/kernel/iommu_groups/2/devices/0000:00:1f.0 /sys/kernel/iommu_groups/2/devices/0000:00:1f.3 /sys/kernel/iommu_groups/0/devices/0000:00:1d.1 /sys/kernel/iommu_groups/0/devices/0000:00:1d.2 /sys/kernel/iommu_groups/0/devices/0000:00:1d.0 /sys/kernel/iommu_groups/0/devices/0000:00:1d.7 2)with AMD host qemu command line: /usr/libexec/qemu-kvm \ -name guest=virtio-iommu-m,debug-threads=on \ -S \ -object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/var/lib/libvirt/qemu/domain-4-virtio-iommu-m/master-key.aes"}' \ -blockdev '{"driver":"file","filename":"/usr/share/edk2/ovmf/OVMF_CODE.secboot.fd","node-name":"libvirt-pflash0-storage","auto-read-only":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-pflash0-format","read-only":true,"driver":"raw","file":"libvirt-pflash0-storage"}' \ -blockdev '{"driver":"file","filename":"/var/lib/libvirt/qemu/nvram/virtio-iommu-m_VARS.fd","node-name":"libvirt-pflash1-storage","auto-read-only":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-pflash1-format","read-only":false,"driver":"raw","file":"libvirt-pflash1-storage"}' \ -machine pc-q35-rhel9.0.0,usb=off,smm=on,dump-guest-core=off,pflash0=libvirt-pflash0-format,pflash1=libvirt-pflash1-format,memory-backend=pc.ram \ -accel kvm \ -cpu host,migratable=on \ -global driver=cfi.pflash01,property=secure,value=on \ -m 8192 \ -object '{"qom-type":"memory-backend-ram","id":"pc.ram","size":8589934592}' \ -overcommit mem-lock=off \ -smp 4,sockets=1,dies=1,cores=4,threads=1 \ -uuid cf73da8b-c0f2-41c4-af15-7174371eda91 \ -no-user-config \ -nodefaults \ -chardev socket,id=charmonitor,fd=30,server=on,wait=off \ -mon chardev=charmonitor,id=monitor,mode=control \ -rtc base=utc,driftfix=slew \ -global kvm-pit.lost_tick_policy=delay \ -no-hpet \ -no-shutdown \ -global ICH9-LPC.disable_s3=1 \ -global ICH9-LPC.disable_s4=1 \ -boot strict=on \ -device '{"driver":"virtio-iommu","bus":"pcie.0","addr":"0x3"}' \ -device '{"driver":"pcie-root-port","port":16,"chassis":1,"id":"pci.1","bus":"pcie.0","multifunction":true,"addr":"0x2"}' \ -device '{"driver":"pcie-root-port","port":17,"chassis":2,"id":"pci.2","bus":"pcie.0","addr":"0x2.0x1"}' \ -device '{"driver":"pcie-root-port","port":18,"chassis":3,"id":"pci.3","bus":"pcie.0","addr":"0x2.0x2"}' \ -device '{"driver":"pcie-root-port","port":19,"chassis":4,"id":"pci.4","bus":"pcie.0","addr":"0x2.0x3"}' \ -device '{"driver":"ich9-usb-ehci1","id":"usb","bus":"pcie.0","addr":"0x1d.0x7"}' \ -device '{"driver":"ich9-usb-uhci1","masterbus":"usb.0","firstport":0,"bus":"pcie.0","multifunction":true,"addr":"0x1d"}' \ -device '{"driver":"ich9-usb-uhci2","masterbus":"usb.0","firstport":2,"bus":"pcie.0","addr":"0x1d.0x1"}' \ -device '{"driver":"ich9-usb-uhci3","masterbus":"usb.0","firstport":4,"bus":"pcie.0","addr":"0x1d.0x2"}' \ -device '{"driver":"virtio-scsi-pci","iommu_platform":true,"ats":true,"id":"scsi0","bus":"pci.2","addr":"0x0"}' \ -blockdev '{"driver":"file","filename":"/home/RHEL-9.1.0-20220629.0-x86_64-ovmf.qcow2","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-1-format","read-only":false,"driver":"qcow2","file":"libvirt-1-storage","backing":null}' \ -device '{"driver":"scsi-hd","bus":"scsi0.0","channel":0,"scsi-id":0,"lun":0,"device_id":"drive-scsi0-0-0-0","drive":"libvirt-1-format","id":"scsi0-0-0-0","bootindex":1}' \ -netdev tap,fd=37,vhost=on,vhostfd=38,id=hostnet0 \ -device '{"driver":"virtio-net-pci","iommu_platform":true,"ats":true,"netdev":"hostnet0","id":"net0","mac":"52:54:00:25:9e:ab","bus":"pci.1","addr":"0x0"}' \ -chardev pty,id=charserial0 \ -device '{"driver":"isa-serial","chardev":"charserial0","id":"serial0","index":0}' \ -device '{"driver":"usb-tablet","id":"input0","bus":"usb.0","port":"1"}' \ -audiodev '{"id":"audio1","driver":"none"}' \ -vnc 0.0.0.0:1,audiodev=audio1 \ -device '{"driver":"bochs-display","id":"video0","vgamem":16777216,"bus":"pcie.0","addr":"0x1"}' \ -device '{"driver":"virtio-balloon-pci","id":"balloon0","bus":"pci.3","addr":"0x0"}' \ -global virtio-net-pci.aer=on \ -global virtio-scsi-pci.aer=on \ -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \ -msg timestamp=on check dmesg: [ 0.532669] iommu: Default domain type: Translated [ 0.533149] iommu: DMA domain TLB invalidation policy: lazy mode [ 0.777849] virtio_iommu virtio0: input address: 64 bits [ 0.778735] virtio_iommu virtio0: page mask: 0xfffffffffffff000 [ 0.785983] ehci-pci 0000:00:1d.7: Adding to iommu group 0 [ 0.786922] iommu: Failed to allocate default IOMMU domain of type 11 for group (null) - Falling back to IOMMU_DOMAIN_DMA [ 0.818243] uhci_hcd 0000:00:1d.0: Adding to iommu group 0 [ 0.838821] uhci_hcd 0000:00:1d.1: Adding to iommu group 0 [ 0.858907] uhci_hcd 0000:00:1d.2: Adding to iommu group 0 [ 1.394244] pcieport 0000:00:02.0: Adding to iommu group 1 [ 1.395927] iommu: Failed to allocate default IOMMU domain of type 11 for group (null) - Falling back to IOMMU_DOMAIN_DMA [ 1.405418] pcieport 0000:00:02.1: Adding to iommu group 1 [ 1.411870] pcieport 0000:00:02.2: Adding to iommu group 1 [ 1.417895] pcieport 0000:00:02.3: Adding to iommu group 1 [ 1.423859] virtio-pci 0000:01:00.0: Adding to iommu group 1 [ 1.426363] virtio-pci 0000:02:00.0: Adding to iommu group 1 [ 1.428792] virtio-pci 0000:03:00.0: Adding to iommu group 1 [ 2.034911] ahci 0000:00:1f.2: Adding to iommu group 2 [ 2.037388] iommu: Failed to allocate default IOMMU domain of type 11 for group (null) - Falling back to IOMMU_DOMAIN_DMA [ 4.315824] lpc_ich 0000:00:1f.0: Adding to iommu group 2 [ 4.349424] i801_smbus 0000:00:1f.3: Adding to iommu group 2 [ 4.412281] bochs-drm 0000:00:01.0: Adding to iommu group 3 [ 4.414419] iommu: Failed to allocate default IOMMU domain of type 11 for group (null) - Falling back to IOMMU_DOMAIN_DMA check iommu group: /sys/kernel/iommu_groups/3/devices/0000:00:01.0 /sys/kernel/iommu_groups/1/devices/0000:03:00.0 /sys/kernel/iommu_groups/1/devices/0000:02:00.0 /sys/kernel/iommu_groups/1/devices/0000:00:02.3 /sys/kernel/iommu_groups/1/devices/0000:01:00.0 /sys/kernel/iommu_groups/1/devices/0000:00:02.1 /sys/kernel/iommu_groups/1/devices/0000:00:02.2 /sys/kernel/iommu_groups/1/devices/0000:00:02.0 /sys/kernel/iommu_groups/2/devices/0000:00:1f.2 /sys/kernel/iommu_groups/2/devices/0000:00:1f.0 /sys/kernel/iommu_groups/2/devices/0000:00:1f.3 /sys/kernel/iommu_groups/0/devices/0000:00:1d.1 /sys/kernel/iommu_groups/0/devices/0000:00:1d.2 /sys/kernel/iommu_groups/0/devices/0000:00:1d.0 /sys/kernel/iommu_groups/0/devices/0000:00:1d.7 Based on the Comment 6 test results, set this bug as verified. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: qemu-kvm security, bug fix, and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:7967 |