Bug 2089545
| Summary: | Not supporting ATS when booting a win2022 guest with '-device virtio-net-pci,aer=on,ats=on' | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 9 | Reporter: | Yiqian Wei <yiwei> |
| Component: | qemu-kvm | Assignee: | Kostiantyn Kostiuk <kkostiuk> |
| qemu-kvm sub component: | Testing | QA Contact: | Yiqian Wei <yiwei> |
| Status: | CLOSED DUPLICATE | Docs Contact: | |
| Severity: | unspecified | ||
| Priority: | unspecified | CC: | ailan, ani, coli, imammedo, jinzhao, juzhang, kkostiuk, mst, virt-maint, yvugenfi |
| Version: | 9.1 | Keywords: | Regression, Triaged |
| Target Milestone: | rc | ||
| Target Release: | --- | ||
| Hardware: | x86_64 | ||
| OS: | Windows | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2022-07-13 03:41:06 UTC | Type: | --- |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Confirm bug. This is regression. Works fine in QEMU v6.1.0. Commit that creates this issue: https://github.com/qemu/qemu/commit/211afe5c69 Possible workaround: add `--global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=off` to the QEMU command line. In this case, the ATS support property will be correct. tested with QEMU v6.2.0 and v7.0.50 Hi Yiqian Wei, From your command line -device pcie-root-port,id=pcie-root-port-3,port=0x3,addr=0x1.0x3,bus=pcie.0,chassis=4 \ -device virtio-net-pci,mac=9a:7f:22:72:3a:80,id=id962Oxs,netdev=idZYW5my,bus=pcie-root-port-3,addr=0x0,aer=on,ats=on \ Please set x-native-hotplug=false for pcie-root-port-3 and let us know what results you get: $ ./qemu-system-x86_64 -device pcie-root-port,? | grep hotplug hotplug=<bool> - (default: true) x-native-hotplug=<bool> - (default: true) (In reply to Kostiantyn Kostiuk from comment #4) > Hi Yiqian Wei, > > From your command line > -device > pcie-root-port,id=pcie-root-port-3,port=0x3,addr=0x1.0x3,bus=pcie.0, > chassis=4 \ > -device > virtio-net-pci,mac=9a:7f:22:72:3a:80,id=id962Oxs,netdev=idZYW5my,bus=pcie- > root-port-3,addr=0x0,aer=on,ats=on \ > > Please set x-native-hotplug=false for pcie-root-port-3 and let us know > what results you get: > > $ ./qemu-system-x86_64 -device pcie-root-port,? | grep hotplug > hotplug=<bool> - (default: true) > x-native-hotplug=<bool> - (default: true) Can reproduce this bug with set x-native-hotplug=false to pcie-root-port-3 -device pcie-root-port,id=pcie-root-port-3,port=0x3,addr=0x1.0x3,bus=pcie.0,chassis=4,x-native-hotplug=false \ -device virtio-net-pci,mac=9a:7f:22:72:3a:80,id=id962Oxs,netdev=idZYW5my,bus=pcie-root-port-3,addr=0x0,aer=on,ats=on \ -netdev tap,id=idZYW5my,vhost=on \ *** This bug has been marked as a duplicate of bug 2073872 *** Hi Kostiantyn, Confirmed that this issue is same as BZ2073872, so close it as a dup, please feel free to correct me if anything is wrong. Thanks. Can you please try the patch I posted here: https://lists.nongnu.org/archive/html/qemu-devel/2022-08/msg03072.html cc: @mst @kkostiuk I tried your patches with Windows Server 2022. The ATS property is still absent in Device Manager. Please try this patch also : https://lists.nongnu.org/archive/html/qemu-devel/2022-09/msg00564.html . Suggested by mst. Issue summary: ================ Windows does not detect ATS property for Ethernet PCI controller when firmware masks PCIE native hotplug capability in _OSC control Versions of windows affected: ============================= Server versions: Windows server 2016, windows server 2019, windows server 2022 Desktop versions: Windows 10, Windows 11. From our testing, the versions of Windows earlier than those mentioned above do not report/check for ATS availability even when the firmware does not mask PCIE native hotplug in _OSC. It seems reporting ATS status is only available in the above newer versions of the OS. How to reproduce the issue? ============================ We have used the QEMU hypervisor to test for this issue. Please refer to the following RedHat BZ for the QEMU commandline used: https://bugzilla.redhat.com/show_bug.cgi?id=2089545 Additionally, one can use the following scripts: https://github.com/ani-sinha/misc/blob/master/test-ats.sh https://github.com/ani-sinha/misc/blob/master/test-ats-win11.sh (Please adjust the locations of the images/binaries as per your host). Note that the ethernet controller is attached to the pcie root port which in turn is attached to the pci root complex. The ATS property is for the pcie root port as mentioned in Intel VTD spec (1). "--global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=on/off" will mask/unmask the PCIE native hotplug reporting in _OSC from the firmware in QEMU and thus the issue can be observed easily. Please see (2). "on" will mask the bit, "off" will unmask. "-snapshot" command line has been used with the disk image to make sure there are no caching effect within the guest OS. Further, we have tested by removing and re-adding the ethernet controller from within the guest OS. The results are the same. References: (1) Intel Virtualization technology for Directed I/O, section 8.5 - Root Port ATS capability reporting structure. (2) ACPI spec 5.1, section 6.2.11.3 - OSC Implementation Example for PCI Host Bridge Devices |
Description of problem: Not supporting ATS when booting a win2022 guest with '-device virtio-net-pci,aer=on,ats=on' Version-Release number of selected component (if applicable): host version: kernel-5.14.0-93.el9.x86_64 qemu-kvm-7.0.0-3.el9.x86_64 edk2-ovmf-20220221gitb24306f15d-1.el9.noarch guest: win2022 How reproducible: 3/3 Steps to Reproduce: 1.boot guest with add "aer=on,ats=on" to pcie virtio device (eg: virtio-net-pci) 2.In win2022 guest, check AER Value: Device Manager -> Red Hat VirtIO Ethernet Adapter -> Properties -> Details -> PCI AER capability present 3.In win2022 guest, check ATS Value: Device Manager -> Red Hat VirtIO Ethernet Adapter -> Properties -> Details -> PCI ATS support Actual results: After step 3, do not find "PCI ATS support" options, please see attachment. Expected results: After step 3, should be able to discover "PCI ATS support" options, and ATS Value is true. Additional info: 1) boot a guest with cmd: /usr/libexec/qemu-kvm \ -name 'avocado-vt-vm1' \ -sandbox on \ -blockdev node-name=file_ovmf_code,driver=file,filename=//usr/share/edk2/ovmf/OVMF_CODE.secboot.fd,auto-read-only=on,discard=unmap \ -blockdev node-name=drive_ovmf_code,driver=raw,read-only=on,file=file_ovmf_code \ -blockdev node-name=file_ovmf_vars,driver=file,filename=/home/OVMF_VARS.fd,auto-read-only=on,discard=unmap \ -blockdev node-name=drive_ovmf_vars,driver=raw,read-only=off,file=file_ovmf_vars \ -machine q35,memory-backend=mem-machine_mem,pflash0=drive_ovmf_code,pflash1=drive_ovmf_vars \ -device pcie-root-port,id=pcie-root-port-0,multifunction=on,bus=pcie.0,addr=0x1,chassis=1 \ -device pcie-pci-bridge,id=pcie-pci-bridge-0,addr=0x0,bus=pcie-root-port-0 \ -nodefaults \ -device VGA,bus=pcie.0,addr=0x2 \ -m 30720 \ -object memory-backend-ram,size=30720M,id=mem-machine_mem \ -smp 28,maxcpus=28,cores=14,threads=1,dies=1,sockets=2 \ -cpu 'Icelake-Server-noTSX',hv_stimer,hv_synic,hv_vpindex,hv_relaxed,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv_frequencies,hv_runtime,hv_tlbflush,hv_reenlightenment,hv_stimer_direct,hv_ipi,+kvm_pv_unhalt \ -device pcie-root-port,id=pcie-root-port-1,port=0x1,addr=0x1.0x1,bus=pcie.0,chassis=2 \ -device qemu-xhci,id=usb1,bus=pcie-root-port-1,addr=0x0 \ -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \ -device pcie-root-port,id=pcie-root-port-2,port=0x2,addr=0x1.0x2,bus=pcie.0,chassis=3 \ -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pcie-root-port-2,addr=0x0 \ -blockdev node-name=file_image1,driver=file,auto-read-only=on,discard=unmap,aio=threads,filename=/home/win2022-ovmf.qcow2,cache.direct=on,cache.no-flush=off \ -blockdev node-name=drive_image1,driver=qcow2,read-only=off,cache.direct=on,cache.no-flush=off,file=file_image1 \ -device scsi-hd,id=image1,drive=drive_image1,write-cache=on \ -device pcie-root-port,id=pcie-root-port-3,port=0x3,addr=0x1.0x3,bus=pcie.0,chassis=4 \ -device virtio-net-pci,mac=9a:7f:22:72:3a:80,id=id962Oxs,netdev=idZYW5my,bus=pcie-root-port-3,addr=0x0,aer=on,ats=on \ -netdev tap,id=idZYW5my,vhost=on \ -vnc :0 \ -rtc base=localtime,clock=host,driftfix=slew \ -boot menu=off,order=cdn,once=c,strict=off \ -enable-kvm \ -monitor stdio \ 2)Hit the same issue when boot a win2022 guest with "-device virtio-net-pci,ats=on" 3)Not hit this issue on linux(rhel 9.1.0) guest