Bug 2073872
Summary: | Devices lost PCI ATS support capability in windows vm | ||
---|---|---|---|
Product: | Red Hat Enterprise Linux 9 | Reporter: | jinl |
Component: | qemu-kvm | Assignee: | Kostiantyn Kostiuk <kkostiuk> |
qemu-kvm sub component: | PCI | QA Contact: | jinl |
Status: | CLOSED WONTFIX | Docs Contact: | Jiri Herrmann <jherrman> |
Severity: | medium | ||
Priority: | high | CC: | anisinha, coli, germano, gveitmic, jasowang, jherrman, jinzhao, juzhang, peterx, virt-maint, yiwei, yvugenfi, zhguo |
Version: | 9.0 | Keywords: | Regression, Triaged |
Target Milestone: | rc | ||
Target Release: | --- | ||
Hardware: | x86_64 | ||
OS: | Windows | ||
Whiteboard: | |||
Fixed In Version: | Doc Type: | Known Issue | |
Doc Text: |
.PCIe ATS devices do not work on Windows VMs
When you configure a PCIe Address Translation Services (ATS) device in the XML configuration of virtual machine (VM) with a Windows guest operating system, the guest does not enable the ATS device after booting the VM. This is because Windows currently does not support ATS on `virtio` devices.
For more information, see the link:https://access.redhat.com/solutions/7025324[Red Hat KnowledgeBase].
|
Story Points: | --- |
Clone Of: | Environment: | ||
Last Closed: | 2023-07-18 13:58:37 UTC | Type: | --- |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: | |||
Bug Depends On: | |||
Bug Blocks: | 2001312 |
Description
jinl
2022-04-11 02:25:23 UTC
Additional info: Devices properties which also lost: PCI ACS capability register PCI ACS compatible up hierarchy PCI ACS support PACI ARI support PCI ECRC errors PCI atomics supported PCI correctable error mask PCI error reporting PCI firmware error handing PCI uncorrectable error mask PCI uncorrectable error severity What about the same qemu cmdline with Linux guest? Since it's specific issue with virtio-net/vhost, copying Jason. (In reply to jinl from comment #1) > Additional info: > Devices properties which also lost: > PCI ACS capability register > PCI ACS compatible up hierarchy > PCI ACS support > PACI ARI support > PCI ECRC errors > PCI atomics supported > PCI correctable error mask > PCI error reporting > PCI firmware error handing > PCI uncorrectable error mask > PCI uncorrectable error severity Is the device identified as a PCI-E device or not in the guest? Thanks (In reply to Peter Xu from comment #2) > What about the same qemu cmdline with Linux guest? Linux guest works well with the same qemu cmdline. (In reply to jason wang from comment #3) > (In reply to jinl from comment #1) > > Additional info: > > Devices properties which also lost: > > PCI ACS capability register > > PCI ACS compatible up hierarchy > > PCI ACS support > > PACI ARI support > > PCI ECRC errors > > PCI atomics supported > > PCI correctable error mask > > PCI error reporting > > PCI firmware error handing > > PCI uncorrectable error mask > > PCI uncorrectable error severity > > Is the device identified as a PCI-E device or not in the guest? > > Thanks virtio-net-pci device is under pcie-root-port, so is identified as a PCI-E device in the guest. hit the same issue with RHEL 9.1 host Related BZ#2089545 *** Bug 2089545 has been marked as a duplicate of this bug. *** MS support ticket: 2210030030001461 hit the same issue with win11 22H2 host env: qemu-kvm-7.1.0-1.el9.x86_64 kernel-5.14.0-171.el9.x86_64 guest cmd: /usr/libexec/qemu-kvm \ -name guest=win11,debug-threads=on \ -S \ -object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/var/lib/libvirt/qemu/domain-9-win11/master-key.aes"}' \ -blockdev '{"driver":"file","filename":"/usr/share/edk2/ovmf/OVMF_CODE.secboot.fd","node-name":"libvirt-pflash0-storage","auto-read-only":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-pflash0-format","read-only":true,"driver":"raw","file":"libvirt-pflash0-storage"}' \ -blockdev '{"driver":"file","filename":"/var/lib/libvirt/qemu/nvram/win11_VARS.fd","node-name":"libvirt-pflash1-storage","auto-read-only":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-pflash1-format","read-only":false,"driver":"raw","file":"libvirt-pflash1-storage"}' \ -machine pc-q35-rhel9.0.0,usb=off,smm=on,kernel_irqchip=split,dump-guest-core=off,memory-backend=pc.ram,pflash0=libvirt-pflash0-format,pflash1=libvirt-pflash1-format \ -accel kvm \ -cpu Icelake-Server,ss=on,vmx=on,pdcm=on,hypervisor=on,tsc-adjust=on,avx512ifma=on,sha-ni=on,rdpid=on,fsrm=on,md-clear=on,stibp=on,arch-capabilities=on,xsaves=on,ibpb=on,ibrs=on,amd-stibp=on,amd-ssbd=on,rdctl-no=on,ibrs-all=on,skip-l1dfl-vmentry=on,mds-no=on,pschange-mc-no=on,tsx-ctrl=on,hle=off,rtm=off,mpx=off,intel-pt=off,hv-time=on,hv-relaxed=on,hv-vapic=on,hv-spinlocks=0x1fff,hv-vpindex=on,hv-runtime=on,hv-synic=on,hv-stimer=on,hv-stimer-direct=on,hv-reset=on,hv-frequencies=on,hv-reenlightenment=on,hv-tlbflush=on,hv-ipi=on,hv-evmcs=on \ -global driver=cfi.pflash01,property=secure,value=on \ -m 8192 \ -object '{"qom-type":"memory-backend-ram","id":"pc.ram","size":8589934592}' \ -overcommit mem-lock=off \ -smp 4,sockets=1,dies=1,cores=4,threads=1 \ -uuid 28807dcd-5c36-4ac8-ba2c-4d4cedbe998a \ -no-user-config \ -nodefaults \ -chardev socket,id=charmonitor,fd=22,server=on,wait=off \ -mon chardev=charmonitor,id=monitor,mode=control \ -rtc base=localtime,driftfix=slew \ -global kvm-pit.lost_tick_policy=delay \ -no-hpet \ -no-shutdown \ -global ICH9-LPC.disable_s3=1 \ -global ICH9-LPC.disable_s4=1 \ -boot strict=on \ -device '{"driver":"intel-iommu","id":"iommu0","intremap":"on","caching-mode":true,"eim":"on","device-iotlb":true}' \ -device '{"driver":"pcie-root-port","port":16,"chassis":1,"id":"pci.1","bus":"pcie.0","multifunction":true,"addr":"0x2"}' \ -device '{"driver":"pcie-root-port","port":17,"chassis":2,"id":"pci.2","bus":"pcie.0","addr":"0x2.0x1"}' \ -device '{"driver":"pcie-root-port","port":18,"chassis":3,"id":"pci.3","bus":"pcie.0","addr":"0x2.0x2"}' \ -device '{"driver":"pcie-root-port","port":19,"chassis":4,"id":"pci.4","bus":"pcie.0","addr":"0x2.0x3"}' \ -device '{"driver":"pcie-root-port","port":20,"chassis":5,"id":"pci.5","bus":"pcie.0","addr":"0x2.0x4"}' \ -device '{"driver":"pcie-root-port","port":21,"chassis":6,"id":"pci.6","bus":"pcie.0","addr":"0x2.0x5"}' \ -device '{"driver":"pcie-root-port","port":22,"chassis":7,"id":"pci.7","bus":"pcie.0","addr":"0x2.0x6"}' \ -device '{"driver":"pcie-root-port","port":23,"chassis":8,"id":"pci.8","bus":"pcie.0","addr":"0x2.0x7"}' \ -device '{"driver":"pcie-root-port","port":24,"chassis":9,"id":"pci.9","bus":"pcie.0","multifunction":true,"addr":"0x3"}' \ -device '{"driver":"pcie-root-port","port":25,"chassis":10,"id":"pci.10","bus":"pcie.0","addr":"0x3.0x1"}' \ -device '{"driver":"pcie-root-port","port":26,"chassis":11,"id":"pci.11","bus":"pcie.0","addr":"0x3.0x2"}' \ -device '{"driver":"pcie-root-port","port":27,"chassis":12,"id":"pci.12","bus":"pcie.0","addr":"0x3.0x3"}' \ -device '{"driver":"pcie-root-port","port":28,"chassis":13,"id":"pci.13","bus":"pcie.0","addr":"0x3.0x4"}' \ -device '{"driver":"pcie-root-port","port":29,"chassis":14,"id":"pci.14","bus":"pcie.0","addr":"0x3.0x5"}' \ -device '{"driver":"qemu-xhci","p2":15,"p3":15,"id":"usb","bus":"pci.3","addr":"0x0"}' \ -device '{"driver":"virtio-scsi-pci","iommu_platform":true,"ats":true,"id":"scsi0","bus":"pci.2","addr":"0x0"}' \ -blockdev '{"driver":"file","filename":"/home/win10.qcow2","node-name":"libvirt-4-storage","cache":{"direct":true,"no-flush":false},"auto-read-only":true,"discard":"unmap"}' \ -blockdev '{"node-name":"libvirt-4-format","read-only":false,"discard":"unmap","cache":{"direct":true,"no-flush":false},"driver":"qcow2","file":"libvirt-4-storage","backing":null}' \ -device '{"driver":"scsi-hd","bus":"scsi0.0","channel":0,"scsi-id":0,"lun":0,"device_id":"drive-scsi0-0-0-0","drive":"libvirt-4-format","id":"scsi0-0-0-0","bootindex":1,"write-cache":"on"}' \ -device '{"driver":"ide-cd","bus":"ide.1","id":"sata0-0-1"}' \ -device '{"driver":"ide-cd","bus":"ide.2","id":"sata0-0-2"}' \ -device '{"driver":"ide-cd","bus":"ide.3","id":"sata0-0-3"}' \ -netdev tap,fd=24,vhost=on,vhostfd=26,id=hostnet0 \ -device '{"driver":"virtio-net-pci","iommu_platform":true,"ats":true,"netdev":"hostnet0","id":"net0","mac":"52:54:00:dc:d0:4c","bus":"pci.1","addr":"0x0"}' \ -add-fd set=0,fd=23,opaque=serial0-source \ -chardev file,id=charserial0,path=/dev/fdset/0,append=on \ -device '{"driver":"isa-serial","chardev":"charserial0","id":"serial0","index":0}' \ -chardev socket,id=chrtpm,path=/run/libvirt/qemu/swtpm/9-win11-swtpm.sock \ -tpmdev emulator,id=tpm-tpm0,chardev=chrtpm \ -device '{"driver":"tpm-crb","tpmdev":"tpm-tpm0","id":"tpm0"}' \ -device '{"driver":"usb-tablet","id":"input0","bus":"usb.0","port":"1"}' \ -audiodev '{"id":"audio1","driver":"none"}' \ -vnc 0.0.0.0:0,audiodev=audio1 \ -device '{"driver":"VGA","id":"video0","vgamem_mb":16,"bus":"pcie.0","addr":"0x1"}' \ -device '{"driver":"virtio-balloon-pci","id":"balloon0","bus":"pci.4","addr":"0x0"}' \ Issue summary: ================ Windows does not detect ATS property for Ethernet PCI controller when firmware masks PCIE native hotplug capability in _OSC control Versions of windows affected: ============================= Server versions: Windows server 2016, windows server 2019, windows server 2022 Desktop versions: Windows 10, Windows 11. From our testing, the versions of Windows earlier than those mentioned above do not report/check for ATS availability even when the firmware does not mask PCIE native hotplug in _OSC. It seems reporting ATS status is only available in the above newer versions of the OS. How to reproduce the issue? ============================ We have used the QEMU hypervisor to test for this issue. Please refer to the following RedHat BZ for the QEMU commandline used: https://bugzilla.redhat.com/show_bug.cgi?id=2089545 Additionally, one can use the following scripts: https://github.com/ani-sinha/misc/blob/master/test-ats.sh https://github.com/ani-sinha/misc/blob/master/test-ats-win11.sh (Please adjust the locations of the images/binaries as per your host). Note that the ethernet controller is attached to the pcie root port which in turn is attached to the pci root complex. The ATS property is for the pcie root port as mentioned in Intel VTD spec (1). "--global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=on/off" will mask/unmask the PCIE native hotplug reporting in _OSC from the firmware in QEMU and thus the issue can be observed easily. Please see (2). "on" will mask the bit, "off" will unmask. "-snapshot" command line has been used with the disk image to make sure there are no caching effect within the guest OS. Further, we have tested by removing and re-adding the ethernet controller from within the guest OS. The results are the same. References: (1) Intel Virtualization technology for Directed I/O, section 8.5 - Root Port ATS capability reporting structure. (2) ACPI spec 5.1, section 6.2.11.3 - OSC Implementation Example for PCI Host Bridge Devices |