Red Hat Bugzilla – Bug 1363648
Add support to attach dmi-to-pci-bridge (i82801b11-bridge) into pxb-pcie
Last modified: 2016-11-03 14:51:31 EDT
Description of problem: Libvirt does not support to attach dmi-to-pci-bridge (i82801b11-bridge) directly into pxb-pcie. From the POV of qemu, if it is supported by pcie.0, pxb-pcie should support it as well. So libvirt should support to attach dmi-to-pci-bridge (i82801b11-bridge) directly into pxb-pcie. Version-Release number of selected component (if applicable): libvirt-2.0.0-4.el7.x86_64 qemu-kvm-rhev-2.6.0-18.el7.x86_64 How reproducible: 100% Steps to Reproduce: 1. create vm with pxb-pcie -- dmi-to-pci-bridge <controller type='pci' index='0' model='pcie-root'/> <controller type='pci' index='1' model='dmi-to-pci-bridge'/> <controller type='pci' index='2' model='pci-bridge'/> <controller type='pci' index='3' model='pcie-expander-bus'> <model name='pxb-pcie'/> <target busNr='10'/> </controller> <controller type='pci' index='4' model='dmi-to-pci-bridge'> <address type='pci' domain='0x0000' bus='0x3' slot='0x0' function='0x0'/> </controller> # virsh create vm1-q35-pxb-tmp.xml error: Failed to create domain from vm1-q35-pxb-tmp.xml error: XML error: PCI bus is not compatible with the device at 0000:03:00.0. Device requires a PCI Express slot, which is not provided by bus 0000:03 Actual results: Libvirt does not support attach dmi-to-pci-bridge into pxb-pcie However, qemu supports that Expected results: Add support to attach dmi-to-pci-bridge into pxb-pcie Additional info:
Marcel - is this actually supported by qemu?
Laine, I attempted to attach dmi-to-pci-bridge (i82801b11-bridge) directly into pxb-pcie by qemu. Guest starts up normally e.g. [root@rhel7_test ~]# /usr/libexec/qemu-kvm -machine q35,accel=kvm,usb=off \ > -smp 4,maxcpus=4,cores=2,threads=2,sockets=1 \ > -cpu SandyBridge,enforce \ > -m 2G \ > -boot strict=on \ > -device i82801b11-bridge,id=pci.1,bus=pcie.0,addr=0x1e \ > -device pci-bridge,chassis_nr=2,id=pci.2,bus=pci.1,addr=0x0 \ > -device pxb-pcie,id=pci.3,bus=pcie.0,bus_nr=10 \ > -device i82801b11-bridge,id=pci.4,bus=pci.3,addr=0x0 \ > -device pci-bridge,chassis_nr=5,id=pci.5,bus=pci.4,addr=0x0 \ > -drive file=/mnt/nfs2/RHEL-7.3-latest.qcow2,format=qcow2,if=none,id=drive-virtio-disk0 \ > -device virtio-blk-pci,scsi=off,bus=pci.5,addr=0x3,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 \ > -monitor stdio -spice port=5931,disable-ticketing -boot menu=on \ > -qmp tcp:0:6666,server,nowait QEMU 2.6.0 monitor - type 'help' for more information (qemu) info pci Bus 10, device 0, function 0: PCI bridge: PCI device 8086:244e BUS 10. secondary bus 11. subordinate bus 12. IO range [0xc000, 0xcfff] memory range [0xfe200000, 0xfe5fffff] prefetchable memory range [0xfd000000, 0xfd1fffff] id "pci.4" Bus 11, device 0, function 0: PCI bridge: PCI device 1b36:0001 IRQ 10. BUS 11. secondary bus 12. subordinate bus 12. IO range [0xc000, 0xcfff] memory range [0xfe200000, 0xfe3fffff] prefetchable memory range [0xfd000000, 0xfd1fffff] BAR0: 64 bit memory at 0xfe400000 [0xfe4000ff]. id "pci.5" Bus 12, device 3, function 0: SCSI controller: PCI device 1af4:1001 IRQ 11. BAR0: I/O at 0xc000 [0xc03f]. BAR1: 32 bit memory at 0xfe200000 [0xfe200fff]. id "virtio-disk0" Bus 0, device 0, function 0: Host bridge: PCI device 8086:29c0 id "" Bus 0, device 1, function 0: VGA controller: PCI device 1234:1111 BAR0: 32 bit prefetchable memory at 0xfc000000 [0xfcffffff]. BAR2: 32 bit memory at 0xfea70000 [0xfea70fff]. BAR6: 32 bit memory at 0xffffffffffffffff [0x0000fffe]. id "" Bus 0, device 2, function 0: Ethernet controller: PCI device 8086:100e IRQ 11. BAR0: 32 bit memory at 0xfea40000 [0xfea5ffff]. BAR1: I/O at 0xe000 [0xe03f]. BAR6: 32 bit memory at 0xffffffffffffffff [0x0003fffe]. id "" Bus 0, device 3, function 0: Host bridge: PCI device 1b36:000b id "pci.3" Bus 0, device 30, function 0: PCI bridge: PCI device 8086:244e BUS 0. secondary bus 1. subordinate bus 2. IO range [0xd000, 0xdfff] memory range [0xfe600000, 0xfe9fffff] prefetchable memory range [0xfd200000, 0xfd3fffff] id "pci.1" Bus 1, device 0, function 0: PCI bridge: PCI device 1b36:0001 IRQ 10. BUS 1. secondary bus 2. subordinate bus 2. IO range [0xd000, 0xdfff] memory range [0xfe600000, 0xfe7fffff] prefetchable memory range [0xfd200000, 0xfd3fffff] BAR0: 64 bit memory at 0xfe800000 [0xfe8000ff]. id "pci.2" Bus 0, device 31, function 0: ISA bridge: PCI device 8086:2918 id "" Bus 0, device 31, function 2: SATA controller: PCI device 8086:2922 IRQ 10. BAR4: I/O at 0xe080 [0xe09f]. BAR5: 32 bit memory at 0xfea71000 [0xfea71fff]. id "" Bus 0, device 31, function 3: SMBus: PCI device 8086:2930 IRQ 10. BAR4: I/O at 0x0700 [0x073f]. id ""
(In reply to Laine Stump from comment #2) > Marcel - is this actually supported by qemu? Yes, and if doesn't work is a bug. The best way to think of pxb/pbx-pcie is that it exposes a new pcie.0 like bus. The only difference is that you cannot have devices plugged directly into it (integrated device). pxb has a built-in bridge pxb-pcie - use pcie root ports. Thanks, Marcel
Since I'm needing to make a separate "connection class" for dmi-to-pci-bridge (previously I allowed a dmi-to-pci-bridge to be plugged into any place a standard PCIe endpoint device could be connected), I have another question - is it supported to connect a dmi-to-pci-bridge into 1) a pcie-root-port? 2) a pcie-switch-downstream-port? Or should I limit it to only pcie-root and pcie-expander-bus?
Marcel answered my questions in Comment 5 during an IRC conversation - the answer is that dmi-to-pci-bridge can be connected *only* to pcie-root or pcie-expander-bus, and nothing else.
Fix pushed upstream: commit b70e3d0123fcb6e22e99d1b272239e03a84262cb Author: Laine Stump <laine@laine.org> Date: Fri Aug 5 15:01:08 2016 -0400 conf: restrict where dmi-to-pci-bridge can be connected
Verified on libvirt-2.0.0-5.el7.x86_64 Steps 1. attach dmi-to-pci-bridge to pxb-pcie <controller type='pci' index='0' model='pcie-root'> </controller> <controller type='pci' index='1' model='dmi-to-pci-bridge'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/> </controller> <controller type='pci' index='2' model='pci-bridge'> <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </controller> <controller type='pci' index='3' model='pcie-expander-bus'> <target busNr='100'> <node>1</node> </target> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </controller> <controller type='pci' index='4' model='dmi-to-pci-bridge'> <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/> </controller> <controller type='pci' index='5' model='pci-bridge'> <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/> </controller> <interface type='direct'> <mac address='52:54:00:65:18:d8'/> <source dev='eno1' mode='bridge'/> <target dev='macvtap0'/> <model type='virtio'/> <alias name='net0'/> <address type='pci' domain='0x0000' bus='0x05' slot='0x01' function='0x0'/> </interface> #virsh define vm1-q35-Sec.xml #virsh start vm1-q35-Sec #ps -ef|grep qemu -device i82801b11-bridge,id=pci.1,bus=pcie.0,addr=0x1e -device pci-bridge,chassis_nr=2,id=pci.2,bus=pci.1,addr=0x0 -device pxb-pcie,bus_nr=100,id=pci.3,numa_node=1,bus=pcie.0,addr=0x2 -device i82801b11-bridge,id=pci.4,bus=pci.3,addr=0x0 -device pci-bridge,chassis_nr=5,id=pci.5,bus=pci.4,addr=0x0 -netdev tap,fd=26,id=hostnet0,vhost=on,vhostfd=28 -device virtionet-pci,netdev=hostnet0,id=net0,mac=52:54:00:65:18:d8,bus=pci.5,addr=0x1 checked in guest, nic is usable 2. attach dmi-to-pci-bridge to dmi-to-pci-bridge <controller type='pci' index='0' model='pcie-root'> </controller> <controller type='pci' index='1' model='dmi-to-pci-bridge'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/> </controller> <controller type='pci' index='2' model='pci-bridge'> <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </controller> <controller type='pci' index='3' model='dmi-to-pci-bridge'> <address type='pci' domain='0x0000' bus='0x01' slot='0x01' function='0x0'/> </controller> # virsh define vm1-q35-sec.xml error: Failed to define domain from vm1-q35-sec.xml error: XML error: The device at PCI address 0000:01:01.0 cannot be plugged into the PCI controller with index='1'. It requires a controller that accepts a dmi-to-pci-bridge. 3.attach dmi-to-pci-bridge to pci-bridge <controller type='pci' index='0' model='pcie-root'> </controller> <controller type='pci' index='1' model='dmi-to-pci-bridge'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/> </controller> <controller type='pci' index='2' model='pci-bridge'> <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </controller> <controller type='pci' index='3' model='dmi-to-pci-bridge'> <address type='pci' domain='0x0000' bus='0x02' slot='0x1f' function='0x0'/> </controller> # virsh define vm1-q35-sec.xml error: Failed to define domain from vm1-q35-sec.xml error: XML error: The device at PCI address 0000:02:1f.0 cannot be plugged into the PCI controller with index='2'. It requires a controller that accepts a dmi-to-pci-bridge. 4. attach dmi-to-pci-bridge to pcie-root-port <controller type='pci' index='0' model='pcie-root'> </controller> <controller type='pci' index='1' model='dmi-to-pci-bridge'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/> </controller> <controller type='pci' index='2' model='pci-bridge'> <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </controller> <controller type='pci' index='3' model='pcie-root-port'/> <controller type='pci' index='4' model='dmi-to-pci-bridge'> <address type='pci' domain='0x0000' bus='0x03' slot='0x0' function='0x0'/> </controller> # virsh define vm1-q35-sec.xml error: Failed to define domain from vm1-q35-sec.xml error: XML error: The device at PCI address 0000:03:00.0 cannot be plugged into the PCI controller with index='3'. It requires a controller that accepts a dmi-to-pci-bridge. 5. attach dmi-to-pci-bridge to pcie-switch-upstream-port <controller type='pci' index='0' model='pcie-root'> </controller> <controller type='pci' index='1' model='dmi-to-pci-bridge'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/> </controller> <controller type='pci' index='2' model='pci-bridge'> <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </controller> <controller type='pci' index='3' model='pcie-root-port'/> <controller type='pci' index='4' model='pcie-switch-upstream-port'/> <controller type='pci' index='5' model='dmi-to-pci-bridge'> <address type='pci' domain='0x0000' bus='0x04' slot='0x0' function='0x0'/> </controller> # virsh define vm1-q35-sec.xml error: Failed to define domain from vm1-q35-sec.xml error: XML error: The device at PCI address 0000:04:00.0 cannot be plugged into the PCI controller with index='4'. It requires a controller that accepts a dmi-to-pci-bridge. 6. attach dmi-to-pci-bridge to pcie-switch-downstream-port <controller type='pci' index='0' model='pcie-root'> </controller> <controller type='pci' index='1' model='dmi-to-pci-bridge'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/> </controller> <controller type='pci' index='2' model='pci-bridge'> <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </controller> <controller type='pci' index='3' model='pcie-root-port'/> <controller type='pci' index='4' model='pcie-switch-upstream-port'/> <controller type='pci' index='5' model='pcie-switch-downstream-port'/> <controller type='pci' index='6' model='dmi-to-pci-bridge'> <address type='pci' domain='0x0000' bus='0x05' slot='0x0' function='0x0'/> </controller> # virsh define vm1-q35-sec.xml error: Failed to define domain from vm1-q35-sec.xml error: XML error: The device at PCI address 0000:05:00.0 cannot be plugged into the PCI controller with index='5'. It requires a controller that accepts a dmi-to-pci-bridge. The results are expected.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHSA-2016-2577.html