Bug 1363648
| Summary: | Add support to attach dmi-to-pci-bridge (i82801b11-bridge) into pxb-pcie | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Yang Yang <yanyang> |
| Component: | libvirt | Assignee: | Laine Stump <laine> |
| Status: | CLOSED ERRATA | QA Contact: | Virtualization Bugs <virt-bugs> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 7.3 | CC: | dyuan, laine, marcel, rbalakri |
| Target Milestone: | rc | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | libvirt-2.0.0-5.el7 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2016-11-03 18:51:31 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Marcel - is this actually supported by qemu? Laine,
I attempted to attach dmi-to-pci-bridge (i82801b11-bridge) directly into pxb-pcie by qemu. Guest starts up normally
e.g.
[root@rhel7_test ~]# /usr/libexec/qemu-kvm -machine q35,accel=kvm,usb=off \
> -smp 4,maxcpus=4,cores=2,threads=2,sockets=1 \
> -cpu SandyBridge,enforce \
> -m 2G \
> -boot strict=on \
> -device i82801b11-bridge,id=pci.1,bus=pcie.0,addr=0x1e \
> -device pci-bridge,chassis_nr=2,id=pci.2,bus=pci.1,addr=0x0 \
> -device pxb-pcie,id=pci.3,bus=pcie.0,bus_nr=10 \
> -device i82801b11-bridge,id=pci.4,bus=pci.3,addr=0x0 \
> -device pci-bridge,chassis_nr=5,id=pci.5,bus=pci.4,addr=0x0 \
> -drive file=/mnt/nfs2/RHEL-7.3-latest.qcow2,format=qcow2,if=none,id=drive-virtio-disk0 \
> -device virtio-blk-pci,scsi=off,bus=pci.5,addr=0x3,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 \
> -monitor stdio -spice port=5931,disable-ticketing -boot menu=on \
> -qmp tcp:0:6666,server,nowait
QEMU 2.6.0 monitor - type 'help' for more information
(qemu) info pci
Bus 10, device 0, function 0:
PCI bridge: PCI device 8086:244e
BUS 10.
secondary bus 11.
subordinate bus 12.
IO range [0xc000, 0xcfff]
memory range [0xfe200000, 0xfe5fffff]
prefetchable memory range [0xfd000000, 0xfd1fffff]
id "pci.4"
Bus 11, device 0, function 0:
PCI bridge: PCI device 1b36:0001
IRQ 10.
BUS 11.
secondary bus 12.
subordinate bus 12.
IO range [0xc000, 0xcfff]
memory range [0xfe200000, 0xfe3fffff]
prefetchable memory range [0xfd000000, 0xfd1fffff]
BAR0: 64 bit memory at 0xfe400000 [0xfe4000ff].
id "pci.5"
Bus 12, device 3, function 0:
SCSI controller: PCI device 1af4:1001
IRQ 11.
BAR0: I/O at 0xc000 [0xc03f].
BAR1: 32 bit memory at 0xfe200000 [0xfe200fff].
id "virtio-disk0"
Bus 0, device 0, function 0:
Host bridge: PCI device 8086:29c0
id ""
Bus 0, device 1, function 0:
VGA controller: PCI device 1234:1111
BAR0: 32 bit prefetchable memory at 0xfc000000 [0xfcffffff].
BAR2: 32 bit memory at 0xfea70000 [0xfea70fff].
BAR6: 32 bit memory at 0xffffffffffffffff [0x0000fffe].
id ""
Bus 0, device 2, function 0:
Ethernet controller: PCI device 8086:100e
IRQ 11.
BAR0: 32 bit memory at 0xfea40000 [0xfea5ffff].
BAR1: I/O at 0xe000 [0xe03f].
BAR6: 32 bit memory at 0xffffffffffffffff [0x0003fffe].
id ""
Bus 0, device 3, function 0:
Host bridge: PCI device 1b36:000b
id "pci.3"
Bus 0, device 30, function 0:
PCI bridge: PCI device 8086:244e
BUS 0.
secondary bus 1.
subordinate bus 2.
IO range [0xd000, 0xdfff]
memory range [0xfe600000, 0xfe9fffff]
prefetchable memory range [0xfd200000, 0xfd3fffff]
id "pci.1"
Bus 1, device 0, function 0:
PCI bridge: PCI device 1b36:0001
IRQ 10.
BUS 1.
secondary bus 2.
subordinate bus 2.
IO range [0xd000, 0xdfff]
memory range [0xfe600000, 0xfe7fffff]
prefetchable memory range [0xfd200000, 0xfd3fffff]
BAR0: 64 bit memory at 0xfe800000 [0xfe8000ff].
id "pci.2"
Bus 0, device 31, function 0:
ISA bridge: PCI device 8086:2918
id ""
Bus 0, device 31, function 2:
SATA controller: PCI device 8086:2922
IRQ 10.
BAR4: I/O at 0xe080 [0xe09f].
BAR5: 32 bit memory at 0xfea71000 [0xfea71fff].
id ""
Bus 0, device 31, function 3:
SMBus: PCI device 8086:2930
IRQ 10.
BAR4: I/O at 0x0700 [0x073f].
id ""
(In reply to Laine Stump from comment #2) > Marcel - is this actually supported by qemu? Yes, and if doesn't work is a bug. The best way to think of pxb/pbx-pcie is that it exposes a new pcie.0 like bus. The only difference is that you cannot have devices plugged directly into it (integrated device). pxb has a built-in bridge pxb-pcie - use pcie root ports. Thanks, Marcel Since I'm needing to make a separate "connection class" for dmi-to-pci-bridge (previously I allowed a dmi-to-pci-bridge to be plugged into any place a standard PCIe endpoint device could be connected), I have another question - is it supported to connect a dmi-to-pci-bridge into 1) a pcie-root-port? 2) a pcie-switch-downstream-port? Or should I limit it to only pcie-root and pcie-expander-bus? Marcel answered my questions in Comment 5 during an IRC conversation - the answer is that dmi-to-pci-bridge can be connected *only* to pcie-root or pcie-expander-bus, and nothing else. Fix pushed upstream:
commit b70e3d0123fcb6e22e99d1b272239e03a84262cb
Author: Laine Stump <laine>
Date: Fri Aug 5 15:01:08 2016 -0400
conf: restrict where dmi-to-pci-bridge can be connected
Verified on libvirt-2.0.0-5.el7.x86_64
Steps
1. attach dmi-to-pci-bridge to pxb-pcie
<controller type='pci' index='0' model='pcie-root'>
</controller>
<controller type='pci' index='1' model='dmi-to-pci-bridge'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/>
</controller>
<controller type='pci' index='2' model='pci-bridge'>
<address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
</controller>
<controller type='pci' index='3' model='pcie-expander-bus'>
<target busNr='100'>
<node>1</node>
</target>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
</controller>
<controller type='pci' index='4' model='dmi-to-pci-bridge'>
<address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
</controller>
<controller type='pci' index='5' model='pci-bridge'>
<address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
</controller>
<interface type='direct'>
<mac address='52:54:00:65:18:d8'/>
<source dev='eno1' mode='bridge'/>
<target dev='macvtap0'/>
<model type='virtio'/>
<alias name='net0'/>
<address type='pci' domain='0x0000' bus='0x05' slot='0x01' function='0x0'/>
</interface>
#virsh define vm1-q35-Sec.xml
#virsh start vm1-q35-Sec
#ps -ef|grep qemu
-device i82801b11-bridge,id=pci.1,bus=pcie.0,addr=0x1e
-device pci-bridge,chassis_nr=2,id=pci.2,bus=pci.1,addr=0x0
-device pxb-pcie,bus_nr=100,id=pci.3,numa_node=1,bus=pcie.0,addr=0x2
-device i82801b11-bridge,id=pci.4,bus=pci.3,addr=0x0
-device pci-bridge,chassis_nr=5,id=pci.5,bus=pci.4,addr=0x0
-netdev tap,fd=26,id=hostnet0,vhost=on,vhostfd=28
-device virtionet-pci,netdev=hostnet0,id=net0,mac=52:54:00:65:18:d8,bus=pci.5,addr=0x1
checked in guest, nic is usable
2. attach dmi-to-pci-bridge to dmi-to-pci-bridge
<controller type='pci' index='0' model='pcie-root'>
</controller>
<controller type='pci' index='1' model='dmi-to-pci-bridge'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/>
</controller>
<controller type='pci' index='2' model='pci-bridge'>
<address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
</controller>
<controller type='pci' index='3' model='dmi-to-pci-bridge'>
<address type='pci' domain='0x0000' bus='0x01' slot='0x01' function='0x0'/>
</controller>
# virsh define vm1-q35-sec.xml
error: Failed to define domain from vm1-q35-sec.xml
error: XML error: The device at PCI address 0000:01:01.0 cannot be plugged into the PCI controller with index='1'. It requires a controller that accepts a dmi-to-pci-bridge.
3.attach dmi-to-pci-bridge to pci-bridge
<controller type='pci' index='0' model='pcie-root'>
</controller>
<controller type='pci' index='1' model='dmi-to-pci-bridge'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/>
</controller>
<controller type='pci' index='2' model='pci-bridge'>
<address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
</controller>
<controller type='pci' index='3' model='dmi-to-pci-bridge'>
<address type='pci' domain='0x0000' bus='0x02' slot='0x1f' function='0x0'/>
</controller>
# virsh define vm1-q35-sec.xml
error: Failed to define domain from vm1-q35-sec.xml
error: XML error: The device at PCI address 0000:02:1f.0 cannot be plugged into the PCI controller with index='2'. It requires a controller that accepts a dmi-to-pci-bridge.
4. attach dmi-to-pci-bridge to pcie-root-port
<controller type='pci' index='0' model='pcie-root'>
</controller>
<controller type='pci' index='1' model='dmi-to-pci-bridge'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/>
</controller>
<controller type='pci' index='2' model='pci-bridge'>
<address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
</controller>
<controller type='pci' index='3' model='pcie-root-port'/>
<controller type='pci' index='4' model='dmi-to-pci-bridge'>
<address type='pci' domain='0x0000' bus='0x03' slot='0x0' function='0x0'/>
</controller>
# virsh define vm1-q35-sec.xml
error: Failed to define domain from vm1-q35-sec.xml
error: XML error: The device at PCI address 0000:03:00.0 cannot be plugged into the PCI controller with index='3'. It requires a controller that accepts a dmi-to-pci-bridge.
5. attach dmi-to-pci-bridge to pcie-switch-upstream-port
<controller type='pci' index='0' model='pcie-root'>
</controller>
<controller type='pci' index='1' model='dmi-to-pci-bridge'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/>
</controller>
<controller type='pci' index='2' model='pci-bridge'>
<address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
</controller>
<controller type='pci' index='3' model='pcie-root-port'/>
<controller type='pci' index='4' model='pcie-switch-upstream-port'/>
<controller type='pci' index='5' model='dmi-to-pci-bridge'>
<address type='pci' domain='0x0000' bus='0x04' slot='0x0' function='0x0'/>
</controller>
# virsh define vm1-q35-sec.xml
error: Failed to define domain from vm1-q35-sec.xml
error: XML error: The device at PCI address 0000:04:00.0 cannot be plugged into the PCI controller with index='4'. It requires a controller that accepts a dmi-to-pci-bridge.
6. attach dmi-to-pci-bridge to pcie-switch-downstream-port
<controller type='pci' index='0' model='pcie-root'>
</controller>
<controller type='pci' index='1' model='dmi-to-pci-bridge'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/>
</controller>
<controller type='pci' index='2' model='pci-bridge'>
<address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
</controller>
<controller type='pci' index='3' model='pcie-root-port'/>
<controller type='pci' index='4' model='pcie-switch-upstream-port'/>
<controller type='pci' index='5' model='pcie-switch-downstream-port'/>
<controller type='pci' index='6' model='dmi-to-pci-bridge'>
<address type='pci' domain='0x0000' bus='0x05' slot='0x0' function='0x0'/>
</controller>
# virsh define vm1-q35-sec.xml
error: Failed to define domain from vm1-q35-sec.xml
error: XML error: The device at PCI address 0000:05:00.0 cannot be plugged into the PCI controller with index='5'. It requires a controller that accepts a dmi-to-pci-bridge.
The results are expected.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHSA-2016-2577.html |
Description of problem: Libvirt does not support to attach dmi-to-pci-bridge (i82801b11-bridge) directly into pxb-pcie. From the POV of qemu, if it is supported by pcie.0, pxb-pcie should support it as well. So libvirt should support to attach dmi-to-pci-bridge (i82801b11-bridge) directly into pxb-pcie. Version-Release number of selected component (if applicable): libvirt-2.0.0-4.el7.x86_64 qemu-kvm-rhev-2.6.0-18.el7.x86_64 How reproducible: 100% Steps to Reproduce: 1. create vm with pxb-pcie -- dmi-to-pci-bridge <controller type='pci' index='0' model='pcie-root'/> <controller type='pci' index='1' model='dmi-to-pci-bridge'/> <controller type='pci' index='2' model='pci-bridge'/> <controller type='pci' index='3' model='pcie-expander-bus'> <model name='pxb-pcie'/> <target busNr='10'/> </controller> <controller type='pci' index='4' model='dmi-to-pci-bridge'> <address type='pci' domain='0x0000' bus='0x3' slot='0x0' function='0x0'/> </controller> # virsh create vm1-q35-pxb-tmp.xml error: Failed to create domain from vm1-q35-pxb-tmp.xml error: XML error: PCI bus is not compatible with the device at 0000:03:00.0. Device requires a PCI Express slot, which is not provided by bus 0000:03 Actual results: Libvirt does not support attach dmi-to-pci-bridge into pxb-pcie However, qemu supports that Expected results: Add support to attach dmi-to-pci-bridge into pxb-pcie Additional info: