1382079 – libvirt mistakenly allows only slot 0 to be used on pcie-expander-bus

Bug 1382079 - libvirt mistakenly allows only slot 0 to be used on pcie-expander-bus

Summary: libvirt mistakenly allows only slot 0 to be used on pcie-expander-bus

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat Enterprise Linux 7
Classification:	Red Hat
Component:	libvirt
Sub Component:
Version:	7.3
Hardware:	Unspecified
OS:	Unspecified
Priority:	high
Severity:	unspecified
Target Milestone:	rc
Target Release:	---
Assignee:	Laine Stump
QA Contact:	Han Han
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:	1389369
TreeView+	depends on / blocked

Reported:	2016-10-05 16:49 UTC by Laine Stump
Modified:	2017-08-01 23:57 UTC (History)
CC List:	9 users (show)
Fixed In Version:	libvirt-2.5.0-1.el7
Doc Type:	Bug Fix
Doc Text:	Previously, the libvirt library allowed only use of slot 0 on the pcie-expander-bus controllers, even though slots 0 - 31 could be used. The restriction has been removed and slots 0 - 31 now can be used.
Clone Of:
Clones:	1389369 (view as bug list)
Environment:
Last Closed:	2017-08-01 17:16:43 UTC
Target Upstream Version:

Attachments	(Terms of Use)
Add an attachment (proposed patch, testcase, etc.)

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Red Hat Product Errata	RHEA-2017:1846	0	normal	SHIPPED_LIVE	libvirt bug fix and enhancement update	2017-08-01 18:02:50 UTC

Description Laine Stump 2016-10-05 16:49:13 UTC

The pcie-expander-bus (known as pxb-pcie in qemu) has 32 slots available to connect pcie-root-ports or dmi-to-pci-bridges, but when I added support for it, I mistakenly believed that it had only a single slot.

There is a patch upstream to fix this behavior:

commit 22afd44171a2df74fecb7058d146e83adddaae45
Author: Laine Stump <laine>
Date:   Wed Oct 5 11:26:07 2016 -0400

    qemu: allow 32 slots on pcie-expander-bus, not just 1

Comment 2 Yang Yang 2016-10-09 07:18:00 UTC

QE can reproduce it. Steps are as follows

#rpm -q libvirt
libvirt-2.0.0-10.el7.x86_64

Attach 1 pcie-root-port and 1 dmi-to-pci-bridge to 1 pcie-pxb controller. Libvirt reports error due to the slot must be 0

<controller type='pci' index='3' model='pcie-expander-bus'>
      <model name='pxb-pcie'/>
      <target busNr='100'>
        <node>1</node>
      </target>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </controller>
    <controller type='pci' index='4' model='pcie-root-port'>
      <model name='ioh3420'/>
      <target chassis='4' port='0x0'/>
      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
    </controller>
    <controller type='pci' index='5' model='pcie-switch-upstream-port'>
      <model name='x3130-upstream'/>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
    </controller>
    <controller type='pci' index='6' model='pcie-switch-downstream-port'>
      <model name='xio3130-downstream'/>
      <target chassis='6' port='0x0'/>
      <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
    </controller>
    <controller type='pci' index='7' model='dmi-to-pci-bridge'>
      <address type='pci' domain='0x0000' bus='0x03' slot='0x1' function='0x0'/>
    </controller>

# virsh edit vm1-q35-pxb
error: XML error: Invalid PCI address 0000:03:01.0. slot must be <= 0
Failed. Try again? [y,n,i,f,?]:

Comment 5 Han Han 2017-04-17 08:45:32 UTC

Verified on libvirt-3.2.0-2.el7.x86_64.
1. Add a pcie-expander-bus to q35 VM. Add 32 pcie-root-port to the pcie-expander-bus:
 <controller type='pci' index='6' model='pcie-expander-bus'>
      <model name='pxb-pcie'/>
      <target busNr='200'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </controller>
    <controller type='pci' index='7' model='pcie-root-port'>
      <model name='ioh3420'/>
      <target chassis='7' port='0x0'/>
      <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
    </controller>
...
    <controller type='pci' index='38' model='pcie-root-port'>
      <model name='ioh3420'/>
      <target chassis='38' port='0xf8'/>
      <address type='pci' domain='0x0000' bus='0x06' slot='0x1f' function='0x0'/>
    </controller>
2. Start VM and check lspci in VM:
# virsh start q35-expander
Domain q35-expander started

In VM:
# lspci|grep c8
c8:00.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
c8:01.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
c8:02.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
c8:03.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
c8:04.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
c8:05.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
c8:06.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
c8:07.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
c8:08.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
c8:09.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
c8:0a.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
c8:0b.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
c8:0c.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
c8:0d.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
c8:0e.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
c8:0f.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
c8:10.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
c8:11.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
c8:12.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
c8:13.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
c8:14.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
c8:15.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
c8:16.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
c8:17.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
c8:18.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
c8:19.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
c8:1a.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
c8:1b.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
c8:1c.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
c8:1d.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
c8:1e.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)
c8:1f.0 PCI bridge: Intel Corporation 7500/5520/5500/X58 I/O Hub PCI Express Root Port 0 (rev 02)

So slot 0x00 to 0x1f can be used in pcie-expander-bus. Bug fixed.

Comment 6 errata-xmlrpc 2017-08-01 17:16:43 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1846

Comment 7 errata-xmlrpc 2017-08-01 23:57:38 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1846

Note You need to log in before you can comment on or make changes to this bug.