1132900 – Failed to start guest if all slots of one bus have been specified by adding pci devices

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1132900 - Failed to start guest if all slots of one bus have been specified by adding pci devices

Summary: Failed to start guest if all slots of one bus have been specified by adding p...

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat Enterprise Linux 7
Classification:	Red Hat
Component:	libvirt
Sub Component:
Version:	7.1
Hardware:	x86_64
OS:	Linux
Priority:	medium
Severity:	medium
Target Milestone:	rc
Target Release:	---
Assignee:	Erik Skultety
QA Contact:	Virtualization Bugs
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2014-08-22 08:54 UTC by Luyao Huang
Modified:	2015-11-19 05:47 UTC (History)
CC List:	7 users (show)
Fixed In Version:	libvirt-1.2.13-1.el7
Doc Type:	Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed:	2015-11-19 05:47:53 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Red Hat Bugzilla	1004593	0	medium	CLOSED	libvirt should provide a more useful error message when a PCI controller is configured to plug into itself (bus = index)	2021-02-22 00:41:40 UTC
Red Hat Product Errata	RHBA-2015:2202	0	normal	SHIPPED_LIVE	libvirt bug fix and enhancement update	2015-11-19 08:17:58 UTC

Internal Links: 1004593

Description Luyao Huang 2014-08-22 08:54:55 UTC

description of problem:
Failed to start guest if all slots of one bus have been
specified by adding pci devices

Version-Release number of selected component (if applicable):
libvirt-1.2.7-1.el7.x86_64
qemu-kvm-rhev-2.1.0-2.el7.x86_64


How reproducible:
100%

Steps to Reproduce:
1.edit guest and use all device address in bus 0
# virsh edit r7
.....
  <devices>
    <emulator>/usr/libexec/qemu-kvm</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='none'/>
      <source file='/var/lib/libvirt/images/r7_latest.img'/>
      <target dev='hda' bus='ide'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <controller type='ide' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='usb' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'/>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </controller>
    <interface type='network'>
      <mac address='52:54:00:a8:e5:e4'/>
      <source network='default'/>
      <model type='rtl8139'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <interface type='network'>
      <mac address='52:54:00:57:99:c3'/>
      <source network='default'/>
      <model type='rtl8139'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </interface>
    <interface type='network'>
      <mac address='52:54:00:a6:c8a'/>
      <source network='default'/>
      <model type='rtl8139'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </interface>
...
Repeat 22 times here.
...
    <interface type='network'>
      <mac address='52:54:00:04:69:75'/>
      <source network='default'/>
      <model type='rtl8139'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x0'/>
    </interface>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <graphics type='vnc' port='-1' autoport='yes'/>
    <sound model='ich6'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </sound>
    <video>
      <model type='qxl' ram='65536' vram='65536' heads='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </memballoon>
  </devices>
....

2.# virsh start r7
error: Failed to start domain r7
error: internal error: early end of file from monitor: possible problem:
2014-08-22T05:01:20.912312Z qemu-kvm: -device pci-bridge,chassis_nr=1,id=pci.1,bus=pci.1,addr=0x1: Bus 'pci.1' not found


3.there are a new pci-bridge create by libvirt
<controller type='pci' index='1' model='pci-bridge'>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x01' function='0x0'/>   wrong bus number
    </controller>

Actual results:
guest can't start 

Expected results:
guest start normal or report error during use the last slot address

Additional info:
and also this bug will cause a guest which hot-plug interface until can't attach will cannot start after managedsave

By Jianwei Hu:
When there are no free slots on bus 0, libvirt will generate bus 1 to extend free slots automatically, generally this is no problem(libvirt can reserve a slot smartly), but for below example, the pci-bridge need a bus 0 address for itself as upstream port, unfortunately, libvirt can not assign a slot on bus 0 for that PCI device(pci-bridge)

Comment 3 Erik Skultety 2015-01-26 09:21:33 UTC

Fixed upstream:

commit 5d6904b9919cd15a5b079d4523b3bfd79f7b0726
Author: Erik Skultety <eskultet>
Date:   Fri Jan 23 13:17:42 2015 +0100

    qemu: Fix auto-adding PCI bridge when all slots are reserved
    
    Commit 93c8ca tried to fix the issue with auto-adding of a PCI bridge
    controller, but didn't work properly in all scenarios.
    
    This patch provides a better fix of the issue when all slots on a PCI bus
    are reserved by devices with user specified addresses and no additional
    bridges need to be created.
    
    Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1132900

v1.2.12-rc1-21-g5d6904b

Comment 5 Hu Jianwei 2015-04-10 08:55:35 UTC

Verified as below:

[root@localhost ~]# rpm -q libvirt
libvirt-1.2.14-1.el7.x86_64

1. Attach more than 32 pci devices,then start guest.(expected)
[root@localhost ~]# for i in `seq 1 33`; do  virsh attach-device r71 net1.xml --config; done
Device attached successfully

Device attached successfully

...

Device attached successfully

Device attached successfully


[root@localhost ~]# virsh dumpxml r71 | grep "<interface" |wc -l
33

[root@localhost ~]# virsh dumpxml r71 | grep pci-bridge -b3
1853-      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x2'/>
1935-    </controller>
1953-    <controller type='pci' index='0' model='pci-root'/>
2009:    <controller type='pci' index='1' model='pci-bridge'>
2066-      <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x0'/>
2148-    </controller>
2166-    <controller type='ide' index='0'>
[root@localhost ~]# 
[root@localhost ~]# virsh start r71
Domain r71 started

2. All slots are reserved as endpoint, libvirt will fail to create a pci bridge.(expected)
...
    <interface type='network'>
      <mac address='52:54:00:43:0b:f9'/>
      <source network='default'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x0'/>
    </interface>
    <interface type='network'>
      <mac address='52:54:00:43:0b:f9'/>
      <source network='default'/>
      <model type='virtio'/>
    </interface>
...
[root@localhost ~]# virsh edit r71
error: unsupported configuration: failed to create PCI bridge on bus 1: too many devices with fixed addresses
Failed. Try again? [y,n,i,f,?]: 

3. Leave 1 available slot, then hot-pulg pci devices, we just attached one devcie.(expected) 
[root@localhost ~]# virsh start r71
Domain r71 started

[root@localhost ~]# virsh attach-device r71 net1.xml 
Device attached successfully

[root@localhost ~]# virsh attach-device r71 net1.xml 
error: Failed to attach device from net1.xml
error: internal error: No more available PCI slots

We can get expected results, changed to Verified.

Comment 6 Hu Jianwei 2015-05-18 01:54:15 UTC

Thanks for your information, I'll validate that bug after changing to ON_QA.

But, I found your patch has been merged into below version,please double check it.

[root@localhost ~]# rpm -q libvirt 
libvirt-1.2.15-1.el7.x86_64

1. add below wrong pci bridge controller, the index is equal to bus of itself
    <controller type='pci' index='2' model='pci-bridge'>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x01' function='0x0'/>
    </controller>
[root@localhost ~]# virsh edit r71
error: unsupported configuration: failed to create PCI bridge on bus 2: too many devices with fixed addresses
Failed. Try again? [y,n,i,f,?]:

2. add below wrong bridge, the index is less than its bus 
    <controller type='pci' index='3' model='pci-bridge'>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x01' function='0x0'/>
    </controller>

[root@localhost ~]# virsh edit r71
error: unsupported configuration: failed to create PCI bridge on bus 4: too many devices with fixed addresses
Failed. Try again? [y,n,i,f,?]: 


Thanks.

Comment 7 Ján Tomko 2015-05-18 14:50:46 UTC

Yes, the patch (by Erik Skultety) I mentioned in bug 1004593 was in libvirt-1.2.13-1.el7 along with the patch mentioned comment 3.

Comment 9 errata-xmlrpc 2015-11-19 05:47:53 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-2202.html

Note You need to log in before you can comment on or make changes to this bug.