1949813 – Some disks are not hotpluged in guest when hotplug many disks

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1949813 - Some disks are not hotpluged in guest when hotplug many disks

Summary: Some disks are not hotpluged in guest when hotplug many disks

Keywords:
Status:	CLOSED DEFERRED
Alias:	None
Product:	Red Hat Enterprise Linux 9
Classification:	Red Hat
Component:	qemu-kvm
Sub Component:
Version:	9.0
Hardware:	x86_64
OS:	Linux
Priority:	medium
Severity:	medium
Target Milestone:	rc
Target Release:	---
Assignee:	Michael S. Tsirkin
QA Contact:	jingzhao
Docs Contact:
URL:
Whiteboard:
Duplicates (1):	2044211 (view as bug list)
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2021-04-15 07:28 UTC by qing.wang
Modified:	2023-11-30 01:41 UTC (History)
CC List:	17 users (show)
Fixed In Version:
Doc Type:	If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:	2023-02-13 20:58:38 UTC
Type:	---
Target Upstream Version:
Embargoed:
Dependent Products:

Attachments	(Terms of Use)
VM xml, script, dmesg (40.68 KB, application/gzip) 2021-04-20 06:26 UTC, Han Han	no flags	Details
View All

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Red Hat Knowledge Base (Solution)	7006978	0	None	None	None	2023-04-10 23:57:58 UTC

Description qing.wang 2021-04-15 07:28:13 UTC

Description of problem:
If we want to using QMP to hot-plug many disks (>150), there are some disks are not hot-plugged in guest. but there is no error on QMP output and "info block" in HMP display those disks already exist.

Version-Release number of selected component (if applicable):

Red Hat Enterprise Linux release 8.4 Beta (Ootpa)
4.18.0-293.el8.x86_64
qemu-kvm-common-5.2.0-15.module+el8.4.0+10650+50781ca0.x86_64

Guest:4.18.0-293.el8.x86_64

How reproducible:
90%


Steps to Reproduce:
1.create 200 disk image files
qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg0.qcow2 128M
...
qemu-img create -f qcow2 /home/kvm_autotest_root/images/stg200.qcow2 128M

2.boot vm

/usr/libexec/qemu-kvm \
    -name 'avocado-vt-vm1'  \
    -sandbox on  \
    -machine q35,accel=kvm \
    -device pcie-root-port,id=pcie-root-port-0,multifunction=on,bus=pcie.0,addr=0x1,chassis=1 \
    -device pcie-pci-bridge,id=pcie-pci-bridge-0,addr=0x0,bus=pcie-root-port-0  \
    -nodefaults \
    -device VGA,bus=pcie.0,addr=0x2 \
    -m 16G  \
    -smp 10,maxcpus=10,cores=5,threads=1,dies=1,sockets=2  \
    -device pcie-root-port,id=pcie-root-port-1,port=0x1,addr=0x1.0x1,bus=pcie.0,chassis=2 \
    -device qemu-xhci,id=usb1,bus=pcie-root-port-1,addr=0x0 \
    -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \
    -object iothread,id=iothread0 \
    -object iothread,id=iothread1 \
    -device pcie-root-port,id=pcie-root-port-2,port=0x2,addr=0x1.0x2,bus=pcie.0,chassis=3 \
    -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pcie-root-port-2,addr=0x0,iothread=iothread0 \
    -blockdev node-name=file_image1,driver=file,aio=native,filename=/home/kvm_autotest_root/images/rhel830-64-virtio-scsi.qcow2,cache.direct=on,cache.no-flush=off \
    -blockdev node-name=drive_image1,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_image1 \
    -device scsi-hd,id=image1,drive=drive_image1,bootindex=0,write-cache=on \
    -device pcie-root-port,id=pcie-root-port-3,port=0x3,addr=0x1.0x3,bus=pcie.0,chassis=4 \
    -device virtio-net-pci,mac=9a:88:7a:d3:8b:74,id=id9QVzBa,netdev=idEb8qPI,bus=pcie-root-port-3,addr=0x0  \
    -netdev tap,id=idEb8qPI,vhost=on  \
    -rtc base=utc,clock=host,driftfix=slew  \
    -boot menu=off,order=cdn,once=c,strict=off \
    -enable-kvm \
    -device pcie-root-port,id=pcie_extra_root_port_0,multifunction=on,bus=pcie.0,addr=0x3,chassis=5 \
    -device pcie-root-port,id=pcie_extra_root_port_1,addr=0x3.0x1,bus=pcie.0,chassis=6 \
    -device pcie-root-port,id=pcie_extra_root_port_2,addr=0x3.0x2,bus=pcie.0,chassis=7 \
    -device pcie-root-port,id=pcie_extra_root_port_3,addr=0x3.0x3,bus=pcie.0,chassis=8 \
    -device pcie-root-port,id=pcie_extra_root_port_4,addr=0x3.0x4,bus=pcie.0,chassis=9 \
    -device pcie-root-port,id=pcie_extra_root_port_5,addr=0x3.0x5,bus=pcie.0,chassis=10 \
    -device pcie-root-port,id=pcie_extra_root_port_6,addr=0x3.0x6,bus=pcie.0,chassis=11 \
    -device pcie-root-port,id=pcie_extra_root_port_7,addr=0x3.0x7,bus=pcie.0,chassis=12 \
    -device pcie-root-port,id=pcie_extra_root_port_8,multifunction=on,bus=pcie.0,addr=0x4,chassis=13 \
    -device pcie-root-port,id=pcie_extra_root_port_9,addr=0x4.0x1,bus=pcie.0,chassis=14 \
    -device pcie-root-port,id=pcie_extra_root_port_10,addr=0x4.0x2,bus=pcie.0,chassis=15 \
    -device pcie-root-port,id=pcie_extra_root_port_11,addr=0x4.0x3,bus=pcie.0,chassis=16 \
    -device pcie-root-port,id=pcie_extra_root_port_12,addr=0x4.0x4,bus=pcie.0,chassis=17 \
    -device pcie-root-port,id=pcie_extra_root_port_13,addr=0x4.0x5,bus=pcie.0,chassis=18 \
    -device pcie-root-port,id=pcie_extra_root_port_14,addr=0x4.0x6,bus=pcie.0,chassis=19 \
    -device pcie-root-port,id=pcie_extra_root_port_15,addr=0x4.0x7,bus=pcie.0,chassis=20 \
    -device pcie-root-port,id=pcie_extra_root_port_16,multifunction=on,bus=pcie.0,addr=0x5,chassis=21 \
    -device pcie-root-port,id=pcie_extra_root_port_17,addr=0x5.0x1,bus=pcie.0,chassis=22 \
    -device pcie-root-port,id=pcie_extra_root_port_18,addr=0x5.0x2,bus=pcie.0,chassis=23 \
    -device pcie-root-port,id=pcie_extra_root_port_19,addr=0x5.0x3,bus=pcie.0,chassis=24 \
    -device pcie-root-port,id=pcie_extra_root_port_20,addr=0x5.0x4,bus=pcie.0,chassis=25 \
    -device pcie-root-port,id=pcie_extra_root_port_21,addr=0x5.0x5,bus=pcie.0,chassis=26 \
    -device pcie-root-port,id=pcie_extra_root_port_22,addr=0x5.0x6,bus=pcie.0,chassis=27 \
    -device pcie-root-port,id=pcie_extra_root_port_23,addr=0x5.0x7,bus=pcie.0,chassis=28 \
    -device pcie-root-port,id=pcie_extra_root_port_24,multifunction=on,bus=pcie.0,addr=0x6,chassis=29 \
    -device pcie-root-port,id=pcie_extra_root_port_25,addr=0x6.0x1,bus=pcie.0,chassis=30 \
    -device pcie-root-port,id=pcie_extra_root_port_26,addr=0x6.0x2,bus=pcie.0,chassis=31 \
    -device pcie-root-port,id=pcie_extra_root_port_27,addr=0x6.0x3,bus=pcie.0,chassis=32 \
    -device pcie-root-port,id=pcie_extra_root_port_28,addr=0x6.0x4,bus=pcie.0,chassis=33 \
    -device pcie-root-port,id=pcie_extra_root_port_29,addr=0x6.0x5,bus=pcie.0,chassis=34 \
    -device pcie-root-port,id=pcie_extra_root_port_30,addr=0x6.0x6,bus=pcie.0,chassis=35 \
    -device pcie-root-port,id=pcie_extra_root_port_31,addr=0x6.0x7,bus=pcie.0,chassis=36 \
    -device pcie-root-port,id=pcie_extra_root_port_32,multifunction=on,bus=pcie.0,addr=0x7,chassis=37 \
    -device pcie-root-port,id=pcie_extra_root_port_33,addr=0x7.0x1,bus=pcie.0,chassis=38 \
    -device pcie-root-port,id=pcie_extra_root_port_34,addr=0x7.0x2,bus=pcie.0,chassis=39 \
    -device pcie-root-port,id=pcie_extra_root_port_35,addr=0x7.0x3,bus=pcie.0,chassis=40 \
    -device pcie-root-port,id=pcie_extra_root_port_36,addr=0x7.0x4,bus=pcie.0,chassis=41 \
    -device pcie-root-port,id=pcie_extra_root_port_37,addr=0x7.0x5,bus=pcie.0,chassis=42 \
    -device pcie-root-port,id=pcie_extra_root_port_38,addr=0x7.0x6,bus=pcie.0,chassis=43 \
    -device pcie-root-port,id=pcie_extra_root_port_39,addr=0x7.0x7,bus=pcie.0,chassis=44 \
    -device pcie-root-port,id=pcie_extra_root_port_40,multifunction=on,bus=pcie.0,addr=0x8,chassis=45 \
    -device pcie-root-port,id=pcie_extra_root_port_41,addr=0x8.0x1,bus=pcie.0,chassis=46 \
    -device pcie-root-port,id=pcie_extra_root_port_42,addr=0x8.0x2,bus=pcie.0,chassis=47 \
    -device pcie-root-port,id=pcie_extra_root_port_43,addr=0x8.0x3,bus=pcie.0,chassis=48 \
    -device pcie-root-port,id=pcie_extra_root_port_44,addr=0x8.0x4,bus=pcie.0,chassis=49 \
    -device pcie-root-port,id=pcie_extra_root_port_45,addr=0x8.0x5,bus=pcie.0,chassis=50 \
    -device pcie-root-port,id=pcie_extra_root_port_46,addr=0x8.0x6,bus=pcie.0,chassis=51 \
    -device pcie-root-port,id=pcie_extra_root_port_47,addr=0x8.0x7,bus=pcie.0,chassis=52 \
    -device pcie-root-port,id=pcie_extra_root_port_48,multifunction=on,bus=pcie.0,addr=0x9,chassis=53 \
    -device pcie-root-port,id=pcie_extra_root_port_49,addr=0x9.0x1,bus=pcie.0,chassis=54 \
    -device pcie-root-port,id=pcie_extra_root_port_50,addr=0x9.0x2,bus=pcie.0,chassis=55 \
    -device pcie-root-port,id=pcie_extra_root_port_51,addr=0x9.0x3,bus=pcie.0,chassis=56 \
    -device pcie-root-port,id=pcie_extra_root_port_52,addr=0x9.0x4,bus=pcie.0,chassis=57 \
    -device pcie-root-port,id=pcie_extra_root_port_53,addr=0x9.0x5,bus=pcie.0,chassis=58 \
    -device pcie-root-port,id=pcie_extra_root_port_54,addr=0x9.0x6,bus=pcie.0,chassis=59 \
    -device pcie-root-port,id=pcie_extra_root_port_55,addr=0x9.0x7,bus=pcie.0,chassis=60 \
    -device pcie-root-port,id=pcie_extra_root_port_56,multifunction=on,bus=pcie.0,addr=0xa,chassis=61 \
    -device pcie-root-port,id=pcie_extra_root_port_57,addr=0xa.0x1,bus=pcie.0,chassis=62 \
    -device pcie-root-port,id=pcie_extra_root_port_58,addr=0xa.0x2,bus=pcie.0,chassis=63 \
    -device pcie-root-port,id=pcie_extra_root_port_59,addr=0xa.0x3,bus=pcie.0,chassis=64 \
    -device pcie-root-port,id=pcie_extra_root_port_60,addr=0xa.0x4,bus=pcie.0,chassis=65 \
    -device pcie-root-port,id=pcie_extra_root_port_61,addr=0xa.0x5,bus=pcie.0,chassis=66 \
    -device pcie-root-port,id=pcie_extra_root_port_62,addr=0xa.0x6,bus=pcie.0,chassis=67 \
    -device pcie-root-port,id=pcie_extra_root_port_63,addr=0xa.0x7,bus=pcie.0,chassis=68 \
    -device pcie-root-port,id=pcie_extra_root_port_64,multifunction=on,bus=pcie.0,addr=0xb,chassis=69 \
    -device pcie-root-port,id=pcie_extra_root_port_65,addr=0xb.0x1,bus=pcie.0,chassis=70 \
    -device pcie-root-port,id=pcie_extra_root_port_66,addr=0xb.0x2,bus=pcie.0,chassis=71 \
    -device pcie-root-port,id=pcie_extra_root_port_67,addr=0xb.0x3,bus=pcie.0,chassis=72 \
    -device pcie-root-port,id=pcie_extra_root_port_68,addr=0xb.0x4,bus=pcie.0,chassis=73 \
    -device pcie-root-port,id=pcie_extra_root_port_69,addr=0xb.0x5,bus=pcie.0,chassis=74 \
    -device pcie-root-port,id=pcie_extra_root_port_70,addr=0xb.0x6,bus=pcie.0,chassis=75 \
    -device pcie-root-port,id=pcie_extra_root_port_71,addr=0xb.0x7,bus=pcie.0,chassis=76 \
    -device pcie-root-port,id=pcie_extra_root_port_72,multifunction=on,bus=pcie.0,addr=0xc,chassis=77 \
    -device pcie-root-port,id=pcie_extra_root_port_73,addr=0xc.0x1,bus=pcie.0,chassis=78 \
    -device pcie-root-port,id=pcie_extra_root_port_74,addr=0xc.0x2,bus=pcie.0,chassis=79 \
    -device pcie-root-port,id=pcie_extra_root_port_75,addr=0xc.0x3,bus=pcie.0,chassis=80 \
    -device pcie-root-port,id=pcie_extra_root_port_76,addr=0xc.0x4,bus=pcie.0,chassis=81 \
    -device pcie-root-port,id=pcie_extra_root_port_77,addr=0xc.0x5,bus=pcie.0,chassis=82 \
    -device pcie-root-port,id=pcie_extra_root_port_78,addr=0xc.0x6,bus=pcie.0,chassis=83 \
    -device pcie-root-port,id=pcie_extra_root_port_79,addr=0xc.0x7,bus=pcie.0,chassis=84 \
    -device pcie-root-port,id=pcie_extra_root_port_80,multifunction=on,bus=pcie.0,addr=0xd,chassis=85 \
    -device pcie-root-port,id=pcie_extra_root_port_81,addr=0xd.0x1,bus=pcie.0,chassis=86 \
    -device pcie-root-port,id=pcie_extra_root_port_82,addr=0xd.0x2,bus=pcie.0,chassis=87 \
    -device pcie-root-port,id=pcie_extra_root_port_83,addr=0xd.0x3,bus=pcie.0,chassis=88 \
    -device pcie-root-port,id=pcie_extra_root_port_84,addr=0xd.0x4,bus=pcie.0,chassis=89 \
    -device pcie-root-port,id=pcie_extra_root_port_85,addr=0xd.0x5,bus=pcie.0,chassis=90 \
    -device pcie-root-port,id=pcie_extra_root_port_86,addr=0xd.0x6,bus=pcie.0,chassis=91 \
    -device pcie-root-port,id=pcie_extra_root_port_87,addr=0xd.0x7,bus=pcie.0,chassis=92 \
    -device pcie-root-port,id=pcie_extra_root_port_88,multifunction=on,bus=pcie.0,addr=0xe,chassis=93 \
    -device pcie-root-port,id=pcie_extra_root_port_89,addr=0xe.0x1,bus=pcie.0,chassis=94 \
    -device pcie-root-port,id=pcie_extra_root_port_90,addr=0xe.0x2,bus=pcie.0,chassis=95 \
    -device pcie-root-port,id=pcie_extra_root_port_91,addr=0xe.0x3,bus=pcie.0,chassis=96 \
    -device pcie-root-port,id=pcie_extra_root_port_92,addr=0xe.0x4,bus=pcie.0,chassis=97 \
    -device pcie-root-port,id=pcie_extra_root_port_93,addr=0xe.0x5,bus=pcie.0,chassis=98 \
    -device pcie-root-port,id=pcie_extra_root_port_94,addr=0xe.0x6,bus=pcie.0,chassis=99 \
    -device pcie-root-port,id=pcie_extra_root_port_95,addr=0xe.0x7,bus=pcie.0,chassis=100 \
    -device pcie-root-port,id=pcie_extra_root_port_96,multifunction=on,bus=pcie.0,addr=0xf,chassis=101 \
    -device pcie-root-port,id=pcie_extra_root_port_97,addr=0xf.0x1,bus=pcie.0,chassis=102 \
    -device pcie-root-port,id=pcie_extra_root_port_98,addr=0xf.0x2,bus=pcie.0,chassis=103 \
    -device pcie-root-port,id=pcie_extra_root_port_99,addr=0xf.0x3,bus=pcie.0,chassis=104 \
    -device pcie-root-port,id=pcie_extra_root_port_100,addr=0xf.0x4,bus=pcie.0,chassis=105 \
    -device pcie-root-port,id=pcie_extra_root_port_101,addr=0xf.0x5,bus=pcie.0,chassis=106 \
    -device pcie-root-port,id=pcie_extra_root_port_102,addr=0xf.0x6,bus=pcie.0,chassis=107 \
    -device pcie-root-port,id=pcie_extra_root_port_103,addr=0xf.0x7,bus=pcie.0,chassis=108 \
    -device pcie-root-port,id=pcie_extra_root_port_104,multifunction=on,bus=pcie.0,addr=0x10,chassis=109 \
    -device pcie-root-port,id=pcie_extra_root_port_105,addr=0x10.0x1,bus=pcie.0,chassis=110 \
    -device pcie-root-port,id=pcie_extra_root_port_106,addr=0x10.0x2,bus=pcie.0,chassis=111 \
    -device pcie-root-port,id=pcie_extra_root_port_107,addr=0x10.0x3,bus=pcie.0,chassis=112 \
    -device pcie-root-port,id=pcie_extra_root_port_108,addr=0x10.0x4,bus=pcie.0,chassis=113 \
    -device pcie-root-port,id=pcie_extra_root_port_109,addr=0x10.0x5,bus=pcie.0,chassis=114 \
    -device pcie-root-port,id=pcie_extra_root_port_110,addr=0x10.0x6,bus=pcie.0,chassis=115 \
    -device pcie-root-port,id=pcie_extra_root_port_111,addr=0x10.0x7,bus=pcie.0,chassis=116 \
    -device pcie-root-port,id=pcie_extra_root_port_112,multifunction=on,bus=pcie.0,addr=0x11,chassis=117 \
    -device pcie-root-port,id=pcie_extra_root_port_113,addr=0x11.0x1,bus=pcie.0,chassis=118 \
    -device pcie-root-port,id=pcie_extra_root_port_114,addr=0x11.0x2,bus=pcie.0,chassis=119 \
    -device pcie-root-port,id=pcie_extra_root_port_115,addr=0x11.0x3,bus=pcie.0,chassis=120 \
    -device pcie-root-port,id=pcie_extra_root_port_116,addr=0x11.0x4,bus=pcie.0,chassis=121 \
    -device pcie-root-port,id=pcie_extra_root_port_117,addr=0x11.0x5,bus=pcie.0,chassis=122 \
    -device pcie-root-port,id=pcie_extra_root_port_118,addr=0x11.0x6,bus=pcie.0,chassis=123 \
    -device pcie-root-port,id=pcie_extra_root_port_119,addr=0x11.0x7,bus=pcie.0,chassis=124 \
    -device pcie-root-port,id=pcie_extra_root_port_120,multifunction=on,bus=pcie.0,addr=0x12,chassis=125 \
    -device pcie-root-port,id=pcie_extra_root_port_121,addr=0x12.0x1,bus=pcie.0,chassis=126 \
    -device pcie-root-port,id=pcie_extra_root_port_122,addr=0x12.0x2,bus=pcie.0,chassis=127 \
    -device pcie-root-port,id=pcie_extra_root_port_123,addr=0x12.0x3,bus=pcie.0,chassis=128 \
    -device pcie-root-port,id=pcie_extra_root_port_124,addr=0x12.0x4,bus=pcie.0,chassis=129 \
    -device pcie-root-port,id=pcie_extra_root_port_125,addr=0x12.0x5,bus=pcie.0,chassis=130 \
    -device pcie-root-port,id=pcie_extra_root_port_126,addr=0x12.0x6,bus=pcie.0,chassis=131 \
    -device pcie-root-port,id=pcie_extra_root_port_127,addr=0x12.0x7,bus=pcie.0,chassis=132 \
    -device pcie-root-port,id=pcie_extra_root_port_128,multifunction=on,bus=pcie.0,addr=0x13,chassis=133 \
    -device pcie-root-port,id=pcie_extra_root_port_129,addr=0x13.0x1,bus=pcie.0,chassis=134 \
    -device pcie-root-port,id=pcie_extra_root_port_130,addr=0x13.0x2,bus=pcie.0,chassis=135 \
    -device pcie-root-port,id=pcie_extra_root_port_131,addr=0x13.0x3,bus=pcie.0,chassis=136 \
    -device pcie-root-port,id=pcie_extra_root_port_132,addr=0x13.0x4,bus=pcie.0,chassis=137 \
    -device pcie-root-port,id=pcie_extra_root_port_133,addr=0x13.0x5,bus=pcie.0,chassis=138 \
    -device pcie-root-port,id=pcie_extra_root_port_134,addr=0x13.0x6,bus=pcie.0,chassis=139 \
    -device pcie-root-port,id=pcie_extra_root_port_135,addr=0x13.0x7,bus=pcie.0,chassis=140 \
    -device pcie-root-port,id=pcie_extra_root_port_136,multifunction=on,bus=pcie.0,addr=0x14,chassis=141 \
    -device pcie-root-port,id=pcie_extra_root_port_137,addr=0x14.0x1,bus=pcie.0,chassis=142 \
    -device pcie-root-port,id=pcie_extra_root_port_138,addr=0x14.0x2,bus=pcie.0,chassis=143 \
    -device pcie-root-port,id=pcie_extra_root_port_139,addr=0x14.0x3,bus=pcie.0,chassis=144 \
    -device pcie-root-port,id=pcie_extra_root_port_140,addr=0x14.0x4,bus=pcie.0,chassis=145 \
    -device pcie-root-port,id=pcie_extra_root_port_141,addr=0x14.0x5,bus=pcie.0,chassis=146 \
    -device pcie-root-port,id=pcie_extra_root_port_142,addr=0x14.0x6,bus=pcie.0,chassis=147 \
    -device pcie-root-port,id=pcie_extra_root_port_143,addr=0x14.0x7,bus=pcie.0,chassis=148 \
    -device pcie-root-port,id=pcie_extra_root_port_144,multifunction=on,bus=pcie.0,addr=0x15,chassis=149 \
    -device pcie-root-port,id=pcie_extra_root_port_145,addr=0x15.0x1,bus=pcie.0,chassis=150 \
    -device pcie-root-port,id=pcie_extra_root_port_146,addr=0x15.0x2,bus=pcie.0,chassis=151 \
    -device pcie-root-port,id=pcie_extra_root_port_147,addr=0x15.0x3,bus=pcie.0,chassis=152 \
    -device pcie-root-port,id=pcie_extra_root_port_148,addr=0x15.0x4,bus=pcie.0,chassis=153 \
    -device pcie-root-port,id=pcie_extra_root_port_149,addr=0x15.0x5,bus=pcie.0,chassis=154 \
    -device pcie-root-port,id=pcie_extra_root_port_150,addr=0x15.0x6,bus=pcie.0,chassis=155 \
    -device pcie-root-port,id=pcie_extra_root_port_151,addr=0x15.0x7,bus=pcie.0,chassis=156 \
    -device pcie-root-port,id=pcie_extra_root_port_152,multifunction=on,bus=pcie.0,addr=0x16,chassis=157 \
    -device pcie-root-port,id=pcie_extra_root_port_153,addr=0x16.0x1,bus=pcie.0,chassis=158 \
    -device pcie-root-port,id=pcie_extra_root_port_154,addr=0x16.0x2,bus=pcie.0,chassis=159 \
    -device pcie-root-port,id=pcie_extra_root_port_155,addr=0x16.0x3,bus=pcie.0,chassis=160 \
    -device pcie-root-port,id=pcie_extra_root_port_156,addr=0x16.0x4,bus=pcie.0,chassis=161 \
    -device pcie-root-port,id=pcie_extra_root_port_157,addr=0x16.0x5,bus=pcie.0,chassis=162 \
    -device pcie-root-port,id=pcie_extra_root_port_158,addr=0x16.0x6,bus=pcie.0,chassis=163 \
    -device pcie-root-port,id=pcie_extra_root_port_159,addr=0x16.0x7,bus=pcie.0,chassis=164 \
    -device pcie-root-port,id=pcie_extra_root_port_160,multifunction=on,bus=pcie.0,addr=0x17,chassis=165 \
    -device pcie-root-port,id=pcie_extra_root_port_161,addr=0x17.0x1,bus=pcie.0,chassis=166 \
    -device pcie-root-port,id=pcie_extra_root_port_162,addr=0x17.0x2,bus=pcie.0,chassis=167 \
    -device pcie-root-port,id=pcie_extra_root_port_163,addr=0x17.0x3,bus=pcie.0,chassis=168 \
    -device pcie-root-port,id=pcie_extra_root_port_164,addr=0x17.0x4,bus=pcie.0,chassis=169 \
    -device pcie-root-port,id=pcie_extra_root_port_165,addr=0x17.0x5,bus=pcie.0,chassis=170 \
    -device pcie-root-port,id=pcie_extra_root_port_166,addr=0x17.0x6,bus=pcie.0,chassis=171 \
    -device pcie-root-port,id=pcie_extra_root_port_167,addr=0x17.0x7,bus=pcie.0,chassis=172 \
    -device pcie-root-port,id=pcie_extra_root_port_168,multifunction=on,bus=pcie.0,addr=0x18,chassis=173 \
    -device pcie-root-port,id=pcie_extra_root_port_169,addr=0x18.0x1,bus=pcie.0,chassis=174 \
    -device pcie-root-port,id=pcie_extra_root_port_170,addr=0x18.0x2,bus=pcie.0,chassis=175 \
    -device pcie-root-port,id=pcie_extra_root_port_171,addr=0x18.0x3,bus=pcie.0,chassis=176 \
    -device pcie-root-port,id=pcie_extra_root_port_172,addr=0x18.0x4,bus=pcie.0,chassis=177 \
    -device pcie-root-port,id=pcie_extra_root_port_173,addr=0x18.0x5,bus=pcie.0,chassis=178 \
    -device pcie-root-port,id=pcie_extra_root_port_174,addr=0x18.0x6,bus=pcie.0,chassis=179 \
    -device pcie-root-port,id=pcie_extra_root_port_175,addr=0x18.0x7,bus=pcie.0,chassis=180 \
    -device pcie-root-port,id=pcie_extra_root_port_176,multifunction=on,bus=pcie.0,addr=0x19,chassis=181 \
    -device pcie-root-port,id=pcie_extra_root_port_177,addr=0x19.0x1,bus=pcie.0,chassis=182 \
    -device pcie-root-port,id=pcie_extra_root_port_178,addr=0x19.0x2,bus=pcie.0,chassis=183 \
    -device pcie-root-port,id=pcie_extra_root_port_179,addr=0x19.0x3,bus=pcie.0,chassis=184 \
    -device pcie-root-port,id=pcie_extra_root_port_180,addr=0x19.0x4,bus=pcie.0,chassis=185 \
    -device pcie-root-port,id=pcie_extra_root_port_181,addr=0x19.0x5,bus=pcie.0,chassis=186 \
    -device pcie-root-port,id=pcie_extra_root_port_182,addr=0x19.0x6,bus=pcie.0,chassis=187 \
    -device pcie-root-port,id=pcie_extra_root_port_183,addr=0x19.0x7,bus=pcie.0,chassis=188 \
    -device pcie-root-port,id=pcie_extra_root_port_184,multifunction=on,bus=pcie.0,addr=0x1a,chassis=189 \
    -device pcie-root-port,id=pcie_extra_root_port_185,addr=0x1a.0x1,bus=pcie.0,chassis=190 \
    -device pcie-root-port,id=pcie_extra_root_port_186,addr=0x1a.0x2,bus=pcie.0,chassis=191 \
    -device pcie-root-port,id=pcie_extra_root_port_187,addr=0x1a.0x3,bus=pcie.0,chassis=192 \
    -device pcie-root-port,id=pcie_extra_root_port_188,addr=0x1a.0x4,bus=pcie.0,chassis=193 \
    -device pcie-root-port,id=pcie_extra_root_port_189,addr=0x1a.0x5,bus=pcie.0,chassis=194 \
    -device pcie-root-port,id=pcie_extra_root_port_190,addr=0x1a.0x6,bus=pcie.0,chassis=195 \
    -device pcie-root-port,id=pcie_extra_root_port_191,addr=0x1a.0x7,bus=pcie.0,chassis=196 \
    -device pcie-root-port,id=pcie_extra_root_port_192,multifunction=on,bus=pcie.0,addr=0x1b,chassis=197 \
    -device pcie-root-port,id=pcie_extra_root_port_193,addr=0x1b.0x1,bus=pcie.0,chassis=198 \
    -device pcie-root-port,id=pcie_extra_root_port_194,addr=0x1b.0x2,bus=pcie.0,chassis=199 \
    -device pcie-root-port,id=pcie_extra_root_port_195,addr=0x1b.0x3,bus=pcie.0,chassis=200 \
    -device pcie-root-port,id=pcie_extra_root_port_196,addr=0x1b.0x4,bus=pcie.0,chassis=201 \
    -device pcie-root-port,id=pcie_extra_root_port_197,addr=0x1b.0x5,bus=pcie.0,chassis=202 \
    -device pcie-root-port,id=pcie_extra_root_port_198,addr=0x1b.0x6,bus=pcie.0,chassis=203 \
    -device pcie-root-port,id=pcie_extra_root_port_199,addr=0x1b.0x7,bus=pcie.0,chassis=204 \
    -monitor stdio \
    -chardev file,id=qmp_id_qmpmonitor1,path=/var/tmp/monitor-qmpdbg.log,server,nowait \
    -mon chardev=qmp_id_qmpmonitor1,mode=control  \
    -qmp tcp:0:5955,server,nowait  \
    -vnc :5  \
    -chardev file,path=/var/tmp/monitor-serialdbg.log,id=serial_id_serial0 \
    -device isa-serial,chardev=serial_id_serial0  \

3.after guest boot, run add_disks.sh

cat add_disk.sh:
NUM_LUNS=150

add_devices() {
  exec 3<>/dev/tcp/localhost/5955
  echo "$@"
  echo -e "{'execute':'qmp_capabilities'}" >&3
  read response <&3
  echo $response
  for i in $(seq 1 $NUM_LUNS) ; do
  cmd="{'execute': 'blockdev-add', 'arguments': {'node-name': 'file_stg${i}', 'driver': 'file','filename': '/home/kvm_autotest_root/images/stg${i}.qcow2'}}"
  echo "$cmd"
  echo -e "$cmd" >&3
  read response <&3
  echo "$response"
  cmd="{'execute': 'blockdev-add', 'arguments': {'node-name': 'drive_stg${i}', 'driver': 'qcow2', 'file': 'file_stg${i}'}} "
  echo "$cmd"
  echo -e "$cmd" >&3
  read response <&3
  echo "$response"
  cmd="{'execute': 'device_add', 'arguments': {'id': 'virtio_scsi_pci${i}', 'driver': 'virtio-scsi-pci', 'bus':'pcie_extra_root_port_${i}','addr':'0x0', 'iothread':'iothread1'}}"
  echo "$cmd"
  echo -e "$cmd" >&3
  read response <&3
  echo "$response"
  cmd="{'execute': 'device_add', 'arguments':{'driver':'scsi-hd', 'id':'stg${i}', 'bus':'virtio_scsi_pci${i}.0', 'drive':'drive_stg${i}'}}"
  echo "$cmd"
  echo -e "$cmd" >&3
  read response <&3
  echo "$response"
#  sleep 0.1

  done
}

add_devices

4.check disk number in guest
lsblk -dn |wc -l
137

Actual results:
The number of disks in guest less than plugged disks.

Expected results:
Those disks should really exist in guest.

Additional info:a
If set the NUM_LUNS=100 then the test pass. 

automation reference:
python ConfigTest.py --testcase=multi_disk_random_hotplug.serial.single_type.q35  --iothread_scheme=roundrobin --nr_iothreads=2 --platform=x86_64 --guestname=RHEL.8.4.0 --driveformat=virtio_scsi --nicmodel=virtio_net --imageformat=qcow2 --machines=q35

Comment 1 qing.wang 2021-04-15 08:05:32 UTC

It looks like have issue on the virtio-scsi-pci bus number?

dmesg and /var/log/messages may be found via
http://fileshare.englab.nay.redhat.com/pub/section2/images_backup/qbugs/1949813/2021-04-15/



BTW There is no issue if we attach all disks under same bus like as.
{'execute': 'device_add', 'arguments':{'driver':'scsi-hd', 'id':'stg1', 'bus':'virtio_scsi_pci0.0', 'drive':'drive_stg199'}}
....
{'execute': 'device_add', 'arguments':{'driver':'scsi-hd', 'id':'stg199', 'bus':'virtio_scsi_pci0.0', 'drive':'drive_stg199'}}

Comment 2 John Ferlan 2021-04-19 20:24:44 UTC

Can you reproduce using libvirt?

Comment 4 Han Han 2021-04-20 06:26:13 UTC

Created attachment 1773628 [details]
VM xml, script, dmesg

Reproduce on qemu-6.0.0-0.1.rc2.fc35.x86_64 libvirt-v7.2.0-224-gac87f612ba:

After attach 151 scsi controller and disks. Only 119 scsi disks appears in guest:
➜  ~ virsh dumpxml hhan|grep 'target.*sd'|wc -l
151

(in guest) [root@fedora ~]# lsblk|grep sd|wc -l
119

However, inside the guest dmesg, there are many errors like:
[  279.740430] pcieport 0000:00:13.0: BAR 13: no space for [io  size 0x1000]
[  279.762785] pcieport 0000:00:13.0: BAR 13: no space for [io  size 0x1000]
[  280.403698] pcieport 0000:00:13.4: BAR 13: no space for [io  size 0x1000]
[  280.421791] pcieport 0000:00:13.4: BAR 13: no space for [io  size 0x1000]
[  285.506665] pcieport 0000:00:13.3: BAR 13: no space for [io  size 0x1000]
[  285.530195] pcieport 0000:00:13.3: BAR 13: no space for [io  size 0x1000]
[  286.861321] pcieport 0000:00:12.7: BAR 13: no space for [io  size 0x1000]
[  286.890969] pcieport 0000:00:12.7: BAR 13: no space for [io  size 0x1000]
[  287.822758] pcieport 0000:00:12.3: BAR 13: no space for [io  size 0x1000]
[  287.853368] pcieport 0000:00:12.3: BAR 13: no space for [io  size 0x1000]
[  292.606822] pcieport 0000:00:02.6: BAR 13: no space for [io  size 0x1000]
[  292.647048] pcieport 0000:00:02.6: BAR 13: no space for [io  size 0x1000]

Files in the attachment:
hhan.xml: The VM xml
run.sh: Script to reproduce the issue
dmesg: The dmesg of guest

Comment 6 John Ferlan 2021-09-08 21:41:34 UTC

Move RHEL-AV bugs to RHEL9. If necessary to resolve in RHEL8, then clone to the current RHEL8 release.

Comment 8 Kevin Wolf 2022-03-16 13:27:53 UTC

This doesn't seem to be related to storage, but to PCI. It may be just hitting a limit rather than an actual bug, but reassigning for the right people to have a look.

Comment 9 Igor Mammedov 2022-03-23 15:12:00 UTC

"no space for" hints that you are exhausting IO which is limited resource.
See comment in related BZs: https://bugzilla.redhat.com/show_bug.cgi?id=2056230#c19

Theoretically PCIe device should work even without IO (not sure why it doesn't work, CCing Michael, Gerd).
Please provide guest logs running on qemu-6.2 (comment 7 only has QEMU start up command).

What could help you is
 1. turning off ACPI hotplug and use native PCIe hotplug using option
         -global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=off
    guest may try to re-assign resources in more fine-grained way and
    it might fail anyways if there is not enough resources are available to accommodate all configured devices.
 2. try to use PCIE-to-PCI bridges instead of a PCIe port per device, and attach devices to bridges.
    It will casuse less resource fragmentation but it still might fail for the same reseasons as (1)
 3. attach as many as possible disks to a single scsi controller, that should minimize IO resource consumption.

Comment 10 Gerd Hoffmann 2022-03-25 09:12:07 UTC

> Theoretically PCIe device should work even without IO (not sure why it
> doesn't work, CCing Michael, Gerd).

Reportedly 100 devices work fine, which is already *way* beyond the
number of pci(e) bridges we have io resources for.  So I don't think
io resources are the problem.

How does /proc/iomem look like?

> What could help you is
>  1. turning off ACPI hotplug and use native PCIe hotplug using option
>          -global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=off

Yes, that might make a difference.

>  3. attach as many as possible disks to a single scsi controller, that
> should minimize IO resource consumption.

That's a good advice too.

Comment 11 Yiqian Wei 2022-04-02 06:54:12 UTC

(In reply to Gerd Hoffmann from comment #10)
> > Theoretically PCIe device should work even without IO (not sure why it
> > doesn't work, CCing Michael, Gerd).
> 
> Reportedly 100 devices work fine, which is already *way* beyond the
> number of pci(e) bridges we have io resources for.  So I don't think
> io resources are the problem.
> 
How does /proc/iomem look like?

Check /proc/iomem in the guest after hotplug 100 devices to the guest, please check the attachment:100_iomem_guest.txt.The hotplug 100 device to the guest is successful.
Check /proc/iomem in the guest after hotplug 150 devices to the guest, please check attachment:150_iomem_guest.txt.The hotplug 150 device to guest failed and found that dmesg have call trace and failed messages,please check the attachment:dmesg_guest.txt,It seems to be the same problem as another Bug 2044211 .

> 
> > What could help you is
> >  1. turning off ACPI hotplug and use native PCIe hotplug using option
> >          -global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=off
> 
> Yes, that might make a difference.
> 
> >  3. attach as many as possible disks to a single scsi controller, that
> > should minimize IO resource consumption.
> 
> That's a good advice too.

Comment 15 Gerd Hoffmann 2022-04-05 11:28:56 UTC

[  137.723139] Trying to free already-free IRQ 1217
[  137.724397] WARNING: CPU: 0 PID: 48 at kernel/irq/manage.c:1833 __free_irq+0x9a/0x300

[  137.771732] Call Trace:
[  137.772403]  ? __irq_put_desc_unlock+0x18/0x40
[  137.773560]  free_irq+0x2e/0x60
[  137.774399]  vp_del_vqs+0x82/0x250
[  137.775286]  vp_find_vqs_msix+0x109/0x510
[  137.776334]  vp_find_vqs+0x3a/0x190
[  137.777241]  vp_modern_find_vqs+0x17/0x60
[  137.778302]  virtscsi_init+0x134/0x310 [virtio_scsi]

Looks like happening in the vp_find_vqs_msix() error code path.
Some IRQ problem?

I'm wondering how we land at IRQ #1217.  virtio-scsi uses 4 lines (config, control, event, request), the pcie root port needs one too, makes 5 irqs per device. 1200 is much larger than 5 * 150 though ...

Cat you attach /proc/interrupts?

Comment 19 Gerd Hoffmann 2022-08-16 10:00:56 UTC

> Looks like happening in the vp_find_vqs_msix() error code path.
> Some IRQ problem?
> 
> I'm wondering how we land at IRQ #1217.  virtio-scsi uses 4 lines (config,
> control, event, request), the pcie root port needs one too, makes 5 irqs per
> device. 1200 is much larger than 5 * 150 though ...

Well, it is one request queue *per cpu*, so 9 not 4 irq lines per virtio device.

> Cat you attach /proc/interrupts?

[ ... ]
1206:          0          0          0          0          0          0   PCI-MSI 61865984-edge      virtio114-config
1207:          0          0          0          0          0          0   PCI-MSI 61865985-edge      virtio114-control
1208:          0          0          0          0          0          0   PCI-MSI 61865986-edge      virtio114-event
1209:          0          0          0          0          0          0   PCI-MSI 61865987-edge      virtio114-request
1210:          0          6          0          0          0          0   PCI-MSI 61865988-edge      virtio114-request
1211:          0          0        101          0          0          0   PCI-MSI 61865989-edge      virtio114-request
1212:          0          0          0          0          0          0   PCI-MSI 61865990-edge      virtio114-request
1213:          0          0          0          0          0          0   PCI-MSI 61865991-edge      virtio114-request
1214:          0          0          0          0          0        288   PCI-MSI 61865992-edge      virtio114-request

1215:          0          0          0          0          0          0   PCI-MSI 62390272-edge      virtio115-config
1216:          0          0          0          0          0        398   PCI-MSI 62390273-edge      virtio115-virtqueues
1217:          0          0          0          0          0          0   PCI-MSI 62914560-edge      virtio116-config
1218:          0        388          0          0          0          0   PCI-MSI 62914561-edge      virtio116-virtqueues
1219:          0          0          0          0          0          0   PCI-MSI 63438848-edge      virtio117-config
1220:          0          0          0        341          0          0   PCI-MSI 63438849-edge      virtio117-virtqueues
1221:          0          0          0          0          0          0   PCI-MSI 63963136-edge      virtio118-config
1222:          0          0          0          0          0        386   PCI-MSI 63963137-edge      virtio118-virtqueues
[ ... ]

So we have 4 devices which got two irqs only (one config, one shared by all virt queues).

So it looks like linux initialized the first 114 devices normally.
Then could not allocate 9 IRQs for 115, falling back to the shared line mode which needs only two IRQs.
Same worked for devices 116, 117, 118.
119+ not working because even allocating only two IRQs failed.

So we are really running against some IRQ line limit here.  Igor?

What happens when you add '-global virtio-scsi-pci.vectors=4' to the qemu command line (configures virtio-scsi to use 4 IRQ lines per device max)?
That should avoid hitting the IRQ vector limit.

Comment 20 Igor Mammedov 2022-08-22 08:38:06 UTC

(In reply to Gerd Hoffmann from comment #19)
> > Looks like happening in the vp_find_vqs_msix() error code path.
> > Some IRQ problem?

[...]

> So we have 4 devices which got two irqs only (one config, one shared by all
> virt queues).
> 
> So it looks like linux initialized the first 114 devices normally.
> Then could not allocate 9 IRQs for 115, falling back to the shared line mode
> which needs only two IRQs.
> Same worked for devices 116, 117, 118.
> 119+ not working because even allocating only two IRQs failed.
> 
> So we are really running against some IRQ line limit here.  Igor?

I don't really know, perhaps Michael have an idea what to do here 

[...]

Comment 23 Michael S. Tsirkin 2022-10-20 11:21:22 UTC

I agree -global virtio-scsi-pci.vectors=4 is the way to go here so
no qemu bug.
 
this message within guest:

[  137.723139] Trying to free already-free IRQ 1217
[  137.724397] WARNING: CPU: 0 PID: 48 at kernel/irq/manage.c:1833 __free_irq+0x9a/0x300

[  137.771732] Call Trace:
[  137.772403]  ? __irq_put_desc_unlock+0x18/0x40
[  137.773560]  free_irq+0x2e/0x60
[  137.774399]  vp_del_vqs+0x82/0x250
[  137.775286]  vp_find_vqs_msix+0x109/0x510
[  137.776334]  vp_find_vqs+0x3a/0x190
[  137.777241]  vp_modern_find_vqs+0x17/0x60
[  137.778302]  virtscsi_init+0x134/0x310 [virtio_scsi]


seems to indicate incorrect error handling within guest.
worth checking with latest RHEL if the error message
is still there open a bugzilla against guest kernel.

Comment 24 John Ferlan 2022-10-26 21:25:05 UTC

Given the above suggestion, can we add '-global virtio-scsi-pci.vectors=4' to command line (not sure it's possible w/ libvirt)?  and of course update that Polarian test too.

Comment 25 qing.wang 2022-11-01 04:01:38 UTC

It passed the script and manual test after add "-global virtio-scsi-pci.vectors=4'"

python ConfigTest.py --testcase=multi_disk_random_hotplug.serial.single_type.q35  --iothread_scheme=roundrobin --nr_iothreads=2 --platform=x86_64 --guestname=RHEL.9.1.0 --driveformat=virtio_scsi --nicmodel=virtio_net --imageformat=qcow2 --machines=q35 --clone=no --firmware=default_bios --customsparams="extra_params = ' -global virtio-scsi-pci.vectors=4'" --nrepeat=5

manual test command line:
/usr/libexec/qemu-kvm \
    -name 'avocado-vt-vm1'  \
    -sandbox on  \
    -machine q35,accel=kvm \
    -device pcie-root-port,id=pcie-root-port-0,multifunction=on,bus=pcie.0,addr=0x1,chassis=1 \
    -device pcie-pci-bridge,id=pcie-pci-bridge-0,addr=0x0,bus=pcie-root-port-0  \
    -nodefaults \
    -device VGA,bus=pcie.0,addr=0x2 \
    -m 16G  \
    -smp 10,maxcpus=10,cores=5,threads=1,dies=1,sockets=2  \
    -cpu host \
    -device pcie-root-port,id=pcie-root-port-1,port=0x1,addr=0x1.0x1,bus=pcie.0,chassis=2 \
    -device qemu-xhci,id=usb1,bus=pcie-root-port-1,addr=0x0 \
    -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \
    -object iothread,id=iothread0 \
    -object iothread,id=iothread1 \
    -device pcie-root-port,id=pcie-root-port-2,port=0x2,addr=0x1.0x2,bus=pcie.0,chassis=3 \
    -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pcie-root-port-2,addr=0x0,iothread=iothread0 \
    -blockdev node-name=file_image1,driver=file,aio=native,filename=/home/kvm_autotest_root/images/rhel920-64-virtio-scsi.qcow2,cache.direct=on,cache.no-flush=off \
    -blockdev node-name=drive_image1,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_image1 \
    -device scsi-hd,id=image1,drive=drive_image1,bootindex=0,write-cache=on \
    -device pcie-root-port,id=pcie-root-port-3,port=0x3,addr=0x1.0x3,bus=pcie.0,chassis=4 \
    -device virtio-net-pci,mac=9a:88:7a:d3:8b:74,id=id9QVzBa,netdev=idEb8qPI,bus=pcie-root-port-3,addr=0x0  \
    -netdev tap,id=idEb8qPI,vhost=on  \
    -rtc base=utc,clock=host,driftfix=slew  \
    -boot menu=off,order=cdn,once=c,strict=off \
    -enable-kvm \
    -device pcie-root-port,id=pcie_extra_root_port_0,multifunction=on,bus=pcie.0,addr=0x3,chassis=5 \
    -device pcie-root-port,id=pcie_extra_root_port_1,addr=0x3.0x1,bus=pcie.0,chassis=6 \
    -device pcie-root-port,id=pcie_extra_root_port_2,addr=0x3.0x2,bus=pcie.0,chassis=7 \
    -device pcie-root-port,id=pcie_extra_root_port_3,addr=0x3.0x3,bus=pcie.0,chassis=8 \
    -device pcie-root-port,id=pcie_extra_root_port_4,addr=0x3.0x4,bus=pcie.0,chassis=9 \
    -device pcie-root-port,id=pcie_extra_root_port_5,addr=0x3.0x5,bus=pcie.0,chassis=10 \
    -device pcie-root-port,id=pcie_extra_root_port_6,addr=0x3.0x6,bus=pcie.0,chassis=11 \
    -device pcie-root-port,id=pcie_extra_root_port_7,addr=0x3.0x7,bus=pcie.0,chassis=12 \
    -device pcie-root-port,id=pcie_extra_root_port_8,multifunction=on,bus=pcie.0,addr=0x4,chassis=13 \
    -device pcie-root-port,id=pcie_extra_root_port_9,addr=0x4.0x1,bus=pcie.0,chassis=14 \
    -device pcie-root-port,id=pcie_extra_root_port_10,addr=0x4.0x2,bus=pcie.0,chassis=15 \
    -device pcie-root-port,id=pcie_extra_root_port_11,addr=0x4.0x3,bus=pcie.0,chassis=16 \
    -device pcie-root-port,id=pcie_extra_root_port_12,addr=0x4.0x4,bus=pcie.0,chassis=17 \
    -device pcie-root-port,id=pcie_extra_root_port_13,addr=0x4.0x5,bus=pcie.0,chassis=18 \
    -device pcie-root-port,id=pcie_extra_root_port_14,addr=0x4.0x6,bus=pcie.0,chassis=19 \
    -device pcie-root-port,id=pcie_extra_root_port_15,addr=0x4.0x7,bus=pcie.0,chassis=20 \
    -device pcie-root-port,id=pcie_extra_root_port_16,multifunction=on,bus=pcie.0,addr=0x5,chassis=21 \
    -device pcie-root-port,id=pcie_extra_root_port_17,addr=0x5.0x1,bus=pcie.0,chassis=22 \
    -device pcie-root-port,id=pcie_extra_root_port_18,addr=0x5.0x2,bus=pcie.0,chassis=23 \
    -device pcie-root-port,id=pcie_extra_root_port_19,addr=0x5.0x3,bus=pcie.0,chassis=24 \
    -device pcie-root-port,id=pcie_extra_root_port_20,addr=0x5.0x4,bus=pcie.0,chassis=25 \
    -device pcie-root-port,id=pcie_extra_root_port_21,addr=0x5.0x5,bus=pcie.0,chassis=26 \
    -device pcie-root-port,id=pcie_extra_root_port_22,addr=0x5.0x6,bus=pcie.0,chassis=27 \
    -device pcie-root-port,id=pcie_extra_root_port_23,addr=0x5.0x7,bus=pcie.0,chassis=28 \
    -device pcie-root-port,id=pcie_extra_root_port_24,multifunction=on,bus=pcie.0,addr=0x6,chassis=29 \
    -device pcie-root-port,id=pcie_extra_root_port_25,addr=0x6.0x1,bus=pcie.0,chassis=30 \
    -device pcie-root-port,id=pcie_extra_root_port_26,addr=0x6.0x2,bus=pcie.0,chassis=31 \
    -device pcie-root-port,id=pcie_extra_root_port_27,addr=0x6.0x3,bus=pcie.0,chassis=32 \
    -device pcie-root-port,id=pcie_extra_root_port_28,addr=0x6.0x4,bus=pcie.0,chassis=33 \
    -device pcie-root-port,id=pcie_extra_root_port_29,addr=0x6.0x5,bus=pcie.0,chassis=34 \
    -device pcie-root-port,id=pcie_extra_root_port_30,addr=0x6.0x6,bus=pcie.0,chassis=35 \
    -device pcie-root-port,id=pcie_extra_root_port_31,addr=0x6.0x7,bus=pcie.0,chassis=36 \
    -device pcie-root-port,id=pcie_extra_root_port_32,multifunction=on,bus=pcie.0,addr=0x7,chassis=37 \
    -device pcie-root-port,id=pcie_extra_root_port_33,addr=0x7.0x1,bus=pcie.0,chassis=38 \
    -device pcie-root-port,id=pcie_extra_root_port_34,addr=0x7.0x2,bus=pcie.0,chassis=39 \
    -device pcie-root-port,id=pcie_extra_root_port_35,addr=0x7.0x3,bus=pcie.0,chassis=40 \
    -device pcie-root-port,id=pcie_extra_root_port_36,addr=0x7.0x4,bus=pcie.0,chassis=41 \
    -device pcie-root-port,id=pcie_extra_root_port_37,addr=0x7.0x5,bus=pcie.0,chassis=42 \
    -device pcie-root-port,id=pcie_extra_root_port_38,addr=0x7.0x6,bus=pcie.0,chassis=43 \
    -device pcie-root-port,id=pcie_extra_root_port_39,addr=0x7.0x7,bus=pcie.0,chassis=44 \
    -device pcie-root-port,id=pcie_extra_root_port_40,multifunction=on,bus=pcie.0,addr=0x8,chassis=45 \
    -device pcie-root-port,id=pcie_extra_root_port_41,addr=0x8.0x1,bus=pcie.0,chassis=46 \
    -device pcie-root-port,id=pcie_extra_root_port_42,addr=0x8.0x2,bus=pcie.0,chassis=47 \
    -device pcie-root-port,id=pcie_extra_root_port_43,addr=0x8.0x3,bus=pcie.0,chassis=48 \
    -device pcie-root-port,id=pcie_extra_root_port_44,addr=0x8.0x4,bus=pcie.0,chassis=49 \
    -device pcie-root-port,id=pcie_extra_root_port_45,addr=0x8.0x5,bus=pcie.0,chassis=50 \
    -device pcie-root-port,id=pcie_extra_root_port_46,addr=0x8.0x6,bus=pcie.0,chassis=51 \
    -device pcie-root-port,id=pcie_extra_root_port_47,addr=0x8.0x7,bus=pcie.0,chassis=52 \
    -device pcie-root-port,id=pcie_extra_root_port_48,multifunction=on,bus=pcie.0,addr=0x9,chassis=53 \
    -device pcie-root-port,id=pcie_extra_root_port_49,addr=0x9.0x1,bus=pcie.0,chassis=54 \
    -device pcie-root-port,id=pcie_extra_root_port_50,addr=0x9.0x2,bus=pcie.0,chassis=55 \
    -device pcie-root-port,id=pcie_extra_root_port_51,addr=0x9.0x3,bus=pcie.0,chassis=56 \
    -device pcie-root-port,id=pcie_extra_root_port_52,addr=0x9.0x4,bus=pcie.0,chassis=57 \
    -device pcie-root-port,id=pcie_extra_root_port_53,addr=0x9.0x5,bus=pcie.0,chassis=58 \
    -device pcie-root-port,id=pcie_extra_root_port_54,addr=0x9.0x6,bus=pcie.0,chassis=59 \
    -device pcie-root-port,id=pcie_extra_root_port_55,addr=0x9.0x7,bus=pcie.0,chassis=60 \
    -device pcie-root-port,id=pcie_extra_root_port_56,multifunction=on,bus=pcie.0,addr=0xa,chassis=61 \
    -device pcie-root-port,id=pcie_extra_root_port_57,addr=0xa.0x1,bus=pcie.0,chassis=62 \
    -device pcie-root-port,id=pcie_extra_root_port_58,addr=0xa.0x2,bus=pcie.0,chassis=63 \
    -device pcie-root-port,id=pcie_extra_root_port_59,addr=0xa.0x3,bus=pcie.0,chassis=64 \
    -device pcie-root-port,id=pcie_extra_root_port_60,addr=0xa.0x4,bus=pcie.0,chassis=65 \
    -device pcie-root-port,id=pcie_extra_root_port_61,addr=0xa.0x5,bus=pcie.0,chassis=66 \
    -device pcie-root-port,id=pcie_extra_root_port_62,addr=0xa.0x6,bus=pcie.0,chassis=67 \
    -device pcie-root-port,id=pcie_extra_root_port_63,addr=0xa.0x7,bus=pcie.0,chassis=68 \
    -device pcie-root-port,id=pcie_extra_root_port_64,multifunction=on,bus=pcie.0,addr=0xb,chassis=69 \
    -device pcie-root-port,id=pcie_extra_root_port_65,addr=0xb.0x1,bus=pcie.0,chassis=70 \
    -device pcie-root-port,id=pcie_extra_root_port_66,addr=0xb.0x2,bus=pcie.0,chassis=71 \
    -device pcie-root-port,id=pcie_extra_root_port_67,addr=0xb.0x3,bus=pcie.0,chassis=72 \
    -device pcie-root-port,id=pcie_extra_root_port_68,addr=0xb.0x4,bus=pcie.0,chassis=73 \
    -device pcie-root-port,id=pcie_extra_root_port_69,addr=0xb.0x5,bus=pcie.0,chassis=74 \
    -device pcie-root-port,id=pcie_extra_root_port_70,addr=0xb.0x6,bus=pcie.0,chassis=75 \
    -device pcie-root-port,id=pcie_extra_root_port_71,addr=0xb.0x7,bus=pcie.0,chassis=76 \
    -device pcie-root-port,id=pcie_extra_root_port_72,multifunction=on,bus=pcie.0,addr=0xc,chassis=77 \
    -device pcie-root-port,id=pcie_extra_root_port_73,addr=0xc.0x1,bus=pcie.0,chassis=78 \
    -device pcie-root-port,id=pcie_extra_root_port_74,addr=0xc.0x2,bus=pcie.0,chassis=79 \
    -device pcie-root-port,id=pcie_extra_root_port_75,addr=0xc.0x3,bus=pcie.0,chassis=80 \
    -device pcie-root-port,id=pcie_extra_root_port_76,addr=0xc.0x4,bus=pcie.0,chassis=81 \
    -device pcie-root-port,id=pcie_extra_root_port_77,addr=0xc.0x5,bus=pcie.0,chassis=82 \
    -device pcie-root-port,id=pcie_extra_root_port_78,addr=0xc.0x6,bus=pcie.0,chassis=83 \
    -device pcie-root-port,id=pcie_extra_root_port_79,addr=0xc.0x7,bus=pcie.0,chassis=84 \
    -device pcie-root-port,id=pcie_extra_root_port_80,multifunction=on,bus=pcie.0,addr=0xd,chassis=85 \
    -device pcie-root-port,id=pcie_extra_root_port_81,addr=0xd.0x1,bus=pcie.0,chassis=86 \
    -device pcie-root-port,id=pcie_extra_root_port_82,addr=0xd.0x2,bus=pcie.0,chassis=87 \
    -device pcie-root-port,id=pcie_extra_root_port_83,addr=0xd.0x3,bus=pcie.0,chassis=88 \
    -device pcie-root-port,id=pcie_extra_root_port_84,addr=0xd.0x4,bus=pcie.0,chassis=89 \
    -device pcie-root-port,id=pcie_extra_root_port_85,addr=0xd.0x5,bus=pcie.0,chassis=90 \
    -device pcie-root-port,id=pcie_extra_root_port_86,addr=0xd.0x6,bus=pcie.0,chassis=91 \
    -device pcie-root-port,id=pcie_extra_root_port_87,addr=0xd.0x7,bus=pcie.0,chassis=92 \
    -device pcie-root-port,id=pcie_extra_root_port_88,multifunction=on,bus=pcie.0,addr=0xe,chassis=93 \
    -device pcie-root-port,id=pcie_extra_root_port_89,addr=0xe.0x1,bus=pcie.0,chassis=94 \
    -device pcie-root-port,id=pcie_extra_root_port_90,addr=0xe.0x2,bus=pcie.0,chassis=95 \
    -device pcie-root-port,id=pcie_extra_root_port_91,addr=0xe.0x3,bus=pcie.0,chassis=96 \
    -device pcie-root-port,id=pcie_extra_root_port_92,addr=0xe.0x4,bus=pcie.0,chassis=97 \
    -device pcie-root-port,id=pcie_extra_root_port_93,addr=0xe.0x5,bus=pcie.0,chassis=98 \
    -device pcie-root-port,id=pcie_extra_root_port_94,addr=0xe.0x6,bus=pcie.0,chassis=99 \
    -device pcie-root-port,id=pcie_extra_root_port_95,addr=0xe.0x7,bus=pcie.0,chassis=100 \
    -device pcie-root-port,id=pcie_extra_root_port_96,multifunction=on,bus=pcie.0,addr=0xf,chassis=101 \
    -device pcie-root-port,id=pcie_extra_root_port_97,addr=0xf.0x1,bus=pcie.0,chassis=102 \
    -device pcie-root-port,id=pcie_extra_root_port_98,addr=0xf.0x2,bus=pcie.0,chassis=103 \
    -device pcie-root-port,id=pcie_extra_root_port_99,addr=0xf.0x3,bus=pcie.0,chassis=104 \
    -device pcie-root-port,id=pcie_extra_root_port_100,addr=0xf.0x4,bus=pcie.0,chassis=105 \
    -device pcie-root-port,id=pcie_extra_root_port_101,addr=0xf.0x5,bus=pcie.0,chassis=106 \
    -device pcie-root-port,id=pcie_extra_root_port_102,addr=0xf.0x6,bus=pcie.0,chassis=107 \
    -device pcie-root-port,id=pcie_extra_root_port_103,addr=0xf.0x7,bus=pcie.0,chassis=108 \
    -device pcie-root-port,id=pcie_extra_root_port_104,multifunction=on,bus=pcie.0,addr=0x10,chassis=109 \
    -device pcie-root-port,id=pcie_extra_root_port_105,addr=0x10.0x1,bus=pcie.0,chassis=110 \
    -device pcie-root-port,id=pcie_extra_root_port_106,addr=0x10.0x2,bus=pcie.0,chassis=111 \
    -device pcie-root-port,id=pcie_extra_root_port_107,addr=0x10.0x3,bus=pcie.0,chassis=112 \
    -device pcie-root-port,id=pcie_extra_root_port_108,addr=0x10.0x4,bus=pcie.0,chassis=113 \
    -device pcie-root-port,id=pcie_extra_root_port_109,addr=0x10.0x5,bus=pcie.0,chassis=114 \
    -device pcie-root-port,id=pcie_extra_root_port_110,addr=0x10.0x6,bus=pcie.0,chassis=115 \
    -device pcie-root-port,id=pcie_extra_root_port_111,addr=0x10.0x7,bus=pcie.0,chassis=116 \
    -device pcie-root-port,id=pcie_extra_root_port_112,multifunction=on,bus=pcie.0,addr=0x11,chassis=117 \
    -device pcie-root-port,id=pcie_extra_root_port_113,addr=0x11.0x1,bus=pcie.0,chassis=118 \
    -device pcie-root-port,id=pcie_extra_root_port_114,addr=0x11.0x2,bus=pcie.0,chassis=119 \
    -device pcie-root-port,id=pcie_extra_root_port_115,addr=0x11.0x3,bus=pcie.0,chassis=120 \
    -device pcie-root-port,id=pcie_extra_root_port_116,addr=0x11.0x4,bus=pcie.0,chassis=121 \
    -device pcie-root-port,id=pcie_extra_root_port_117,addr=0x11.0x5,bus=pcie.0,chassis=122 \
    -device pcie-root-port,id=pcie_extra_root_port_118,addr=0x11.0x6,bus=pcie.0,chassis=123 \
    -device pcie-root-port,id=pcie_extra_root_port_119,addr=0x11.0x7,bus=pcie.0,chassis=124 \
    -device pcie-root-port,id=pcie_extra_root_port_120,multifunction=on,bus=pcie.0,addr=0x12,chassis=125 \
    -device pcie-root-port,id=pcie_extra_root_port_121,addr=0x12.0x1,bus=pcie.0,chassis=126 \
    -device pcie-root-port,id=pcie_extra_root_port_122,addr=0x12.0x2,bus=pcie.0,chassis=127 \
    -device pcie-root-port,id=pcie_extra_root_port_123,addr=0x12.0x3,bus=pcie.0,chassis=128 \
    -device pcie-root-port,id=pcie_extra_root_port_124,addr=0x12.0x4,bus=pcie.0,chassis=129 \
    -device pcie-root-port,id=pcie_extra_root_port_125,addr=0x12.0x5,bus=pcie.0,chassis=130 \
    -device pcie-root-port,id=pcie_extra_root_port_126,addr=0x12.0x6,bus=pcie.0,chassis=131 \
    -device pcie-root-port,id=pcie_extra_root_port_127,addr=0x12.0x7,bus=pcie.0,chassis=132 \
    -device pcie-root-port,id=pcie_extra_root_port_128,multifunction=on,bus=pcie.0,addr=0x13,chassis=133 \
    -device pcie-root-port,id=pcie_extra_root_port_129,addr=0x13.0x1,bus=pcie.0,chassis=134 \
    -device pcie-root-port,id=pcie_extra_root_port_130,addr=0x13.0x2,bus=pcie.0,chassis=135 \
    -device pcie-root-port,id=pcie_extra_root_port_131,addr=0x13.0x3,bus=pcie.0,chassis=136 \
    -device pcie-root-port,id=pcie_extra_root_port_132,addr=0x13.0x4,bus=pcie.0,chassis=137 \
    -device pcie-root-port,id=pcie_extra_root_port_133,addr=0x13.0x5,bus=pcie.0,chassis=138 \
    -device pcie-root-port,id=pcie_extra_root_port_134,addr=0x13.0x6,bus=pcie.0,chassis=139 \
    -device pcie-root-port,id=pcie_extra_root_port_135,addr=0x13.0x7,bus=pcie.0,chassis=140 \
    -device pcie-root-port,id=pcie_extra_root_port_136,multifunction=on,bus=pcie.0,addr=0x14,chassis=141 \
    -device pcie-root-port,id=pcie_extra_root_port_137,addr=0x14.0x1,bus=pcie.0,chassis=142 \
    -device pcie-root-port,id=pcie_extra_root_port_138,addr=0x14.0x2,bus=pcie.0,chassis=143 \
    -device pcie-root-port,id=pcie_extra_root_port_139,addr=0x14.0x3,bus=pcie.0,chassis=144 \
    -device pcie-root-port,id=pcie_extra_root_port_140,addr=0x14.0x4,bus=pcie.0,chassis=145 \
    -device pcie-root-port,id=pcie_extra_root_port_141,addr=0x14.0x5,bus=pcie.0,chassis=146 \
    -device pcie-root-port,id=pcie_extra_root_port_142,addr=0x14.0x6,bus=pcie.0,chassis=147 \
    -device pcie-root-port,id=pcie_extra_root_port_143,addr=0x14.0x7,bus=pcie.0,chassis=148 \
    -device pcie-root-port,id=pcie_extra_root_port_144,multifunction=on,bus=pcie.0,addr=0x15,chassis=149 \
    -device pcie-root-port,id=pcie_extra_root_port_145,addr=0x15.0x1,bus=pcie.0,chassis=150 \
    -device pcie-root-port,id=pcie_extra_root_port_146,addr=0x15.0x2,bus=pcie.0,chassis=151 \
    -device pcie-root-port,id=pcie_extra_root_port_147,addr=0x15.0x3,bus=pcie.0,chassis=152 \
    -device pcie-root-port,id=pcie_extra_root_port_148,addr=0x15.0x4,bus=pcie.0,chassis=153 \
    -device pcie-root-port,id=pcie_extra_root_port_149,addr=0x15.0x5,bus=pcie.0,chassis=154 \
    -device pcie-root-port,id=pcie_extra_root_port_150,addr=0x15.0x6,bus=pcie.0,chassis=155 \
    -device pcie-root-port,id=pcie_extra_root_port_151,addr=0x15.0x7,bus=pcie.0,chassis=156 \
    -device pcie-root-port,id=pcie_extra_root_port_152,multifunction=on,bus=pcie.0,addr=0x16,chassis=157 \
    -device pcie-root-port,id=pcie_extra_root_port_153,addr=0x16.0x1,bus=pcie.0,chassis=158 \
    -device pcie-root-port,id=pcie_extra_root_port_154,addr=0x16.0x2,bus=pcie.0,chassis=159 \
    -device pcie-root-port,id=pcie_extra_root_port_155,addr=0x16.0x3,bus=pcie.0,chassis=160 \
    -device pcie-root-port,id=pcie_extra_root_port_156,addr=0x16.0x4,bus=pcie.0,chassis=161 \
    -device pcie-root-port,id=pcie_extra_root_port_157,addr=0x16.0x5,bus=pcie.0,chassis=162 \
    -device pcie-root-port,id=pcie_extra_root_port_158,addr=0x16.0x6,bus=pcie.0,chassis=163 \
    -device pcie-root-port,id=pcie_extra_root_port_159,addr=0x16.0x7,bus=pcie.0,chassis=164 \
    -device pcie-root-port,id=pcie_extra_root_port_160,multifunction=on,bus=pcie.0,addr=0x17,chassis=165 \
    -device pcie-root-port,id=pcie_extra_root_port_161,addr=0x17.0x1,bus=pcie.0,chassis=166 \
    -device pcie-root-port,id=pcie_extra_root_port_162,addr=0x17.0x2,bus=pcie.0,chassis=167 \
    -device pcie-root-port,id=pcie_extra_root_port_163,addr=0x17.0x3,bus=pcie.0,chassis=168 \
    -device pcie-root-port,id=pcie_extra_root_port_164,addr=0x17.0x4,bus=pcie.0,chassis=169 \
    -device pcie-root-port,id=pcie_extra_root_port_165,addr=0x17.0x5,bus=pcie.0,chassis=170 \
    -device pcie-root-port,id=pcie_extra_root_port_166,addr=0x17.0x6,bus=pcie.0,chassis=171 \
    -device pcie-root-port,id=pcie_extra_root_port_167,addr=0x17.0x7,bus=pcie.0,chassis=172 \
    -device pcie-root-port,id=pcie_extra_root_port_168,multifunction=on,bus=pcie.0,addr=0x18,chassis=173 \
    -device pcie-root-port,id=pcie_extra_root_port_169,addr=0x18.0x1,bus=pcie.0,chassis=174 \
    -device pcie-root-port,id=pcie_extra_root_port_170,addr=0x18.0x2,bus=pcie.0,chassis=175 \
    -device pcie-root-port,id=pcie_extra_root_port_171,addr=0x18.0x3,bus=pcie.0,chassis=176 \
    -device pcie-root-port,id=pcie_extra_root_port_172,addr=0x18.0x4,bus=pcie.0,chassis=177 \
    -device pcie-root-port,id=pcie_extra_root_port_173,addr=0x18.0x5,bus=pcie.0,chassis=178 \
    -device pcie-root-port,id=pcie_extra_root_port_174,addr=0x18.0x6,bus=pcie.0,chassis=179 \
    -device pcie-root-port,id=pcie_extra_root_port_175,addr=0x18.0x7,bus=pcie.0,chassis=180 \
    -device pcie-root-port,id=pcie_extra_root_port_176,multifunction=on,bus=pcie.0,addr=0x19,chassis=181 \
    -device pcie-root-port,id=pcie_extra_root_port_177,addr=0x19.0x1,bus=pcie.0,chassis=182 \
    -device pcie-root-port,id=pcie_extra_root_port_178,addr=0x19.0x2,bus=pcie.0,chassis=183 \
    -device pcie-root-port,id=pcie_extra_root_port_179,addr=0x19.0x3,bus=pcie.0,chassis=184 \
    -device pcie-root-port,id=pcie_extra_root_port_180,addr=0x19.0x4,bus=pcie.0,chassis=185 \
    -device pcie-root-port,id=pcie_extra_root_port_181,addr=0x19.0x5,bus=pcie.0,chassis=186 \
    -device pcie-root-port,id=pcie_extra_root_port_182,addr=0x19.0x6,bus=pcie.0,chassis=187 \
    -device pcie-root-port,id=pcie_extra_root_port_183,addr=0x19.0x7,bus=pcie.0,chassis=188 \
    -device pcie-root-port,id=pcie_extra_root_port_184,multifunction=on,bus=pcie.0,addr=0x1a,chassis=189 \
    -device pcie-root-port,id=pcie_extra_root_port_185,addr=0x1a.0x1,bus=pcie.0,chassis=190 \
    -device pcie-root-port,id=pcie_extra_root_port_186,addr=0x1a.0x2,bus=pcie.0,chassis=191 \
    -device pcie-root-port,id=pcie_extra_root_port_187,addr=0x1a.0x3,bus=pcie.0,chassis=192 \
    -device pcie-root-port,id=pcie_extra_root_port_188,addr=0x1a.0x4,bus=pcie.0,chassis=193 \
    -device pcie-root-port,id=pcie_extra_root_port_189,addr=0x1a.0x5,bus=pcie.0,chassis=194 \
    -device pcie-root-port,id=pcie_extra_root_port_190,addr=0x1a.0x6,bus=pcie.0,chassis=195 \
    -device pcie-root-port,id=pcie_extra_root_port_191,addr=0x1a.0x7,bus=pcie.0,chassis=196 \
    -device pcie-root-port,id=pcie_extra_root_port_192,multifunction=on,bus=pcie.0,addr=0x1b,chassis=197 \
    -device pcie-root-port,id=pcie_extra_root_port_193,addr=0x1b.0x1,bus=pcie.0,chassis=198 \
    -device pcie-root-port,id=pcie_extra_root_port_194,addr=0x1b.0x2,bus=pcie.0,chassis=199 \
    -device pcie-root-port,id=pcie_extra_root_port_195,addr=0x1b.0x3,bus=pcie.0,chassis=200 \
    -device pcie-root-port,id=pcie_extra_root_port_196,addr=0x1b.0x4,bus=pcie.0,chassis=201 \
    -device pcie-root-port,id=pcie_extra_root_port_197,addr=0x1b.0x5,bus=pcie.0,chassis=202 \
    -device pcie-root-port,id=pcie_extra_root_port_198,addr=0x1b.0x6,bus=pcie.0,chassis=203 \
    -device pcie-root-port,id=pcie_extra_root_port_199,addr=0x1b.0x7,bus=pcie.0,chassis=204 \
    -monitor stdio \
    -chardev file,id=qmp_id_qmpmonitor1,path=/var/tmp/monitor-qmpdbg.log,server=on,wait=off \
    -mon chardev=qmp_id_qmpmonitor1,mode=control  \
    -qmp tcp:0:5955,server=on,wait=off  \
    -vnc :5  \
    -chardev file,path=/var/tmp/monitor-serial.log,id=id_serial0 \
    -device isa-serial,chardev=id_serial0  \
    -chardev file,path=/var/tmp/monitor-seabios.log,id=id_seabios0,server=on,wait=off \
    -device isa-debugcon,chardev=id_seabios0,iobase=0x402 \
    -global virtio-scsi-pci.vectors=4


Does "global virtio-scsi-pci.vectors=4" become the default setting or just a temporary solution?

Comment 26 John Ferlan 2022-11-02 10:59:01 UTC

Redirecting the needinfo to Michael. I doubt it becomes the default, but a final decision is up to the developers. 

Perhaps documentation or some other means for a customer to know to add that is required for this type of situation.

Comment 27 John Ferlan 2023-02-13 20:58:38 UTC

This aging bug was planned to be resolved within the 9.2.0 release timeframe; however, since it has not been addressed by ITM 24, we are closing as deferred. 

If this bug is reopened, there must be a downstream commit ready to be posted for merge or the bug must be part of some rebase. When reopening, add the release ITR and appropriate DTM.

Comment 28 Julia Suvorova 2023-03-28 14:57:17 UTC

*** Bug 2044211 has been marked as a duplicate of this bug. ***

Comment 30 qing.wang 2023-03-31 09:26:57 UTC

Hit this issue without "global virtio-scsi-pci.vectors=4"

Red Hat Enterprise Linux release 9.2 Beta (Plow)
5.14.0-289.el9.x86_64
qemu-kvm-7.2.0-14.el9_2.x86_64
seabios-bin-1.16.1-1.el9.noarch
edk2-ovmf-20230301gitf80f052277c8-1.el9.noarch
libvirt-9.0.0-9.el9_2.x86_64
virtio-win-prewhql-0.1-235.iso

Note You need to log in before you can comment on or make changes to this bug.