Bug 1946716

Initial support is merged upstream for 'pc' machine type.

Q35 support will come as part of ACPI PCI hotplug, that's works in progress.

Assigned to Amnon for initial triage per bz process and age of bug created or assigned to virt-maint without triage.

'pc' machine got support fro it in qemu-6.0,
'q35' support depends on adding ACPI PCI hotplug to it (work in progress)

(In reply to Igor Mammedov from comment #3)
> 'pc' machine got support fro it in qemu-6.0,
Hi, Igor

Since this feature is already supported on the ‘PC’ machine,I would like to confirm with you how to test it. Could you please help me review it？
==>My Test Steps
1.Boot a guest with 'PC' machine.
/usr/libexec/qemu-kvm \
-name 'avocado-vt-vm1'  \
-sandbox on  \
-machine pc  \
-nodefaults \
-device VGA,bus=pci.0,addr=0x2 \
-m 6144  \
-smp 6,maxcpus=6,cores=3,threads=1,dies=1,sockets=2  \
-cpu 'Haswell-noTSX',+kvm_pv_unhalt \
-device qemu-xhci,id=usb1,bus=pci.0,addr=0x3 \
-device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \
-blockdev node-name=file_image1,driver=file,auto-read-only=on,discard=unmap,aio=threads,filename=/home/rhel850-64.qcow2,cache.direct=on,cache.no-flush=off \
-blockdev node-name=drive_image1,driver=qcow2,read-only=off,cache.direct=on,cache.no-flush=off,file=file_image1 \
-device virtio-blk-pci,id=image1,drive=drive_image1,write-cache=on,bus=pci.0,addr=0x4 \
-device virtio-net-pci,mac=9a:72:f6:17:9c:8e,id=idZDyy06,netdev=idM5WilQ,bus=pci.0,addr=0x5  \
-netdev tap,id=idM5WilQ,vhost=on  \
-vnc :4 \
-rtc base=utc,clock=host,driftfix=slew  \
-boot menu=off,order=cdn,once=c,strict=off \
-enable-kvm \
-monitor stdio \
-qmp tcp:0:5555,server,nowait \

2.Hot plug a nic
Such as:
{'execute': 'netdev_add', 'arguments': {'type': 'tap', 'id': 'hostnet'}}
{"return": {}}
{'execute': 'device_add', 'arguments': {'id':'net','driver':'virtio-net-pci','netdev':'hostnet','mac':'9a:97:91:78:f3:44','bus':'pci.0','addr':'0x7'}}
{"return": {}}
3.Check the name of the nic inside the guest, it's ens7.

==>My Question
1.Does the nic name depend on the location of the slot?Is there such a correspondence between nic name and slot? For example: "addr=0x5 <-> ens5" , "addr=0x6 <-> ens6"
2.The checkpoint of the new function could be understood as: After multiple hot plugs/unplugs(on the same slot) or multiple reboot, the nic name is stable. Please correct me if I'M wrong, thanks in advance.

> 'q35' support depends on adding ACPI PCI hotplug to it (work in progress)
1. 'pc' machine has been supported in qemu-6.0, will 'q35' machine continue to use this bug for tracking?
2. Will 'q35' and 'pc' take the same test method?

Best Regards
Lei

(In reply to Lei Yang from comment #4)
> (In reply to Igor Mammedov from comment #3)
> > 'pc' machine got support fro it in qemu-6.0,
> Hi, Igor
> 
> Since this feature is already supported on the ‘PC’ machine,I would like to
> confirm with you how to test it. Could you please help me review it？
[...]

> ==>My Question
> 1.Does the nic name depend on the location of the slot?Is there such a
> correspondence between nic name and slot? For example: "addr=0x5 <-> ens5" ,
> "addr=0x6 <-> ens6"

to use feature, you have to add "acpi-index=N" property to NIC,
where N is > 0. The default value of acpi-index is 0 and means that the feature
is disabled on that PCI device.

When feature is enabled, NIC name in the guest follows onboard naming scheme
(for details see the 1st link in BZ description, it works only on linux guests only).

In short, name will be 'enoN' where N is the acpi-index value you provided on CLI or
during hotplug.

> 2.The checkpoint of the new function could be understood as: After multiple
> hot plugs/unplugs(on the same slot) or multiple reboot, the nic name is
> stable. Please correct me if I'M wrong, thanks in advance

with default naming, NIC name depends on PCI address assigned to it.

with acpi-index, NIC name stays the same as long as acpi-index is not changed.
i.e. If you plug NIC, in to a different PCI slot/bridge, you should still
get the same NIC name, i.e. naming no longer depends on PCI topology.

It should work both with:
  * hotplug and coldplug
  * replug into the same slot or another
  * unplug acpi-index nic from the slot, and plug a nic with a different acpi-index in that slot

> > 'q35' support depends on adding ACPI PCI hotplug to it (work in progress)
> 1. 'pc' machine has been supported in qemu-6.0, will 'q35' machine continue
> to use this bug for tracking?
I think we should use this BZ to track both 'pc' and 'q35'

> 2. Will 'q35' and 'pc' take the same test method?

I guess PCI ports where NIc is attached will differ between the two,
but otherwise testing method is the same

> Best Regards
> Lei

Hi Igor

I have a question to confirm with you. Could you please help me check it? Thank in advace.

1.Does this new feature affect other PCI devices?
2.If other devices are affected, is the test method the same as the test steps of the nic?

Best Regards
Lei

It looks like this option does not effect disk drive name ? is it expected ?

Test steps:

1.create images

qemu-img create -f qcow2 /home/kvm_autotest_root/images/data1.qcow2 1G
qemu-img create -f qcow2 /home/kvm_autotest_root/images/data2.qcow2 2G
qemu-img create -f qcow2 /home/kvm_autotest_root/images/data3.qcow2 3G


2,boot vm  with multi disks with different acpi-index

/usr/libexec/qemu-kvm \
  -name testvm \
  -machine pc \
  -m 8G \
  -smp 8 \
  -cpu host,svm,+kvm_pv_unhalt \
  -device qemu-xhci,id=usb1 \
  -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \
  -device \
  virtio-scsi-pci,id=scsi0,bus=pci.0 \
  -device virtio-scsi-pci,id=scsi1,bus=pci.0 \
  -blockdev driver=qcow2,file.driver=file,cache.direct=off,cache.no-flush=on,file.filename=/home/kvm_autotest_root/images/rhel840-64-virtio.qcow2,node-name=drive_image1 \
  -device virtio-blk-pci,id=os,drive=drive_image1,bus=pci.0,bootindex=0,serial=OS_DISK \
  -blockdev \
  driver=qcow2,file.driver=file,file.filename=/home/kvm_autotest_root/images/data1.qcow2,node-name=data_image1 \
  -device virtio-blk-pci,id=data1,drive=data_image1,bus=pci.0,bootindex=1,acpi-index=1,serial=DATA1_DISK \
  -blockdev driver=qcow2,file.driver=file,file.filename=/home/kvm_autotest_root/images/data2.qcow2,node-name=data_image2 \
  -device virtio-blk-pci,id=data2,drive=data_image2,bus=pci.0,bootindex=2,acpi-index=3,serial=DATA2_DISK \
  -blockdev driver=qcow2,file.driver=file,file.filename=/home/kvm_autotest_root/images/data3.qcow2,node-name=data_image3 \
  -device virtio-blk-pci,id=data3,drive=data_image3,bus=pci.0,bootindex=3,acpi-index=2,serial=DATA3_DISK \
  -vnc :5 \
  -monitor stdio \
  -qmp tcp:0:5955,server=on,wait=off \
  -device virtio-net-pci,mac=9a:b5:b6:b1:b2:b7,id=idMmq1jH,vectors=4,netdev=idxgXAlm,bus=pci.0 \
  -netdev tap,id=idxgXAlm \
  -chardev socket,id=qmp_id_qmpmonitor1,path=/var/tmp/monitor-qmp7.log,server=on,wait=off \
  -mon chardev=qmp_id_qmpmonitor1,mode=control \
  -chardev file,path=/var/tmp/monitor-serial7.log,id=serial_id_serial0 \
  -device isa-serial,chardev=serial_id_serial0 \
  -D debug.log \
  -boot menu=on,reboot-timeout=1000

3. login guest and check disks, 
lsblk
root@vm-115-242 /home $ lsblk -nd
sr0   11:0    1 1024M  0 rom  
vda  252:0    0   20G  0 disk 
vdb  252:16   0    1G  0 disk 
vdc  252:32   0    2G  0 disk 
vdd  252:48   0    3G  0 disk
 
(The disk drive name not be controlled  by acpi-index)

4 hot-unplug data3 disk

  {"execute": "qmp_capabilities"}

  {"execute": "device_del", "arguments": {"id": "data3"}}

5 hot-unplug data3 disk with different acpi-index

  {"execute": "device_add", "arguments": {"driver": "virtio-blk-pci", "id": "data3", "drive": "data_image3", "bus": "pci.0", "acpi-index": "5"}}

6.check disk drive name in guest
it is same as step3, no new drive name .


Please help to check step 3 and step6, the result is expected?

(In reply to Lei Yang from comment #6)
> Hi Igor
> 
> I have a question to confirm with you. Could you please help me check it?
> Thank in advace.
> 
> 1.Does this new feature affect other PCI devices?
> 2.If other devices are affected, is the test method the same as the test
> steps of the nic?

it only affects NICs so far

> 
> Best Regards
> Lei

(In reply to qing.wang from comment #7)
> It looks like this option does not effect disk drive name ? is it expected ?

yes. option has effect only on NICs and only on non ancient linux
(pls, see the 1st link in BZ description)

1.Test env:
host:
4.18.0-310.el8.x86_64
qemu-kvm-6.0.0-18.module+el8.5.0+11243+5269aaa1.x86_64
guest:
4.18.0-310.el8.x86_64



2.Test step:

(1) check if the qemu cmd line of PF/VF has acpi-index option
# /usr/libexec/qemu-kvm -device  vfio-pci,?
vfio-pci options:
  acpi-index=<uint32>    -  (default: 0)
...


(2) Use the following qemu cmd to test:
...
-machine pc  \
-device vfio-pci,host=$domain:$bug:$device.$function,id=hostdev1,bus=pci.0,addr=$pci_address,acpi-index=$number \
...

    Testing scenarios:
        (1)start a vm with PF/VF which has acpi-index
        (2) hotplug a PF/VF which has acpi-index into VF
        (3) replug  a PF/VF which has acpi-index into the same slot or another slot
        (4) unplug PF/VF which has acpi-index from a slot, and plug PF/VF which has a different acpi-index into that slot




3.Test result:
When "acpi-index=N" (N>0) is enabled, PF/VF interface name in the vm is "enoN".

Hi Igor, 

could you please help check the comment 10?

I found that this acpi-index option can also affect vfio-vf/vfio-pf.

Could you please tell me whether QE needs to use vf/pf to add additional test cases and test for this option ?

Is it enough to test this attribute only with virtio-net-pci ?

(In reply to Yanghang Liu from comment #11)
> Hi Igor, 
> 
> could you please help check the comment 10?
> 
> I found that this acpi-index option can also affect vfio-vf/vfio-pf.

it should affect any PCI device from guest point of view as long as it is the NIC

> 
> Could you please tell me whether QE needs to use vf/pf to add additional
> test cases and test for this option ?
> 
> Is it enough to test this attribute only with virtio-net-pci ?

Yes, virtio-net-pci should be sufficient.

Bulk update: Move RHEL-AV bugs to RHEL9. If necessary to resolve in RHEL8, then clone to the current RHEL8 release.

Hi Daniel， I have a question about this feature, could you please help to have a look?
In bug 1946719, libvirt has added the support for acpi index, and the bug was verified only on PC machine type. For q35 machine type, we can set "acpi index" on a q35 machine successfully, but it takes no effect(depend on this bug). Do you think we need to add a machine type and qemu version check in libvirt? or mention it in the document?

We did not want to restrict it to 'pc' in libvirt, because future QEMU changes are expected to make it work for 'q35' too, whereupon libvirt's check / docs would be incorrect.

(In reply to Igor Mammedov from comment #21)
> (In reply to Daniel Berrangé from comment #20)
> > (In reply to Igor Mammedov from comment #18)
> > > Q35 support is not complete,
> > > I'm working in this upstream (hopefully it will make into 7.1)
> > 
> > What's the status of ACPI Index support for Q35. AFAICT, it still doesn't
> > work with Q35 in pending 7.2 code.  Is there a pointer to the upstream code
> > posting thus far ?
> 
> Ihad impression that acpi-index should work by default on latest Q35 machine
> type
> (machine should have ACPI PCI hotplug enabled y default or explicitly for
> older versions)
> when NIC is attached to hotpluggable root-port/bridge, have you tried?

Yes, my bad, it does indeed work for a pcie-root-port attached to the root complex.

> It doesn't work for:
>   1. NICs attached to the root PCIe bus.
>          I'm working on fixing that
>   2. hotplugged root-ports/bridges (all machine types)
>          (theoretically possible but it would require run-time ACPI tables
> injection,
>           and I have doubts about Windows tolerating it)
>   3. extra host-bridges (all machine types)
>          (we would need to port ACPI PCI hotplug to it, but I don't see
> inherent issues, it's just needs time to implement if somebody asks for it)

I tried this config to demo a few different topology scenarios:

$ qemu-system-x86_64 \
    -M q35 \
    -serial stdio \
    -display none \
    -m 1000 \
    -kernel /boot/vmlinuz-5.19.11-200.fc36.x86_64 \
    -initrd ../tiny-vm-tools/tiny-initrd.img \
    -append "console=ttyS0 quiet" \
    -accel kvm \
    -device virtio-net,bus=pcie.0,acpi-index=3 \
    -device pcie-root-port,id=p1,bus=pcie.0 \
    -device virtio-net,bus=p1,acpi-index=7 \
    -device pcie-pci-bridge,id=b1 \
    -device virtio-net,bus=b1,addr=3,acpi-index=11 \
    -device pxb-pcie,id=x1,bus=pcie.0,bus_nr=253 \
    -device pcie-root-port,bus=x1,addr=2,chassis=3,id=p3 \
    -device virtio-net,bus=p3,acpi-index=17
# cat /sys/bus/pci/devices/*/acpi_index
7

So we can see only the NIC in the pcie-root-port, on the root complex got a ACPI index

(In reply to Daniel Berrangé from comment #22)
> (In reply to Igor Mammedov from comment #21)
> > (In reply to Daniel Berrangé from comment #20)
> > > (In reply to Igor Mammedov from comment #18)
> > > > Q35 support is not complete,
> > > > I'm working in this upstream (hopefully it will make into 7.1)
> > > 
> > > What's the status of ACPI Index support for Q35. AFAICT, it still doesn't
> > > work with Q35 in pending 7.2 code.  Is there a pointer to the upstream code
> > > posting thus far ?
> > 
> > Ihad impression that acpi-index should work by default on latest Q35 machine
> > type
> > (machine should have ACPI PCI hotplug enabled y default or explicitly for
> > older versions)
> > when NIC is attached to hotpluggable root-port/bridge, have you tried?
> 
> Yes, my bad, it does indeed work for a pcie-root-port attached to the root
> complex.
> 
> > It doesn't work for:
> >   1. NICs attached to the root PCIe bus.
> >          I'm working on fixing that
> >   2. hotplugged root-ports/bridges (all machine types)
> >          (theoretically possible but it would require run-time ACPI tables
> > injection,
> >           and I have doubts about Windows tolerating it)
> >   3. extra host-bridges (all machine types)
> >          (we would need to port ACPI PCI hotplug to it, but I don't see
> > inherent issues, it's just needs time to implement if somebody asks for it)
> 
> I tried this config to demo a few different topology scenarios:

Thanks!

> 
> $ qemu-system-x86_64 \
>     -M q35 \
>     -serial stdio \
>     -display none \
>     -m 1000 \
>     -kernel /boot/vmlinuz-5.19.11-200.fc36.x86_64 \
>     -initrd ../tiny-vm-tools/tiny-initrd.img \
>     -append "console=ttyS0 quiet" \
>     -accel kvm \
>     -device virtio-net,bus=pcie.0,acpi-index=3 \

not supported yet

>     -device pcie-root-port,id=p1,bus=pcie.0 \
>     -device virtio-net,bus=p1,acpi-index=7 \

>     -device pcie-pci-bridge,id=b1 \
>     -device virtio-net,bus=b1,addr=3,acpi-index=11 \

looks like a bug,
I'll check and if fix doesn't get in the way it could be merged during soft-freeze

>     -device pxb-pcie,id=x1,bus=pcie.0,bus_nr=253 \
>     -device pcie-root-port,bus=x1,addr=2,chassis=3,id=p3 \
>     -device virtio-net,bus=p3,acpi-index=17

expected, anything hanging off from pxb doesn't have ACPI description
so only native hotplug supported there.

> # cat /sys/bus/pci/devices/*/acpi_index
> 7
> 
> So we can see only the NIC in the pcie-root-port, on the root complex got a
> ACPI index

(In reply to Igor Mammedov from comment #23)
> (In reply to Daniel Berrangé from comment #22)
> > (In reply to Igor Mammedov from comment #21)
> > > (In reply to Daniel Berrangé from comment #20)
> > > > (In reply to Igor Mammedov from comment #18)
> > > > > Q35 support is not complete,
> > > > > I'm working in this upstream (hopefully it will make into 7.1)
> > > > 
> > > > What's the status of ACPI Index support for Q35. AFAICT, it still doesn't
> > > > work with Q35 in pending 7.2 code.  Is there a pointer to the upstream code
> > > > posting thus far ?
> > > 
> > > Ihad impression that acpi-index should work by default on latest Q35 machine
> > > type
> > > (machine should have ACPI PCI hotplug enabled y default or explicitly for
> > > older versions)
> > > when NIC is attached to hotpluggable root-port/bridge, have you tried?
> > 
> > Yes, my bad, it does indeed work for a pcie-root-port attached to the root
> > complex.
> > 
> > > It doesn't work for:
> > >   1. NICs attached to the root PCIe bus.
> > >          I'm working on fixing that
> > >   2. hotplugged root-ports/bridges (all machine types)
> > >          (theoretically possible but it would require run-time ACPI tables
> > > injection,
> > >           and I have doubts about Windows tolerating it)
> > >   3. extra host-bridges (all machine types)
> > >          (we would need to port ACPI PCI hotplug to it, but I don't see
> > > inherent issues, it's just needs time to implement if somebody asks for it)
> > 
> > I tried this config to demo a few different topology scenarios:
> 
> Thanks!
> 
> > 
> > $ qemu-system-x86_64 \
> >     -M q35 \
> >     -serial stdio \
> >     -display none \
> >     -m 1000 \
> >     -kernel /boot/vmlinuz-5.19.11-200.fc36.x86_64 \
> >     -initrd ../tiny-vm-tools/tiny-initrd.img \
> >     -append "console=ttyS0 quiet" \
> >     -accel kvm \
> >     -device virtio-net,bus=pcie.0,acpi-index=3 \
> 
> not supported yet
> 
> >     -device pcie-root-port,id=p1,bus=pcie.0 \
> >     -device virtio-net,bus=p1,acpi-index=7 \
> 
> >     -device pcie-pci-bridge,id=b1 \
> >     -device virtio-net,bus=b1,addr=3,acpi-index=11 \
> 
> looks like a bug,
> I'll check and if fix doesn't get in the way it could be merged during
> soft-freeze
> 
> >     -device pxb-pcie,id=x1,bus=pcie.0,bus_nr=253 \
> >     -device pcie-root-port,bus=x1,addr=2,chassis=3,id=p3 \
> >     -device virtio-net,bus=p3,acpi-index=17
> 
> expected, anything hanging off from pxb doesn't have ACPI description
> so only native hotplug supported there.
> 
> > # cat /sys/bus/pci/devices/*/acpi_index
> > 7
> > 
> > So we can see only the NIC in the pcie-root-port, on the root complex got a
> > ACPI index

Hmm, bridge works for me too with current QEMU master:
Fedora 34 (Server Edition)
Kernel 5.11.12-300.fc34.x86_64 on an x86_64 (ttyS0)
[root@fedora ~]# cat /sys/bus/pci/devices/*/acpi_index
7
11

Maybe something is broken on guest side in later versions. Will check tomorrow.

(In reply to Lei Yang from comment #29)
[...]
> Please feel free to ping if there are any testing.
[...]

There are things to test if if you'd test upstream QEMU.
current master (8.0) added support for acpi-index for
devices attached to directly to q35 hostbridge and
also for a set of non-hotpluggable multifunction devices
(pc | q35).

(In reply to Igor Mammedov from comment #30)
> (In reply to Lei Yang from comment #29)
> [...]
> > Please feel free to ping if there are any testing.
> [...]
> 
> There are things to test if if you'd test upstream QEMU.
> current master (8.0) added support for acpi-index for
> devices attached to directly to q35 hostbridge and
> also for a set of non-hotpluggable multifunction devices
> (pc | q35).

==> Scenario 1: Test on the Q35&multifunction=on

==>Test steps:
1. clone the upstream qemu
# git clone https://gitlab.com/qemu-project/qemu.git
# mkdir build
# cd build/
# ../configure --target-list=x86_64-softmmu --enable-debug
# make

2. Boot a guest with this qemu verison
/root/qemu/build/x86_64-softmmu/qemu-system-x86_64 \
-name 'vm1'  \
-blockdev '{"node-name": "file_ovmf_code", "driver": "file", "filename": "/usr/share/OVMF/OVMF_CODE.secboot.fd", "auto-read-only": true, "discard": "unmap"}' \
-blockdev '{"node-name": "drive_ovmf_code", "driver": "raw", "read-only": true, "file": "file_ovmf_code"}' \
-blockdev '{"node-name": "file_ovmf_vars", "driver": "file", "filename": "/root/avocado/data/avocado-vt/avocado-vt-vm1_rhel930-64-virtio-scsi_qcow2_filesystem_VARS.fd", "auto-read-only": true, "discard": "unmap"}' \
-blockdev '{"node-name": "drive_ovmf_vars", "driver": "raw", "read-only": false, "file": "file_ovmf_vars"}' \
-machine q35,memory-backend=mem-machine_mem,pflash0=drive_ovmf_code,pflash1=drive_ovmf_vars \
-device '{"id": "pcie-root-port-0", "driver": "pcie-root-port", "multifunction": true, "bus": "pcie.0", "addr": "0x1", "chassis": 1}' \
-device '{"id": "pcie-pci-bridge-0", "driver": "pcie-pci-bridge", "addr": "0x0", "bus": "pcie-root-port-0"}'  \
-nodefaults \
-device '{"driver": "VGA", "bus": "pcie.0", "addr": "0x2"}' \
-m 62464 \
-object '{"size": 65498251264, "id": "mem-machine_mem", "qom-type": "memory-backend-ram"}'  \
-smp 24,maxcpus=24,cores=12,threads=1,dies=1,sockets=2  \
-cpu 'Icelake-Server-noTSX',+kvm_pv_unhalt \
-device '{"id": "pcie-root-port-1", "port": 1, "driver": "pcie-root-port", "addr": "0x1.0x1", "bus": "pcie.0", "chassis": 2}' \
-device '{"driver": "qemu-xhci", "id": "usb1", "bus": "pcie-root-port-1", "addr": "0x0"}' \
-device '{"driver": "usb-tablet", "id": "usb-tablet1", "bus": "usb1.0", "port": "1"}' \
-device '{"id": "pcie-root-port-2", "port": 2, "driver": "pcie-root-port", "addr": "0x1.0x2", "bus": "pcie.0", "chassis": 3}' \
-device '{"id": "virtio_scsi_pci0", "driver": "virtio-scsi-pci", "bus": "pcie-root-port-2", "addr": "0x0"}' \
-blockdev '{"node-name": "file_image1", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "threads", "filename": "/home/kvm_autotest_root/images/rhel930-64-virtio-scsi.qcow2", "cache": {"direct": true, "no-flush": false}}' \
-blockdev '{"node-name": "drive_image1", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_image1"}' \
-device '{"driver": "scsi-hd", "id": "image1", "drive": "drive_image1", "write-cache": "on"}' \
-device '{"id": "pcie-root-port-3", "port": 3, "driver": "pcie-root-port", "addr": "0x1.0x3", "bus": "pcie.0", "chassis": 4}' \
-device '{"driver": "virtio-net-pci", "mac": "9a:03:a2:11:64:35", "id": "net0", "netdev": "hostnet0", "acpi-index": 1, "bus": "pcie-root-port-3", "addr": "0x0"}'  \
-netdev tap,id=hostnet0,vhost=on,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown \
-vnc :0  \
-rtc base=utc,clock=host,driftfix=slew  \
-boot menu=off,order=cdn,once=c,strict=off \
-enable-kvm \
-device '{"id": "pcie_extra_root_port_0", "driver": "pcie-root-port", "multifunction": true, "bus": "pcie.0", "addr": "0x3", "chassis": 5}' \
-monitor stdio \
-qmp tcp:0:5555,server,nowait \

3.After guest boot up and check the nic name. "enoN"=="acpi-index=N", this is expect result.

4. hotplug/unplug it via qmp, then reboot it there is no any error messages in the host&guest.
{"return": {}}
{"timestamp": {"seconds": 1681297140, "microseconds": 893488}, "event": "DEVICE_DELETED", "data": {"path": "/machine/peripheral/net0/virtio-backend"}}
{"timestamp": {"seconds": 1681297140, "microseconds": 944221}, "event": "DEVICE_DELETED", "data": {"device": "net0", "path": "/machine/peripheral/net0"}}
{"execute": "netdev_del", "arguments": {"id": "hostnet0"}}
{"return": {}}
{"timestamp": {"seconds": 1681297152, "microseconds": 431097}, "event": "RESET", "data": {"guest": true, "reason": "guest-reset"}}
{"timestamp": {"seconds": 1681297153, "microseconds": 747611}, "event": "RTC_CHANGE", "data": {"offset": 0, "qom-path": "/machine/unattached/device[32]"}}
{"timestamp": {"seconds": 1681297154, "microseconds": 35473}, "event": "RTC_CHANGE", "data": {"offset": 0, "qom-path": "/machine/unattached/device[32]"}}
{"execute": "netdev_add", "arguments": { "type":"tap","id":"hostnet0","vhost": true,"script":"/etc/qemu-ifup","downscript":"/etc/qemu-ifdown"}}
{"return": {}}
{"execute":"device_add","arguments":{"driver":"virtio-net-pci","netdev":"hostnet0","mac":"00:11:22:33:44:00","id":"net0","acpi-index":"1","bus":"pcie-root-port-3","addr","0x0"}}
{"return": {}}
{"timestamp": {"seconds": 1681297217, "microseconds": 813367}, "event": "NIC_RX_FILTER_CHANGED", "data": {"name": "net0", "path": "/machine/peripheral/net0/virtio-backend"}}


==> Scenario 2: Test on the Q35&devices attached to directly to q35 hostbridge
1. Compare to the scenario 1, only the qemu CLI are different
/root/qemu/build/x86_64-softmmu/qemu-system-x86_64 \
-name 'vm1'  \
-blockdev '{"node-name": "file_ovmf_code", "driver": "file", "filename": "/usr/share/OVMF/OVMF_CODE.secboot.fd", "auto-read-only": true, "discard": "unmap"}' \
-blockdev '{"node-name": "drive_ovmf_code", "driver": "raw", "read-only": true, "file": "file_ovmf_code"}' \
-blockdev '{"node-name": "file_ovmf_vars", "driver": "file", "filename": "/root/avocado/data/avocado-vt/avocado-vt-vm1_rhel930-64-virtio-scsi_qcow2_filesystem_VARS.fd", "auto-read-only": true, "discard": "unmap"}' \
-blockdev '{"node-name": "drive_ovmf_vars", "driver": "raw", "read-only": false, "file": "file_ovmf_vars"}' \
-machine q35,memory-backend=mem-machine_mem,pflash0=drive_ovmf_code,pflash1=drive_ovmf_vars \
-device '{"id": "pcie-root-port-0", "driver": "pcie-root-port", "bus": "pcie.0", "addr": "0x1", "chassis": 1}' \
-nodefaults \
-device '{"driver": "VGA", "bus": "pcie.0", "addr": "0x2"}' \
-m 62464 \
-object '{"size": 65498251264, "id": "mem-machine_mem", "qom-type": "memory-backend-ram"}'  \
-smp 24,maxcpus=24,cores=12,threads=1,dies=1,sockets=2  \
-cpu 'Icelake-Server-noTSX',+kvm_pv_unhalt \
-device '{"id": "pcie-root-port-3", "driver": "pcie-root-port", "addr": "0x3", "chassis": 3}' \
-device '{"id": "virtio_scsi_pci0", "driver": "virtio-scsi-pci", "bus": "pcie-root-port-3", "addr": "0x0"}' \
-blockdev '{"node-name": "file_image1", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "threads", "filename": "/home/kvm_autotest_root/images/rhel930-64-virtio-scsi.qcow2", "cache": {"direct": true, "no-flush": false}}' \
-blockdev '{"node-name": "drive_image1", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_image1"}' \
-device '{"driver": "scsi-hd", "id": "image1", "drive": "drive_image1", "write-cache": "on"}' \
-device '{"id": "pcie-root-port-4", "driver": "pcie-root-port", "addr": "0x4", "chassis": 4}' \
-device '{"driver": "virtio-net-pci", "mac": "9a:03:a2:11:64:35", "id": "net0", "netdev": "hostnet0", "acpi-index": 1, "bus": "pcie-root-port-4", "addr": "0x0"}'  \
-netdev tap,id=hostnet0,vhost=on,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown \
-vnc :0  \
-rtc base=utc,clock=host,driftfix=slew  \
-boot menu=off,order=cdn,once=c,strict=off \
-enable-kvm \
-monitor stdio \
-qmp tcp:0:5555,server,nowait \

2. Test the same cases with scenarios 1 and got the same result

Based on the above test result, it should support well on the upstream. Please correct me if I'm wrong.

For pc mode，I will update the test results tomorrow.

==> Scenario 1: Test on the PC

==>Test steps
1. Boot a guest
/root/qemu/build/x86_64-softmmu/qemu-system-x86_64 \
-name 'vm1'  \
-machine pc,memory-backend=mem-machine_mem  \
-nodefaults \
-device '{"driver": "VGA", "bus": "pci.0", "addr": "0x2"}' \
-m 62464 \
-object '{"size": 65498251264, "id": "mem-machine_mem", "qom-type": "memory-backend-ram"}'  \
-smp 24,maxcpus=24,cores=12,threads=1,dies=1,sockets=2  \
-cpu 'Icelake-Server',ds=on,ss=on,dtes64=on,vmx=on,pdcm=on,hypervisor=on,tsc-adjust=on,avx512ifma=on,sha-ni=on,rdpid=on,fsrm=on,md-clear=on,stibp=on,arch-capabilities=on,xsaves=on,ibpb=on,ibrs=on,amd-stibp=on,amd-ssbd=on,rdctl-no=on,ibrs-all=on,skip-l1dfl-vmentry=on,mds-no=on,pschange-mc-no=on,tsx-ctrl=on,hle=off,rtm=off,mpx=off,intel-pt=off,kvm_pv_unhalt=on \
-device '{"driver": "ich9-usb-ehci1", "id": "usb1", "addr": "0x1d.0x7", "multifunction": true, "bus": "pci.0"}' \
-device '{"driver": "ich9-usb-uhci1", "id": "usb1.0", "multifunction": true, "masterbus": "usb1.0", "addr": "0x1d.0x0", "firstport": 0, "bus": "pci.0"}' \
-device '{"driver": "ich9-usb-uhci2", "id": "usb1.1", "multifunction": true, "masterbus": "usb1.0", "addr": "0x1d.0x2", "firstport": 2, "bus": "pci.0"}' \
-device '{"driver": "ich9-usb-uhci3", "id": "usb1.2", "multifunction": true, "masterbus": "usb1.0", "addr": "0x1d.0x4", "firstport": 4, "bus": "pci.0"}' \
-device '{"driver": "usb-tablet", "id": "usb-tablet1", "bus": "usb1.0", "port": "1"}' \
-device '{"id": "virtio_scsi_pci0", "driver": "virtio-scsi-pci", "bus": "pci.0", "addr": "0x3"}' \
-blockdev '{"node-name": "file_image1", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "threads", "filename": "/home/kvm_autotest_root/images/rhel930-64-virtio-scsi.qcow2", "cache": {"direct": true, "no-flush": false}}' \
-blockdev '{"node-name": "drive_image1", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_image1"}' \
-device '{"driver": "scsi-hd", "id": "image1", "drive": "drive_image1", "write-cache": "on"}' \
-device '{"driver": "virtio-net-pci", "mac": "9a:56:74:43:29:a0", "acpi-index": 1, "id": "net0", "netdev": "hostnet0", "bus": "pci.0", "addr": "0x4"}'  \
-netdev tap,id=hostnet0,vhost=on,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown \
-vnc :0  \
-rtc base=utc,clock=host,driftfix=slew  \
-boot menu=off,order=cdn,once=c,strict=off \
-enable-kvm \
-monitor stdio \
-qmp tcp:0:5555,server,nowait \

2. After guest boot up and check the nic name. "enoN"=="acpi-index=N", this is expect result.

3. hotplug/unplug it via qmp, then reboot it there is no any error messages in the host&guest.
{"execute": "device_del", "arguments": {"id": "net0"}}	
{"return": {}}
{"timestamp": {"seconds": 1681362763, "microseconds": 503548}, "event": "DEVICE_DELETED", "data": {"path": "/machine/peripheral/net0/virtio-backend"}}
{"timestamp": {"seconds": 1681362763, "microseconds": 554157}, "event": "DEVICE_DELETED", "data": {"device": "net0", "path": "/machine/peripheral/net0"}}
{"execute": "netdev_del", "arguments": {"id": "hostnet0"}}
{"return": {}}
{"timestamp": {"seconds": 1681362783, "microseconds": 778654}, "event": "RESET", "data": {"guest": true, "reason": "guest-reset"}}
{"timestamp": {"seconds": 1681362783, "microseconds": 785609}, "event": "RESET", "data": {"guest": true, "reason": "guest-reset"}}
{"execute": "netdev_add", "arguments": { "type":"tap","id":"hostnet0","vhost": true,"script":"/etc/qemu-ifup","downscript":"/etc/qemu-ifdown"}}
{"return": {}}
{"execute":"device_add","arguments":{"driver":"virtio-net-pci","netdev":"hostnet0","mac":"00:11:22:33:44:00","id":"net0","acpi-index":"1","bus":"pci.0","addr":"0x4"}}
{"return": {}}
{"timestamp": {"seconds": 1681362851, "microseconds": 489285}, "event": "NIC_RX_FILTER_CHANGED", "data": {"name": "net0", "path": "/machine/peripheral/net0/virtio-backend"}}
{"timestamp": {"seconds": 1681362893, "microseconds": 912085}, "event": "RESET", "data": {"guest": true, "reason": "guest-reset"}}
{"timestamp": {"seconds": 1681362893, "microseconds": 919163}, "event": "RESET", "data": {"guest": true, "reason": "guest-reset"}}

(In reply to Lei Yang from comment #31)
> (In reply to Igor Mammedov from comment #30)
> > (In reply to Lei Yang from comment #29)
> > [...]
> > > Please feel free to ping if there are any testing.
> > [...]
> > 
> > There are things to test if if you'd test upstream QEMU.
> > current master (8.0) added support for acpi-index for
> > devices attached to directly to q35 hostbridge and
> > also for a set of non-hotpluggable multifunction devices
> > (pc | q35).
> 
> ==> Scenario 1: Test on the Q35&multifunction=on
> 
> ==>Test steps:
> 1. clone the upstream qemu
> # git clone https://gitlab.com/qemu-project/qemu.git
> # mkdir build
> # cd build/
> # ../configure --target-list=x86_64-softmmu --enable-debug
> # make
> 
> 2. Boot a guest with this qemu verison
> /root/qemu/build/x86_64-softmmu/qemu-system-x86_64 \
> -name 'vm1'  \
> -blockdev '{"node-name": "file_ovmf_code", "driver": "file", "filename":
> "/usr/share/OVMF/OVMF_CODE.secboot.fd", "auto-read-only": true, "discard":
> "unmap"}' \
> -blockdev '{"node-name": "drive_ovmf_code", "driver": "raw", "read-only":
> true, "file": "file_ovmf_code"}' \
> -blockdev '{"node-name": "file_ovmf_vars", "driver": "file", "filename":
> "/root/avocado/data/avocado-vt/avocado-vt-vm1_rhel930-64-virtio-
> scsi_qcow2_filesystem_VARS.fd", "auto-read-only": true, "discard": "unmap"}'
> \
> -blockdev '{"node-name": "drive_ovmf_vars", "driver": "raw", "read-only":
> false, "file": "file_ovmf_vars"}' \
> -machine
> q35,memory-backend=mem-machine_mem,pflash0=drive_ovmf_code,
> pflash1=drive_ovmf_vars \
> -device '{"id": "pcie-root-port-0", "driver": "pcie-root-port",
> "multifunction": true, "bus": "pcie.0", "addr": "0x1", "chassis": 1}' \
> -device '{"id": "pcie-pci-bridge-0", "driver": "pcie-pci-bridge", "addr":
> "0x0", "bus": "pcie-root-port-0"}'  \
> -nodefaults \
> -device '{"driver": "VGA", "bus": "pcie.0", "addr": "0x2"}' \
> -m 62464 \
> -object '{"size": 65498251264, "id": "mem-machine_mem", "qom-type":
> "memory-backend-ram"}'  \
> -smp 24,maxcpus=24,cores=12,threads=1,dies=1,sockets=2  \
> -cpu 'Icelake-Server-noTSX',+kvm_pv_unhalt \
> -device '{"id": "pcie-root-port-1", "port": 1, "driver": "pcie-root-port",
> "addr": "0x1.0x1", "bus": "pcie.0", "chassis": 2}' \
> -device '{"driver": "qemu-xhci", "id": "usb1", "bus": "pcie-root-port-1",
> "addr": "0x0"}' \
> -device '{"driver": "usb-tablet", "id": "usb-tablet1", "bus": "usb1.0",
> "port": "1"}' \
> -device '{"id": "pcie-root-port-2", "port": 2, "driver": "pcie-root-port",
> "addr": "0x1.0x2", "bus": "pcie.0", "chassis": 3}' \
> -device '{"id": "virtio_scsi_pci0", "driver": "virtio-scsi-pci", "bus":
> "pcie-root-port-2", "addr": "0x0"}' \
> -blockdev '{"node-name": "file_image1", "driver": "file", "auto-read-only":
> true, "discard": "unmap", "aio": "threads", "filename":
> "/home/kvm_autotest_root/images/rhel930-64-virtio-scsi.qcow2", "cache":
> {"direct": true, "no-flush": false}}' \
> -blockdev '{"node-name": "drive_image1", "driver": "qcow2", "read-only":
> false, "cache": {"direct": true, "no-flush": false}, "file": "file_image1"}'
> \
> -device '{"driver": "scsi-hd", "id": "image1", "drive": "drive_image1",
> "write-cache": "on"}' \
> -device '{"id": "pcie-root-port-3", "port": 3, "driver": "pcie-root-port",
> "addr": "0x1.0x3", "bus": "pcie.0", "chassis": 4}' \

> -device '{"driver": "virtio-net-pci", "mac": "9a:03:a2:11:64:35", "id":
> "net0", "netdev": "hostnet0", "acpi-index": 1, "bus": "pcie-root-port-3",
> "addr": "0x0"}'  \

ACK

> -netdev
> tap,id=hostnet0,vhost=on,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown \
> -vnc :0  \
> -rtc base=utc,clock=host,driftfix=slew  \
> -boot menu=off,order=cdn,once=c,strict=off \
> -enable-kvm \
> -device '{"id": "pcie_extra_root_port_0", "driver": "pcie-root-port",
> "multifunction": true, "bus": "pcie.0", "addr": "0x3", "chassis": 5}' \
> -monitor stdio \
> -qmp tcp:0:5555,server,nowait \
>
> 3.After guest boot up and check the nic name. "enoN"=="acpi-index=N", this
> is expect result.
 
> 4. hotplug/unplug it via qmp, then reboot it there is no any error messages
> in the host&guest.
> {"return": {}}
> {"timestamp": {"seconds": 1681297140, "microseconds": 893488}, "event":
> "DEVICE_DELETED", "data": {"path":
> "/machine/peripheral/net0/virtio-backend"}}
> {"timestamp": {"seconds": 1681297140, "microseconds": 944221}, "event":
> "DEVICE_DELETED", "data": {"device": "net0", "path":
> "/machine/peripheral/net0"}}
> {"execute": "netdev_del", "arguments": {"id": "hostnet0"}}
> {"return": {}}
> {"timestamp": {"seconds": 1681297152, "microseconds": 431097}, "event":
> "RESET", "data": {"guest": true, "reason": "guest-reset"}}
> {"timestamp": {"seconds": 1681297153, "microseconds": 747611}, "event":
> "RTC_CHANGE", "data": {"offset": 0, "qom-path":
> "/machine/unattached/device[32]"}}
> {"timestamp": {"seconds": 1681297154, "microseconds": 35473}, "event":
> "RTC_CHANGE", "data": {"offset": 0, "qom-path":
> "/machine/unattached/device[32]"}}
> {"execute": "netdev_add", "arguments": {
> "type":"tap","id":"hostnet0","vhost":
> true,"script":"/etc/qemu-ifup","downscript":"/etc/qemu-ifdown"}}
> {"return": {}}
> {"execute":"device_add","arguments":{"driver":"virtio-net-pci","netdev":
> "hostnet0","mac":"00:11:22:33:44:00","id":"net0","acpi-index":"1","bus":
> "pcie-root-port-3","addr","0x0"}}

ack

> {"return": {}}
> {"timestamp": {"seconds": 1681297217, "microseconds": 813367}, "event":
> "NIC_RX_FILTER_CHANGED", "data": {"name": "net0", "path":
> "/machine/peripheral/net0/virtio-backend"}}
> 
> 
> ==> Scenario 2: Test on the Q35&devices attached to directly to q35
> hostbridge
> 1. Compare to the scenario 1, only the qemu CLI are different
> /root/qemu/build/x86_64-softmmu/qemu-system-x86_64 \
> -name 'vm1'  \
> -blockdev '{"node-name": "file_ovmf_code", "driver": "file", "filename":
> "/usr/share/OVMF/OVMF_CODE.secboot.fd", "auto-read-only": true, "discard":
> "unmap"}' \
> -blockdev '{"node-name": "drive_ovmf_code", "driver": "raw", "read-only":
> true, "file": "file_ovmf_code"}' \
> -blockdev '{"node-name": "file_ovmf_vars", "driver": "file", "filename":
> "/root/avocado/data/avocado-vt/avocado-vt-vm1_rhel930-64-virtio-
> scsi_qcow2_filesystem_VARS.fd", "auto-read-only": true, "discard": "unmap"}'
> \
> -blockdev '{"node-name": "drive_ovmf_vars", "driver": "raw", "read-only":
> false, "file": "file_ovmf_vars"}' \
> -machine
> q35,memory-backend=mem-machine_mem,pflash0=drive_ovmf_code,
> pflash1=drive_ovmf_vars \
> -device '{"id": "pcie-root-port-0", "driver": "pcie-root-port", "bus":
> "pcie.0", "addr": "0x1", "chassis": 1}' \
> -nodefaults \
> -device '{"driver": "VGA", "bus": "pcie.0", "addr": "0x2"}' \
> -m 62464 \
> -object '{"size": 65498251264, "id": "mem-machine_mem", "qom-type":
> "memory-backend-ram"}'  \
> -smp 24,maxcpus=24,cores=12,threads=1,dies=1,sockets=2  \
> -cpu 'Icelake-Server-noTSX',+kvm_pv_unhalt \
> -device '{"id": "pcie-root-port-3", "driver": "pcie-root-port", "addr":
> "0x3", "chassis": 3}' \
> -device '{"id": "virtio_scsi_pci0", "driver": "virtio-scsi-pci", "bus":
> "pcie-root-port-3", "addr": "0x0"}' \
> -blockdev '{"node-name": "file_image1", "driver": "file", "auto-read-only":
> true, "discard": "unmap", "aio": "threads", "filename":
> "/home/kvm_autotest_root/images/rhel930-64-virtio-scsi.qcow2", "cache":
> {"direct": true, "no-flush": false}}' \
> -blockdev '{"node-name": "drive_image1", "driver": "qcow2", "read-only":
> false, "cache": {"direct": true, "no-flush": false}, "file": "file_image1"}'
> \
> -device '{"driver": "scsi-hd", "id": "image1", "drive": "drive_image1",
> "write-cache": "on"}' \
> -device '{"id": "pcie-root-port-4", "driver": "pcie-root-port", "addr":
> "0x4", "chassis": 4}' \
> -device '{"driver": "virtio-net-pci", "mac": "9a:03:a2:11:64:35", "id":
> "net0", "netdev": "hostnet0", "acpi-index": 1, "bus": "pcie-root-port-4",
> "addr": "0x0"}'  \

this is attached to root port, should be attached to hostbridge (bus=pcie.0)

> -netdev
> tap,id=hostnet0,vhost=on,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown \
> -vnc :0  \
> -rtc base=utc,clock=host,driftfix=slew  \
> -boot menu=off,order=cdn,once=c,strict=off \
> -enable-kvm \
> -monitor stdio \
> -qmp tcp:0:5555,server,nowait \
> 
> 2. Test the same cases with scenarios 1 and got the same result
> 
> Based on the above test result, it should support well on the upstream.
> Please correct me if I'm wrong.
> 
> For pc mode，I will update the test results tomorrow.

(In reply to Lei Yang from comment #32)
> ==> Scenario 1: Test on the PC
> 
> ==>Test steps
> 1. Boot a guest
> /root/qemu/build/x86_64-softmmu/qemu-system-x86_64 \
> -name 'vm1'  \
> -machine pc,memory-backend=mem-machine_mem  \
> -nodefaults \
> -device '{"driver": "VGA", "bus": "pci.0", "addr": "0x2"}' \
> -m 62464 \
> -object '{"size": 65498251264, "id": "mem-machine_mem", "qom-type":
> "memory-backend-ram"}'  \
> -smp 24,maxcpus=24,cores=12,threads=1,dies=1,sockets=2  \
> -cpu
> 'Icelake-Server',ds=on,ss=on,dtes64=on,vmx=on,pdcm=on,hypervisor=on,tsc-
> adjust=on,avx512ifma=on,sha-ni=on,rdpid=on,fsrm=on,md-clear=on,stibp=on,arch-
> capabilities=on,xsaves=on,ibpb=on,ibrs=on,amd-stibp=on,amd-ssbd=on,rdctl-
> no=on,ibrs-all=on,skip-l1dfl-vmentry=on,mds-no=on,pschange-mc-no=on,tsx-
> ctrl=on,hle=off,rtm=off,mpx=off,intel-pt=off,kvm_pv_unhalt=on \
> -device '{"driver": "ich9-usb-ehci1", "id": "usb1", "addr": "0x1d.0x7",
> "multifunction": true, "bus": "pci.0"}' \
> -device '{"driver": "ich9-usb-uhci1", "id": "usb1.0", "multifunction": true,
> "masterbus": "usb1.0", "addr": "0x1d.0x0", "firstport": 0, "bus": "pci.0"}' \
> -device '{"driver": "ich9-usb-uhci2", "id": "usb1.1", "multifunction": true,
> "masterbus": "usb1.0", "addr": "0x1d.0x2", "firstport": 2, "bus": "pci.0"}' \
> -device '{"driver": "ich9-usb-uhci3", "id": "usb1.2", "multifunction": true,
> "masterbus": "usb1.0", "addr": "0x1d.0x4", "firstport": 4, "bus": "pci.0"}' \
> -device '{"driver": "usb-tablet", "id": "usb-tablet1", "bus": "usb1.0",
> "port": "1"}' \
> -device '{"id": "virtio_scsi_pci0", "driver": "virtio-scsi-pci", "bus":
> "pci.0", "addr": "0x3"}' \
> -blockdev '{"node-name": "file_image1", "driver": "file", "auto-read-only":
> true, "discard": "unmap", "aio": "threads", "filename":
> "/home/kvm_autotest_root/images/rhel930-64-virtio-scsi.qcow2", "cache":
> {"direct": true, "no-flush": false}}' \
> -blockdev '{"node-name": "drive_image1", "driver": "qcow2", "read-only":
> false, "cache": {"direct": true, "no-flush": false}, "file": "file_image1"}'
> \
> -device '{"driver": "scsi-hd", "id": "image1", "drive": "drive_image1",
> "write-cache": "on"}' \

> -device '{"driver": "virtio-net-pci", "mac": "9a:56:74:43:29:a0",
> "acpi-index": 1, "id": "net0", "netdev": "hostnet0", "bus": "pci.0", "addr":
> "0x4"}'  \

this worked before new patches (i.e. on PC acpi hotplug is enabled on both hostbridge
and coldplugged bridges by default).

> -netdev
> tap,id=hostnet0,vhost=on,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown \
> -vnc :0  \
> -rtc base=utc,clock=host,driftfix=slew  \
> -boot menu=off,order=cdn,once=c,strict=off \
> -enable-kvm \
> -monitor stdio \
> -qmp tcp:0:5555,server,nowait \
> 
> 2. After guest boot up and check the nic name. "enoN"=="acpi-index=N", this
> is expect result.

> 3. hotplug/unplug it via qmp, then reboot it there is no any error messages
> in the host&guest.
> {"execute": "device_del", "arguments": {"id": "net0"}}	
> {"return": {}}
> {"timestamp": {"seconds": 1681362763, "microseconds": 503548}, "event":
> "DEVICE_DELETED", "data": {"path":
> "/machine/peripheral/net0/virtio-backend"}}
> {"timestamp": {"seconds": 1681362763, "microseconds": 554157}, "event":
> "DEVICE_DELETED", "data": {"device": "net0", "path":
> "/machine/peripheral/net0"}}
> {"execute": "netdev_del", "arguments": {"id": "hostnet0"}}
> {"return": {}}
> {"timestamp": {"seconds": 1681362783, "microseconds": 778654}, "event":
> "RESET", "data": {"guest": true, "reason": "guest-reset"}}
> {"timestamp": {"seconds": 1681362783, "microseconds": 785609}, "event":
> "RESET", "data": {"guest": true, "reason": "guest-reset"}}
> {"execute": "netdev_add", "arguments": {
> "type":"tap","id":"hostnet0","vhost":
> true,"script":"/etc/qemu-ifup","downscript":"/etc/qemu-ifdown"}}
> {"return": {}}
> {"execute":"device_add","arguments":{"driver":"virtio-net-pci","netdev":
> "hostnet0","mac":"00:11:22:33:44:00","id":"net0","acpi-index":"1","bus":"pci.
> 0","addr":"0x4"}}

ditto

> {"return": {}}
> {"timestamp": {"seconds": 1681362851, "microseconds": 489285}, "event":
> "NIC_RX_FILTER_CHANGED", "data": {"name": "net0", "path":
> "/machine/peripheral/net0/virtio-backend"}}
> {"timestamp": {"seconds": 1681362893, "microseconds": 912085}, "event":
> "RESET", "data": {"guest": true, "reason": "guest-reset"}}
> {"timestamp": {"seconds": 1681362893, "microseconds": 919163}, "event":
> "RESET", "data": {"guest": true, "reason": "guest-reset"}}

what you haven't tried is disabling ACPI hotplug on hostbridge and/or bridges
and then checking if acpi-index still works there.

perhaps QEMU change log will give you more ideas about what to test:
https://wiki.qemu.org/ChangeLog/8.0#x86

(In reply to Igor Mammedov from comment #33)
> (In reply to Lei Yang from comment #31)
> > (In reply to Igor Mammedov from comment #30)
> > > (In reply to Lei Yang from comment #29)
> > > [...]
> > > > Please feel free to ping if there are any testing.
> > > [...]
> > > 
> > > There are things to test if if you'd test upstream QEMU.
> > > current master (8.0) added support for acpi-index for
> > > devices attached to directly to q35 hostbridge and
> > > also for a set of non-hotpluggable multifunction devices
> > > (pc | q35).
> > 
> > ==> Scenario 1: Test on the Q35&multifunction=on
> > 
> > ==>Test steps:
> > 1. clone the upstream qemu
> > # git clone https://gitlab.com/qemu-project/qemu.git
> > # mkdir build
> > # cd build/
> > # ../configure --target-list=x86_64-softmmu --enable-debug
> > # make
> > 
> > 2. Boot a guest with this qemu verison
> > /root/qemu/build/x86_64-softmmu/qemu-system-x86_64 \
> > -name 'vm1'  \
> > -blockdev '{"node-name": "file_ovmf_code", "driver": "file", "filename":
> > "/usr/share/OVMF/OVMF_CODE.secboot.fd", "auto-read-only": true, "discard":
> > "unmap"}' \
> > -blockdev '{"node-name": "drive_ovmf_code", "driver": "raw", "read-only":
> > true, "file": "file_ovmf_code"}' \
> > -blockdev '{"node-name": "file_ovmf_vars", "driver": "file", "filename":
> > "/root/avocado/data/avocado-vt/avocado-vt-vm1_rhel930-64-virtio-
> > scsi_qcow2_filesystem_VARS.fd", "auto-read-only": true, "discard": "unmap"}'
> > \
> > -blockdev '{"node-name": "drive_ovmf_vars", "driver": "raw", "read-only":
> > false, "file": "file_ovmf_vars"}' \
> > -machine
> > q35,memory-backend=mem-machine_mem,pflash0=drive_ovmf_code,
> > pflash1=drive_ovmf_vars \
> > -device '{"id": "pcie-root-port-0", "driver": "pcie-root-port",
> > "multifunction": true, "bus": "pcie.0", "addr": "0x1", "chassis": 1}' \
> > -device '{"id": "pcie-pci-bridge-0", "driver": "pcie-pci-bridge", "addr":
> > "0x0", "bus": "pcie-root-port-0"}'  \
> > -nodefaults \
> > -device '{"driver": "VGA", "bus": "pcie.0", "addr": "0x2"}' \
> > -m 62464 \
> > -object '{"size": 65498251264, "id": "mem-machine_mem", "qom-type":
> > "memory-backend-ram"}'  \
> > -smp 24,maxcpus=24,cores=12,threads=1,dies=1,sockets=2  \
> > -cpu 'Icelake-Server-noTSX',+kvm_pv_unhalt \
> > -device '{"id": "pcie-root-port-1", "port": 1, "driver": "pcie-root-port",
> > "addr": "0x1.0x1", "bus": "pcie.0", "chassis": 2}' \
> > -device '{"driver": "qemu-xhci", "id": "usb1", "bus": "pcie-root-port-1",
> > "addr": "0x0"}' \
> > -device '{"driver": "usb-tablet", "id": "usb-tablet1", "bus": "usb1.0",
> > "port": "1"}' \
> > -device '{"id": "pcie-root-port-2", "port": 2, "driver": "pcie-root-port",
> > "addr": "0x1.0x2", "bus": "pcie.0", "chassis": 3}' \
> > -device '{"id": "virtio_scsi_pci0", "driver": "virtio-scsi-pci", "bus":
> > "pcie-root-port-2", "addr": "0x0"}' \
> > -blockdev '{"node-name": "file_image1", "driver": "file", "auto-read-only":
> > true, "discard": "unmap", "aio": "threads", "filename":
> > "/home/kvm_autotest_root/images/rhel930-64-virtio-scsi.qcow2", "cache":
> > {"direct": true, "no-flush": false}}' \
> > -blockdev '{"node-name": "drive_image1", "driver": "qcow2", "read-only":
> > false, "cache": {"direct": true, "no-flush": false}, "file": "file_image1"}'
> > \
> > -device '{"driver": "scsi-hd", "id": "image1", "drive": "drive_image1",
> > "write-cache": "on"}' \
> > -device '{"id": "pcie-root-port-3", "port": 3, "driver": "pcie-root-port",
> > "addr": "0x1.0x3", "bus": "pcie.0", "chassis": 4}' \
> 
> > -device '{"driver": "virtio-net-pci", "mac": "9a:03:a2:11:64:35", "id":
> > "net0", "netdev": "hostnet0", "acpi-index": 1, "bus": "pcie-root-port-3",
> > "addr": "0x0"}'  \
> 
> ACK
> 
> > -netdev
> > tap,id=hostnet0,vhost=on,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown \
> > -vnc :0  \
> > -rtc base=utc,clock=host,driftfix=slew  \
> > -boot menu=off,order=cdn,once=c,strict=off \
> > -enable-kvm \
> > -device '{"id": "pcie_extra_root_port_0", "driver": "pcie-root-port",
> > "multifunction": true, "bus": "pcie.0", "addr": "0x3", "chassis": 5}' \
> > -monitor stdio \
> > -qmp tcp:0:5555,server,nowait \
> >
> > 3.After guest boot up and check the nic name. "enoN"=="acpi-index=N", this
> > is expect result.
>  
> > 4. hotplug/unplug it via qmp, then reboot it there is no any error messages
> > in the host&guest.
> > {"return": {}}
> > {"timestamp": {"seconds": 1681297140, "microseconds": 893488}, "event":
> > "DEVICE_DELETED", "data": {"path":
> > "/machine/peripheral/net0/virtio-backend"}}
> > {"timestamp": {"seconds": 1681297140, "microseconds": 944221}, "event":
> > "DEVICE_DELETED", "data": {"device": "net0", "path":
> > "/machine/peripheral/net0"}}
> > {"execute": "netdev_del", "arguments": {"id": "hostnet0"}}
> > {"return": {}}
> > {"timestamp": {"seconds": 1681297152, "microseconds": 431097}, "event":
> > "RESET", "data": {"guest": true, "reason": "guest-reset"}}
> > {"timestamp": {"seconds": 1681297153, "microseconds": 747611}, "event":
> > "RTC_CHANGE", "data": {"offset": 0, "qom-path":
> > "/machine/unattached/device[32]"}}
> > {"timestamp": {"seconds": 1681297154, "microseconds": 35473}, "event":
> > "RTC_CHANGE", "data": {"offset": 0, "qom-path":
> > "/machine/unattached/device[32]"}}
> > {"execute": "netdev_add", "arguments": {
> > "type":"tap","id":"hostnet0","vhost":
> > true,"script":"/etc/qemu-ifup","downscript":"/etc/qemu-ifdown"}}
> > {"return": {}}
> > {"execute":"device_add","arguments":{"driver":"virtio-net-pci","netdev":
> > "hostnet0","mac":"00:11:22:33:44:00","id":"net0","acpi-index":"1","bus":
> > "pcie-root-port-3","addr","0x0"}}
> 
> ack
> 
> > {"return": {}}
> > {"timestamp": {"seconds": 1681297217, "microseconds": 813367}, "event":
> > "NIC_RX_FILTER_CHANGED", "data": {"name": "net0", "path":
> > "/machine/peripheral/net0/virtio-backend"}}
> > 
> > 
> > ==> Scenario 2: Test on the Q35&devices attached to directly to q35
> > hostbridge
> > 1. Compare to the scenario 1, only the qemu CLI are different
> > /root/qemu/build/x86_64-softmmu/qemu-system-x86_64 \
> > -name 'vm1'  \
> > -blockdev '{"node-name": "file_ovmf_code", "driver": "file", "filename":
> > "/usr/share/OVMF/OVMF_CODE.secboot.fd", "auto-read-only": true, "discard":
> > "unmap"}' \
> > -blockdev '{"node-name": "drive_ovmf_code", "driver": "raw", "read-only":
> > true, "file": "file_ovmf_code"}' \
> > -blockdev '{"node-name": "file_ovmf_vars", "driver": "file", "filename":
> > "/root/avocado/data/avocado-vt/avocado-vt-vm1_rhel930-64-virtio-
> > scsi_qcow2_filesystem_VARS.fd", "auto-read-only": true, "discard": "unmap"}'
> > \
> > -blockdev '{"node-name": "drive_ovmf_vars", "driver": "raw", "read-only":
> > false, "file": "file_ovmf_vars"}' \
> > -machine
> > q35,memory-backend=mem-machine_mem,pflash0=drive_ovmf_code,
> > pflash1=drive_ovmf_vars \
> > -device '{"id": "pcie-root-port-0", "driver": "pcie-root-port", "bus":
> > "pcie.0", "addr": "0x1", "chassis": 1}' \
> > -nodefaults \
> > -device '{"driver": "VGA", "bus": "pcie.0", "addr": "0x2"}' \
> > -m 62464 \
> > -object '{"size": 65498251264, "id": "mem-machine_mem", "qom-type":
> > "memory-backend-ram"}'  \
> > -smp 24,maxcpus=24,cores=12,threads=1,dies=1,sockets=2  \
> > -cpu 'Icelake-Server-noTSX',+kvm_pv_unhalt \
> > -device '{"id": "pcie-root-port-3", "driver": "pcie-root-port", "addr":
> > "0x3", "chassis": 3}' \
> > -device '{"id": "virtio_scsi_pci0", "driver": "virtio-scsi-pci", "bus":
> > "pcie-root-port-3", "addr": "0x0"}' \
> > -blockdev '{"node-name": "file_image1", "driver": "file", "auto-read-only":
> > true, "discard": "unmap", "aio": "threads", "filename":
> > "/home/kvm_autotest_root/images/rhel930-64-virtio-scsi.qcow2", "cache":
> > {"direct": true, "no-flush": false}}' \
> > -blockdev '{"node-name": "drive_image1", "driver": "qcow2", "read-only":
> > false, "cache": {"direct": true, "no-flush": false}, "file": "file_image1"}'
> > \
> > -device '{"driver": "scsi-hd", "id": "image1", "drive": "drive_image1",
> > "write-cache": "on"}' \
> > -device '{"id": "pcie-root-port-4", "driver": "pcie-root-port", "addr":
> > "0x4", "chassis": 4}' \
> > -device '{"driver": "virtio-net-pci", "mac": "9a:03:a2:11:64:35", "id":
> > "net0", "netdev": "hostnet0", "acpi-index": 1, "bus": "pcie-root-port-4",
> > "addr": "0x0"}'  \
> 
> this is attached to root port, should be attached to hostbridge (bus=pcie.0)
> 
Used the following CLI still can test pass:

/root/qemu/build/x86_64-softmmu/qemu-system-x86_64 \
-name 'vm1'  \
-blockdev '{"node-name": "file_ovmf_code", "driver": "file", "filename": "/usr/share/OVMF/OVMF_CODE.secboot.fd", "auto-read-only": true, "discard": "unmap"}' \
-blockdev '{"node-name": "drive_ovmf_code", "driver": "raw", "read-only": true, "file": "file_ovmf_code"}' \
-blockdev '{"node-name": "file_ovmf_vars", "driver": "file", "filename": "/root/avocado/data/avocado-vt/avocado-vt-vm1_rhel930-64-virtio-scsi_qcow2_filesystem_VARS.fd", "auto-read-only": tr\
-blockdev '{"node-name": "drive_ovmf_vars", "driver": "raw", "read-only": false, "file": "file_ovmf_vars"}' \
-machine q35,memory-backend=mem-machine_mem,pflash0=drive_ovmf_code,pflash1=drive_ovmf_vars \
-device '{"id": "pcie-root-port-0", "driver": "pcie-root-port", "bus": "pcie.0", "addr": "0x1", "chassis": 1}' \
-nodefaults \
-device '{"driver": "VGA", "bus": "pcie.0", "addr": "0x2"}' \
-m 7168 \
-object '{"size": 7516192768, "id": "mem-machine_mem", "qom-type": "memory-backend-ram"}'  \
-smp 6,maxcpus=6,cores=3,threads=1,dies=1,sockets=2  \
-cpu 'Haswell-noTSX-IBRS',vme=on,ss=on,vmx=on,pdcm=on,f16c=on,rdrand=on,hypervisor=on,arat=on,tsc-adjust=on,umip=on,md-clear=on,stibp=on,arch-capabilities=on,ssbd=on,xsaveopt=on,pdpe1gb=on,\
-device '{"driver": "qemu-xhci", "id": "usb1", "bus": "pcie.0", "addr": "0x3"}' \
-device '{"driver": "usb-tablet", "id": "usb-tablet1", "bus": "usb1.0", "port": "1"}' \
-device '{"id": "virtio_scsi_pci0", "driver": "virtio-scsi-pci", "bus": "pcie.0", "addr": "0x4"}' \
-blockdev '{"node-name": "file_image1", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "threads", "filename": "/home/rhel930-64-virtio-scsi.qcow2", "cache": {"direct":\
-blockdev '{"node-name": "drive_image1", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_image1"}' \
-device '{"driver": "scsi-hd", "id": "image1", "drive": "drive_image1", "write-cache": "on"}' \
-device '{"driver": "virtio-net-pci", "mac": "9a:a8:eb:cb:30:e7", "id": "net0", "netdev": "hostnet0", "acpi-index": 1, "bus": "pcie.0", "addr": "0x5"}'  \
-netdev tap,id=hostnet0,vhost=on,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown \
-vnc :0  \
-rtc base=utc,clock=host,driftfix=slew  \
-boot menu=off,order=cdn,once=d,strict=off  \
-enable-kvm \
-monitor stdio \

(In reply to Igor Mammedov from comment #34)
> (In reply to Lei Yang from comment #32)
> > ==> Scenario 1: Test on the PC
> > 
> > ==>Test steps
> > 1. Boot a guest
> > /root/qemu/build/x86_64-softmmu/qemu-system-x86_64 \
> > -name 'vm1'  \
> > -machine pc,memory-backend=mem-machine_mem  \
> > -nodefaults \
> > -device '{"driver": "VGA", "bus": "pci.0", "addr": "0x2"}' \
> > -m 62464 \
> > -object '{"size": 65498251264, "id": "mem-machine_mem", "qom-type":
> > "memory-backend-ram"}'  \
> > -smp 24,maxcpus=24,cores=12,threads=1,dies=1,sockets=2  \
> > -cpu
> > 'Icelake-Server',ds=on,ss=on,dtes64=on,vmx=on,pdcm=on,hypervisor=on,tsc-
> > adjust=on,avx512ifma=on,sha-ni=on,rdpid=on,fsrm=on,md-clear=on,stibp=on,arch-
> > capabilities=on,xsaves=on,ibpb=on,ibrs=on,amd-stibp=on,amd-ssbd=on,rdctl-
> > no=on,ibrs-all=on,skip-l1dfl-vmentry=on,mds-no=on,pschange-mc-no=on,tsx-
> > ctrl=on,hle=off,rtm=off,mpx=off,intel-pt=off,kvm_pv_unhalt=on \
> > -device '{"driver": "ich9-usb-ehci1", "id": "usb1", "addr": "0x1d.0x7",
> > "multifunction": true, "bus": "pci.0"}' \
> > -device '{"driver": "ich9-usb-uhci1", "id": "usb1.0", "multifunction": true,
> > "masterbus": "usb1.0", "addr": "0x1d.0x0", "firstport": 0, "bus": "pci.0"}' \
> > -device '{"driver": "ich9-usb-uhci2", "id": "usb1.1", "multifunction": true,
> > "masterbus": "usb1.0", "addr": "0x1d.0x2", "firstport": 2, "bus": "pci.0"}' \
> > -device '{"driver": "ich9-usb-uhci3", "id": "usb1.2", "multifunction": true,
> > "masterbus": "usb1.0", "addr": "0x1d.0x4", "firstport": 4, "bus": "pci.0"}' \
> > -device '{"driver": "usb-tablet", "id": "usb-tablet1", "bus": "usb1.0",
> > "port": "1"}' \
> > -device '{"id": "virtio_scsi_pci0", "driver": "virtio-scsi-pci", "bus":
> > "pci.0", "addr": "0x3"}' \
> > -blockdev '{"node-name": "file_image1", "driver": "file", "auto-read-only":
> > true, "discard": "unmap", "aio": "threads", "filename":
> > "/home/kvm_autotest_root/images/rhel930-64-virtio-scsi.qcow2", "cache":
> > {"direct": true, "no-flush": false}}' \
> > -blockdev '{"node-name": "drive_image1", "driver": "qcow2", "read-only":
> > false, "cache": {"direct": true, "no-flush": false}, "file": "file_image1"}'
> > \
> > -device '{"driver": "scsi-hd", "id": "image1", "drive": "drive_image1",
> > "write-cache": "on"}' \
> 
> > -device '{"driver": "virtio-net-pci", "mac": "9a:56:74:43:29:a0",
> > "acpi-index": 1, "id": "net0", "netdev": "hostnet0", "bus": "pci.0", "addr":
> > "0x4"}'  \
> 
> this worked before new patches (i.e. on PC acpi hotplug is enabled on both
> hostbridge
> and coldplugged bridges by default).
> 
> > -netdev
> > tap,id=hostnet0,vhost=on,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown \
> > -vnc :0  \
> > -rtc base=utc,clock=host,driftfix=slew  \
> > -boot menu=off,order=cdn,once=c,strict=off \
> > -enable-kvm \
> > -monitor stdio \
> > -qmp tcp:0:5555,server,nowait \
> > 
> > 2. After guest boot up and check the nic name. "enoN"=="acpi-index=N", this
> > is expect result.
> 
> > 3. hotplug/unplug it via qmp, then reboot it there is no any error messages
> > in the host&guest.
> > {"execute": "device_del", "arguments": {"id": "net0"}}	
> > {"return": {}}
> > {"timestamp": {"seconds": 1681362763, "microseconds": 503548}, "event":
> > "DEVICE_DELETED", "data": {"path":
> > "/machine/peripheral/net0/virtio-backend"}}
> > {"timestamp": {"seconds": 1681362763, "microseconds": 554157}, "event":
> > "DEVICE_DELETED", "data": {"device": "net0", "path":
> > "/machine/peripheral/net0"}}
> > {"execute": "netdev_del", "arguments": {"id": "hostnet0"}}
> > {"return": {}}
> > {"timestamp": {"seconds": 1681362783, "microseconds": 778654}, "event":
> > "RESET", "data": {"guest": true, "reason": "guest-reset"}}
> > {"timestamp": {"seconds": 1681362783, "microseconds": 785609}, "event":
> > "RESET", "data": {"guest": true, "reason": "guest-reset"}}
> > {"execute": "netdev_add", "arguments": {
> > "type":"tap","id":"hostnet0","vhost":
> > true,"script":"/etc/qemu-ifup","downscript":"/etc/qemu-ifdown"}}
> > {"return": {}}
> > {"execute":"device_add","arguments":{"driver":"virtio-net-pci","netdev":
> > "hostnet0","mac":"00:11:22:33:44:00","id":"net0","acpi-index":"1","bus":"pci.
> > 0","addr":"0x4"}}
> 
> ditto

Yes, for the PC part, QE had add case and automated it in the past.

(In reply to Igor Mammedov from comment #34)
> (In reply to Lei Yang from comment #32)
> what you haven't tried is disabling ACPI hotplug on hostbridge and/or bridges
> and then checking if acpi-index still works there.
> 
> perhaps QEMU change log will give you more ideas about what to test:
> https://wiki.qemu.org/ChangeLog/8.0#x86
Hello Igor

Before testing, could you please help me to review whether the method of these two qemu command lines I use is correct? 

1.devices attached to directly to q35 hostbridge
/usr/libexec/qemu-kvm \
-name 'vm1'  \
-sandbox on  \
-blockdev '{"node-name": "file_ovmf_code", "driver": "file", "filename": "/usr/share/OVMF/OVMF_CODE.secboot.fd", "auto-read-only": true, "discard": "unmap"}'\
-blockdev '{"node-name": "drive_ovmf_code", "driver": "raw", "read-only": true, "file": "file_ovmf_code"}' \
-blockdev '{"node-name": "file_ovmf_vars", "driver": "file", "filename": "/root/avocado/data/avocado-vt/avocado-vt-vm1_rhel930-64-virtio-scsi_qcow2_filesyste\
-blockdev '{"node-name": "drive_ovmf_vars", "driver": "raw", "read-only": false, "file": "file_ovmf_vars"}' \
-machine q35,memory-backend=mem-machine_mem,pflash0=drive_ovmf_code,pflash1=drive_ovmf_vars \
-device '{"id": "pcie-root-port-0", "driver": "pcie-root-port", "multifunction": true, "bus": "pcie.0", "addr": "0x1", "chassis": 1}' \
-nodefaults \
-device '{"driver": "VGA", "bus": "pcie.0", "addr": "0x2"}' \
-m 7168 \
-object '{"size": 7516192768, "id": "mem-machine_mem", "qom-type": "memory-backend-ram"}'  \
-smp 6,maxcpus=6,cores=3,threads=1,dies=1,sockets=2  \
-cpu 'Haswell-noTSX',+kvm_pv_unhalt \
-device '{"id": "pcie-root-port-1", "port": 1, "driver": "pcie-root-port", "addr": "0x1.0x1", "bus": "pcie.0", "chassis": 2}' \
-device '{"driver": "qemu-xhci", "id": "usb1", "bus": "pcie-root-port-1", "addr": "0x0"}' \
-device '{"driver": "usb-tablet", "id": "usb-tablet1", "bus": "usb1.0", "port": "1"}' \
-device '{"id": "pcie-root-port-2", "port": 2, "driver": "pcie-root-port", "addr": "0x1.0x2", "bus": "pcie.0", "chassis": 3}' \
-device '{"id": "virtio_scsi_pci0", "driver": "virtio-scsi-pci", "bus": "pcie-root-port-2", "addr": "0x0"}' \
-blockdev '{"node-name": "file_image1", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "threads", "filename": "/home/kvm_autotest_root/\
-blockdev '{"node-name": "drive_image1", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_image1"}' \
-device '{"driver": "scsi-hd", "id": "image1", "drive": "drive_image1", "write-cache": "on"}' \
-device '{"id": "pcie-root-port-3", "port": 3, "driver": "pcie-root-port", "addr": "0x1.0x3", "bus": "pcie.0", "chassis": 4}' \
-device '{"driver": "virtio-net-pci", "mac": "00:11:22:33:44:00", "id": "net0", "netdev": "hostnet0", "bus": "pcie-root-port-3", "addr": "0x0"}'  \
-netdev vhost-vdpa,id=hostnet0,vhostdev=/dev/vhost-vdpa-0 \
-vnc :0  \
-rtc base=utc,clock=host,driftfix=slew  \
-boot menu=off,order=cdn,once=c,strict=off \
-enable-kvm \
-device '{"id": "pcie_extra_root_port_0", "driver": "pcie-root-port", "multifunction": true, "bus": "pcie.0", "addr": "0x3", "chassis": 5}' \
-monitor stdio \
-qmp tcp:0:5555,server,nowait \

2.non-hotpluggable multifunction devices
/root/qemu/build/x86_64-softmmu/qemu-system-x86_64 \
-name 'vm1'  \
-blockdev '{"node-name": "file_ovmf_code", "driver": "file", "filename": "/usr/share/OVMF/OVMF_CODE.secboot.fd", "auto-read-only": true, "discard": "unmap"}' \
-blockdev '{"node-name": "drive_ovmf_code", "driver": "raw", "read-only": true, "file": "file_ovmf_code"}' \
-blockdev '{"node-name": "file_ovmf_vars", "driver": "file", "filename": "/root/avocado/data/avocado-vt/avocado-vt-vm1_rhel930-64-virtio-scsi_qcow2_filesystem_VARS.fd", "auto-read-only": true, "discard": "unmap"}' \
-blockdev '{"node-name": "drive_ovmf_vars", "driver": "raw", "read-only": false, "file": "file_ovmf_vars"}' \
-machine q35,memory-backend=mem-machine_mem,pflash0=drive_ovmf_code,pflash1=drive_ovmf_vars \
-device '{"id": "pcie-root-port-0", "driver": "pcie-root-port", "bus": "pcie.0", "addr": "0x1", "chassis": 1}' \
-nodefaults \
-device '{"driver": "VGA", "bus": "pcie.0", "addr": "0x2"}' \
-m 7168 \
-object '{"size": 7516192768, "id": "mem-machine_mem", "qom-type": "memory-backend-ram"}'  \
-smp 6,maxcpus=6,cores=3,threads=1,dies=1,sockets=2  \
-cpu 'Haswell-noTSX-IBRS',vme=on,ss=on,vmx=on,pdcm=on,f16c=on,rdrand=on,hypervisor=on,arat=on,tsc-adjust=on,umip=on,md-clear=on,stibp=on,arch-capabilities=on,ssbd=on,xsaveopt=on,pdpe1gb=on,abm=on,ibpb=on,ibrs=on,amd-stibp=on,amd-ssbd=on,skip-l1dfl-vmentry=on,pschange-mc-\
-device '{"driver": "qemu-xhci", "id": "usb1", "bus": "pcie.0", "addr": "0x3"}' \
-device '{"driver": "usb-tablet", "id": "usb-tablet1", "bus": "usb1.0", "port": "1"}' \
-device '{"id": "virtio_scsi_pci0", "driver": "virtio-scsi-pci", "bus": "pcie.0", "addr": "0x4"}' \
-blockdev '{"node-name": "file_image1", "driver": "file", "auto-read-only": true, "discard": "unmap", "aio": "threads", "filename": "/home/rhel930-64-virtio-scsi.qcow2", "cache": {"direct": true, "no-flush": false}}' \
-blockdev '{"node-name": "drive_image1", "driver": "qcow2", "read-only": false, "cache": {"direct": true, "no-flush": false}, "file": "file_image1"}' \
-device '{"driver": "scsi-hd", "id": "image1", "drive": "drive_image1", "write-cache": "on"}' \
-vnc :0  \
-rtc base=utc,clock=host,driftfix=slew  \
-boot menu=off,order=cdn,once=d,strict=off  \
-enable-kvm \
-monitor stdio \
-qmp tcp:0:5555,server,nowait \
-device '{"driver": "virtio-net-pci", "mac": "9a:a8:eb:cb:30:e7", "id": "net0", "netdev": "hostnet0", "bus": "pcie.0", "addr": "0x5"}'  \
-netdev tap,id=hostnet0,vhost=on,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown \

Thanks
Lei

(In reply to Lei Yang from comment #37)
> (In reply to Igor Mammedov from comment #34)
> > (In reply to Lei Yang from comment #32)
> > what you haven't tried is disabling ACPI hotplug on hostbridge and/or bridges
> > and then checking if acpi-index still works there.
> > 
> > perhaps QEMU change log will give you more ideas about what to test:
> > https://wiki.qemu.org/ChangeLog/8.0#x86
> Hello Igor
> 
> Before testing, could you please help me to review whether the method of
> these two qemu command lines I use is correct? 
> 
> 1.devices attached to directly to q35 hostbridge
> /usr/libexec/qemu-kvm \
> -name 'vm1'  \
> -sandbox on  \
> -blockdev '{"node-name": "file_ovmf_code", "driver": "file", "filename":
> "/usr/share/OVMF/OVMF_CODE.secboot.fd", "auto-read-only": true, "discard":
> "unmap"}'\
> -blockdev '{"node-name": "drive_ovmf_code", "driver": "raw", "read-only":
> true, "file": "file_ovmf_code"}' \
> -blockdev '{"node-name": "file_ovmf_vars", "driver": "file", "filename":
> "/root/avocado/data/avocado-vt/avocado-vt-vm1_rhel930-64-virtio-
> scsi_qcow2_filesyste\
> -blockdev '{"node-name": "drive_ovmf_vars", "driver": "raw", "read-only":
> false, "file": "file_ovmf_vars"}' \
> -machine
> q35,memory-backend=mem-machine_mem,pflash0=drive_ovmf_code,
> pflash1=drive_ovmf_vars \
> -device '{"id": "pcie-root-port-0", "driver": "pcie-root-port",
> "multifunction": true, "bus": "pcie.0", "addr": "0x1", "chassis": 1}' \
> -nodefaults \
> -device '{"driver": "VGA", "bus": "pcie.0", "addr": "0x2"}' \
> -m 7168 \
> -object '{"size": 7516192768, "id": "mem-machine_mem", "qom-type":
> "memory-backend-ram"}'  \
> -smp 6,maxcpus=6,cores=3,threads=1,dies=1,sockets=2  \
> -cpu 'Haswell-noTSX',+kvm_pv_unhalt \
> -device '{"id": "pcie-root-port-1", "port": 1, "driver": "pcie-root-port",
> "addr": "0x1.0x1", "bus": "pcie.0", "chassis": 2}' \
> -device '{"driver": "qemu-xhci", "id": "usb1", "bus": "pcie-root-port-1",
> "addr": "0x0"}' \
> -device '{"driver": "usb-tablet", "id": "usb-tablet1", "bus": "usb1.0",
> "port": "1"}' \
> -device '{"id": "pcie-root-port-2", "port": 2, "driver": "pcie-root-port",
> "addr": "0x1.0x2", "bus": "pcie.0", "chassis": 3}' \
> -device '{"id": "virtio_scsi_pci0", "driver": "virtio-scsi-pci", "bus":
> "pcie-root-port-2", "addr": "0x0"}' \
> -blockdev '{"node-name": "file_image1", "driver": "file", "auto-read-only":
> true, "discard": "unmap", "aio": "threads", "filename":
> "/home/kvm_autotest_root/\
> -blockdev '{"node-name": "drive_image1", "driver": "qcow2", "read-only":
> false, "cache": {"direct": true, "no-flush": false}, "file": "file_image1"}'
> \
> -device '{"driver": "scsi-hd", "id": "image1", "drive": "drive_image1",
> "write-cache": "on"}' \
> -device '{"id": "pcie-root-port-3", "port": 3, "driver": "pcie-root-port",
> "addr": "0x1.0x3", "bus": "pcie.0", "chassis": 4}' \
> -device '{"driver": "virtio-net-pci", "mac": "00:11:22:33:44:00", "id":
> "net0", "netdev": "hostnet0", "bus": "pcie-root-port-3", "addr": "0x0"}'  \
> -netdev vhost-vdpa,id=hostnet0,vhostdev=/dev/vhost-vdpa-0 \
> -vnc :0  \
> -rtc base=utc,clock=host,driftfix=slew  \
> -boot menu=off,order=cdn,once=c,strict=off \
> -enable-kvm \
> -device '{"id": "pcie_extra_root_port_0", "driver": "pcie-root-port",
> "multifunction": true, "bus": "pcie.0", "addr": "0x3", "chassis": 5}' \
> -monitor stdio \
> -qmp tcp:0:5555,server,nowait \

this doesn't have any NIC directly on hostbridge bus (pcie.0) with acpi-index
 
> 2.non-hotpluggable multifunction devices
> /root/qemu/build/x86_64-softmmu/qemu-system-x86_64 \
> -name 'vm1'  \
> -blockdev '{"node-name": "file_ovmf_code", "driver": "file", "filename":
> "/usr/share/OVMF/OVMF_CODE.secboot.fd", "auto-read-only": true, "discard":
> "unmap"}' \
> -blockdev '{"node-name": "drive_ovmf_code", "driver": "raw", "read-only":
> true, "file": "file_ovmf_code"}' \
> -blockdev '{"node-name": "file_ovmf_vars", "driver": "file", "filename":
> "/root/avocado/data/avocado-vt/avocado-vt-vm1_rhel930-64-virtio-
> scsi_qcow2_filesystem_VARS.fd", "auto-read-only": true, "discard": "unmap"}'
> \
> -blockdev '{"node-name": "drive_ovmf_vars", "driver": "raw", "read-only":
> false, "file": "file_ovmf_vars"}' \
> -machine
> q35,memory-backend=mem-machine_mem,pflash0=drive_ovmf_code,
> pflash1=drive_ovmf_vars \
> -device '{"id": "pcie-root-port-0", "driver": "pcie-root-port", "bus":
> "pcie.0", "addr": "0x1", "chassis": 1}' \
> -nodefaults \
> -device '{"driver": "VGA", "bus": "pcie.0", "addr": "0x2"}' \
> -m 7168 \
> -object '{"size": 7516192768, "id": "mem-machine_mem", "qom-type":
> "memory-backend-ram"}'  \
> -smp 6,maxcpus=6,cores=3,threads=1,dies=1,sockets=2  \
> -cpu
> 'Haswell-noTSX-IBRS',vme=on,ss=on,vmx=on,pdcm=on,f16c=on,rdrand=on,
> hypervisor=on,arat=on,tsc-adjust=on,umip=on,md-clear=on,stibp=on,arch-
> capabilities=on,ssbd=on,xsaveopt=on,pdpe1gb=on,abm=on,ibpb=on,ibrs=on,amd-
> stibp=on,amd-ssbd=on,skip-l1dfl-vmentry=on,pschange-mc-\
> -device '{"driver": "qemu-xhci", "id": "usb1", "bus": "pcie.0", "addr":
> "0x3"}' \
> -device '{"driver": "usb-tablet", "id": "usb-tablet1", "bus": "usb1.0",
> "port": "1"}' \
> -device '{"id": "virtio_scsi_pci0", "driver": "virtio-scsi-pci", "bus":
> "pcie.0", "addr": "0x4"}' \
> -blockdev '{"node-name": "file_image1", "driver": "file", "auto-read-only":
> true, "discard": "unmap", "aio": "threads", "filename":
> "/home/rhel930-64-virtio-scsi.qcow2", "cache": {"direct": true, "no-flush":
> false}}' \
> -blockdev '{"node-name": "drive_image1", "driver": "qcow2", "read-only":
> false, "cache": {"direct": true, "no-flush": false}, "file": "file_image1"}'
> \
> -device '{"driver": "scsi-hd", "id": "image1", "drive": "drive_image1",
> "write-cache": "on"}' \
> -vnc :0  \
> -rtc base=utc,clock=host,driftfix=slew  \
> -boot menu=off,order=cdn,once=d,strict=off  \
> -enable-kvm \
> -monitor stdio \
> -qmp tcp:0:5555,server,nowait \
> -device '{"driver": "virtio-net-pci", "mac": "9a:a8:eb:cb:30:e7", "id":
> "net0", "netdev": "hostnet0", "bus": "pcie.0", "addr": "0x5"}'  \
> -netdev
> tap,id=hostnet0,vhost=on,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown \

I might be blind, but I don't see anything with multifunction here.
Though this time you have NIC on hostbridge, albeit without ACPI index.

I'm not telling you how to fix it/write tests cases intentionally,
since I have most of the new cases covered in unit tests.

My hope is that you starting from blank sheet (without looking at unit tests)
my come up with some config that will discover a bug. (and I think that above
Changelog has enough data on new supported configurations, to write tests
from user point of view).


BTW:
while discussing tests it not necessary to provide full command line,
It's enough to just describe root-ports/bridges and NICs that are related,
which hotplug is enabled and machine type.
The rest doesn't have an impact and just clutters CLI.

> Thanks
> Lei

> > Before testing, could you please help me to review whether the method of
> > these two qemu command lines I use is correct? 
> > 
> > 1.devices attached to directly to q35 hostbridge
> > -device '{"id": "pcie-root-port-3", "port": 3, "driver": "pcie-root-port",
> > "addr": "0x1.0x3", "bus": "pcie.0", "chassis": 4}' \
> > -device '{"driver": "virtio-net-pci", "mac": "00:11:22:33:44:00", "id":
> > "net0", "netdev": "hostnet0", "bus": "pcie-root-port-3", "addr": "0x0"}'  \
> > -netdev vhost-vdpa,id=hostnet0,vhostdev=/dev/vhost-vdpa-0 \
> >  
> > 2.non-hotpluggable multifunction devices
> > -device '{"driver": "virtio-net-pci", "mac": "9a:a8:eb:cb:30:e7", "id":
> > "net0", "netdev": "hostnet0", "bus": "pcie.0", "addr": "0x5"}'  \
> > -netdev
> > tap,id=hostnet0,vhost=on,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown \
> 


Maybe I didn't express clearly, I am not confirming the writing of test cases with you, I just want to confirm whether the command line format I listed corresponds to your requirements, if not, how do I need to rectify, please be more specific , as I'm not an expert on this. If it is correct, I will add some cases according to the changelog.

Thanks
Lei

(In reply to Lei Yang from comment #39)
> > > Before testing, could you please help me to review whether the method of
> > > these two qemu command lines I use is correct? 
> > > 
> > > 1.devices attached to directly to q35 hostbridge
> > > -device '{"id": "pcie-root-port-3", "port": 3, "driver": "pcie-root-port",
> > > "addr": "0x1.0x3", "bus": "pcie.0", "chassis": 4}' \
> > > -device '{"driver": "virtio-net-pci", "mac": "00:11:22:33:44:00", "id":
> > > "net0", "netdev": "hostnet0", "bus": "pcie-root-port-3", "addr": "0x0"}'  \
> > > -netdev vhost-vdpa,id=hostnet0,vhostdev=/dev/vhost-vdpa-0 \
> > >  
> > > 2.non-hotpluggable multifunction devices
> > > -device '{"driver": "virtio-net-pci", "mac": "9a:a8:eb:cb:30:e7", "id":
> > > "net0", "netdev": "hostnet0", "bus": "pcie.0", "addr": "0x5"}'  \
> > > -netdev
> > > tap,id=hostnet0,vhost=on,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown \
> > 
> 
> 
> Maybe I didn't express clearly, I am not confirming the writing of test
> cases with you, I just want to confirm whether the command line format I
> listed corresponds to your requirements, if not, how do I need to rectify,
> please be more specific , as I'm not an expert on this. If it is correct, I
> will add some cases according to the changelog.

If you test acpi-index property, it should be present on CLI (i.e. on NIC attached to PCI slot:function of interest)

> 
> Thanks
> Lei

==>Test on Q35

Case 1:Attached to pcie-root-port
1. qemu CLI
-global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=on | off \
-device '{"id": "pcie-root-port-3", "port": 3, "driver": "pcie-root-port", "addr": "0x1.0x3", "bus": "pcie.0", "chassis": 4}' \
-device '{"driver": "virtio-net-pci", "mac": "9a:21:59:c8:7f:7e", "id": "net0", "netdev": "hostnet0", "acpi-index": 5, "bus": "pcie-root-port-3", "addr": "0x0"}'  \

2. Test scenarios: boot,reboot,hotplug/unplug
3. Test result: "enoN"=="acpi-index=N", got expect result.

Case 2: Attached to pcie.0
1. qemu CLI
-global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=on | off \
-device '{"driver": "virtio-net-pci", "mac": "9a:21:59:c8:7f:7e", "id": "net0", "netdev": "hostnet0", "acpi-index": 5, "bus": "pcie.0", "addr": "0x4"}'  \
-netdev tap,id=hostnet0,vhost=on  \

2.Test scenarios: boot,reboot
3. Test result: "enoN"=="acpi-index=N", got expect result.

==>Test on PC
1.qemu CLI
-global PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on | off \
-device '{"driver": "virtio-net-pci", "mac": "9a:81:28:7d:47:39", "id": "net0", "netdev": "hostnet0", "acpi-index": 5, "bus": "pci.0", "addr": "0x4"}'  \
-netdev tap,id=hostnet0,vhost=on  \

2. Test scenarios: boot,reboot,hotplug/unplug
3. Test result: "enoN"=="acpi-index=N", got expect result.


==>Test Version:
kernel-5.14.0-302.el9.x86_64
qemu-kvm-8.0.0-1.el9.x86_64

Hello Igor

Based on the above test result, it has been support very well on qemu-kvm-8.0.0-1.el9.x86_64. Since QE would like to confirm two question with you, could you please help review it? Thanks in advance.

1. QEMU had rebase to qemu-kvm-8.0.0-1.el9.x86_64, could you please help move to "MODIFIED" and add fix version?
2. Since layer products use the method of attaching to pcie-root-port by default，can QE only focus on this method in future testing?

Best Regards
Lei

(In reply to Lei Yang from comment #41)
> ==>Test on Q35
> 
> Case 1:Attached to pcie-root-port
> 1. qemu CLI
> -global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=on | off \
                                                        ^^^^^^^^
such value is invalid, it should be either 'on' or 'off'
suggest to split test in 2, one for 'on' and another for 'off'

same applies to test case 2 and 3

> -device '{"id": "pcie-root-port-3", "port": 3, "driver": "pcie-root-port",
> "addr": "0x1.0x3", "bus": "pcie.0", "chassis": 4}' \
> -device '{"driver": "virtio-net-pci", "mac": "9a:21:59:c8:7f:7e", "id":
> "net0", "netdev": "hostnet0", "acpi-index": 5, "bus": "pcie-root-port-3",
> "addr": "0x0"}'  \
> 
> 2. Test scenarios: boot,reboot,hotplug/unplug

in case hotplug is enabled, you should be able to change acpi-index to another value when hotplugging nic

another test case for root port, is have
   -global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=on
enabled but disable hotplug on root port directly 
   -device pcie-root-port,hotplug=off,...

> 3. Test result: "enoN"=="acpi-index=N", got expect result.
> 
> Case 2: Attached to pcie.0
> 1. qemu CLI
> -global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=on | off \
> -device '{"driver": "virtio-net-pci", "mac": "9a:21:59:c8:7f:7e", "id":
> "net0", "netdev": "hostnet0", "acpi-index": 5, "bus": "pcie.0", "addr":
> "0x4"}'  \
> -netdev tap,id=hostnet0,vhost=on  \
> 
> 2.Test scenarios: boot,reboot
> 3. Test result: "enoN"=="acpi-index=N", got expect result.
> 
> ==>Test on PC
> 1.qemu CLI
> -global PIIX4_PM.acpi-pci-hotplug-with-bridge-support=on | off \
> -device '{"driver": "virtio-net-pci", "mac": "9a:81:28:7d:47:39", "id":
> "net0", "netdev": "hostnet0", "acpi-index": 5, "bus": "pci.0", "addr":
> "0x4"}'  \
> -netdev tap,id=hostnet0,vhost=on  \
> 
> 2. Test scenarios: boot,reboot,hotplug/unplug
> 3. Test result: "enoN"=="acpi-index=N", got expect result.
> 
> 
> ==>Test Version:
> kernel-5.14.0-302.el9.x86_64
> qemu-kvm-8.0.0-1.el9.x86_64

(In reply to Lei Yang from comment #42)
> Hello Igor
> 
> Based on the above test result, it has been support very well on
> qemu-kvm-8.0.0-1.el9.x86_64. Since QE would like to confirm two question
> with you, could you please help review it? Thanks in advance.
> 
> 1. QEMU had rebase to qemu-kvm-8.0.0-1.el9.x86_64, could you please help
> move to "MODIFIED" and add fix version?

Done.

> 2. Since layer products use the method of attaching to pcie-root-port by
> default，can QE only focus on this method in future testing?

I guess so (q35 assumed), for 'pc' machine you still need to test
devices on root bus (pci.0)

> 
> Best Regards
> Lei

QE bot(pre verify): Set 'Verified:Tested,SanityOnly' as gating/tier1 test pass.

(In reply to Igor Mammedov from comment #43)
> (In reply to Lei Yang from comment #41)
> > ==>Test on Q35
> > 
> > Case 1:Attached to pcie-root-port
> > 1. qemu CLI
> > -global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=on | off \
>                                                         ^^^^^^^^
> such value is invalid, it should be either 'on' or 'off'
> suggest to split test in 2, one for 'on' and another for 'off'
> 
> same applies to test case 2 and 3
Sorry, my description is always misleading, what I want to express is that the "on" and "off" scenarios are tested here.

> > 2. Test scenarios: boot,reboot,hotplug/unplug
> 
> in case hotplug is enabled, you should be able to change acpi-index to
> another value when hotplugging nic
> 
> another test case for root port, is have
>    -global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=on
> enabled but disable hotplug on root port directly 
>    -device pcie-root-port,hotplug=off,...
-global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=on \
-device '{"id": "pcie-root-port-3", "hotplug": false, "port": 3, "driver": "pcie-root-port", "addr": "0x1.0x3", "bus": "pcie.0", "chassis": 4}' \

Used the above qemu CLI to boot guest, then hotplug a nic to this root port, it will print: {"error": {"class": "GenericError", "desc": "Bus 'pcie-root-port-3' does not support hotplugging"}}. I think this is a expect result, please correct me if I'm wrong.

Based the above testing result this bug has been support very well on qemu-kvm-8.0.0-1.el9.x86_64.

(In reply to Lei Yang from comment #46)
> (In reply to Igor Mammedov from comment #43)
> > (In reply to Lei Yang from comment #41)
> > > ==>Test on Q35
> > > 
> > > Case 1:Attached to pcie-root-port
> > > 1. qemu CLI
> > > -global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=on | off \
> >                                                         ^^^^^^^^
> > such value is invalid, it should be either 'on' or 'off'
> > suggest to split test in 2, one for 'on' and another for 'off'
> > 
> > same applies to test case 2 and 3
> Sorry, my description is always misleading, what I want to express is that
> the "on" and "off" scenarios are tested here.
> 
> > > 2. Test scenarios: boot,reboot,hotplug/unplug
> > 
> > in case hotplug is enabled, you should be able to change acpi-index to
> > another value when hotplugging nic
> > 
> > another test case for root port, is have
> >    -global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=on
> > enabled but disable hotplug on root port directly 
> >    -device pcie-root-port,hotplug=off,...
> -global ICH9-LPC.acpi-pci-hotplug-with-bridge-support=on \
> -device '{"id": "pcie-root-port-3", "hotplug": false, "port": 3, "driver":
> "pcie-root-port", "addr": "0x1.0x3", "bus": "pcie.0", "chassis": 4}' \
> 
> Used the above qemu CLI to boot guest, then hotplug a nic to this root port,
> it will print: {"error": {"class": "GenericError", "desc": "Bus
> 'pcie-root-port-3' does not support hotplugging"}}. I think this is a expect
> result, please correct me if I'm wrong.

Yes, it's expected result since you explocitly disabled hotplug on that root port.

> 
> Based the above testing result this bug has been support very well on
> qemu-kvm-8.0.0-1.el9.x86_64.

Based on the Comment 41,46,49, move to "VERIFIED".

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: qemu-kvm security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2023:6368