1353141 – [Q35] disk disk didn't be found when hot-plug to the downstream port

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1353141 - [Q35] disk disk didn't be found when hot-plug to the downstream port

Summary: [Q35] disk disk didn't be found when hot-plug to the downstream port

Keywords:
Status:	CLOSED DUPLICATE of bug 1352808
Alias:	None
Product:	Red Hat Enterprise Linux 7
Classification:	Red Hat
Component:	qemu-kvm-rhev
Sub Component:
Version:	7.3
Hardware:	Unspecified
OS:	Unspecified
Priority:	high
Severity:	high
Target Milestone:	rc
Target Release:	---
Assignee:	Marcel Apfelbaum
QA Contact:	jingzhao
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2016-07-06 08:42 UTC by jingzhao
Modified:	2016-08-08 13:16 UTC (History)
CC List:	6 users (show)
Fixed In Version:
Doc Type:	If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:	2016-08-08 13:16:25 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Description jingzhao 2016-07-06 08:42:56 UTC

Description of problem:
virtio block disk didn't be found when hot-plug to the downstream port which the downstream port is not the last one

Version-Release number of selected component (if applicable):
kernel-3.10.0-461.el7.x86_64
qemu-kvm-rhev-2.6.0-11.el7.x86_64
seabios-1.9.1-4.el7.x86_64

How reproducible:
3/3

Steps to Reproduce:
1. Boot guest with following cli:
/usr/libexec/qemu-kvm \
-M q35 \
-cpu SandyBridge \
-nodefaults -rtc base=utc \
-m 4G \
-smp 2,sockets=2,cores=1,threads=1 \
-enable-kvm \
-name rhel7.3 \
-uuid 990ea161-6b67-47b2-b803-19fb01d30d12 \
-smbios type=1,manufacturer='Red Hat',product='RHEV Hypervisor',version=el6,serial=koTUXQrb,uuid=feebc8fd-f8b0-4e75-abc3-e63fcdb67170 \
-k en-us \
-serial unix:/tmp/serial0,server,nowait \
-boot menu=on \
-bios /usr/share/seabios/bios.bin \
-chardev file,path=/home/seabios.log,id=seabios \
-device isa-debugcon,chardev=seabios,iobase=0x402 \
-vga qxl \
-spice port=5932,disable-ticketing \
-qmp tcp:0:6666,server,nowait \
-device ioh3420,id=root.0,slot=1 \
-drive file=/home/rhel73.img,if=none,id=drive-virtio-disk0,format=qcow2,cache=none,werror=stop,rerror=stop \
-device virtio-blk-pci,bus=root.0,drive=drive-virtio-disk0,id=virtio-disk0,disable-legacy=on,disable-modern=off,bootindex=1 \
-device ioh3420,id=root.1,slot=2 \
-device x3130-upstream,bus=root.1,id=upstream1 \
-device xio3130-downstream,bus=upstream1,id=downstream1,chassis=1 \
-device xio3130-downstream,bus=upstream1,id=downstream2,chassis=2 \
-device xio3130-downstream,bus=upstream1,id=downstream3,chassis=3 \
-device xio3130-downstream,bus=upstream1,id=downstream4,chassis=4 \
-drive file=/home/block1.qcow2,if=none,id=drive-virtio-disk1,format=qcow2,cache=none,werror=stop,rerror=stop \
-device virtio-blk-pci,bus=downstream1,drive=drive-virtio-disk1,id=virtio-disk1,disable-legacy=on,disable-modern=off \
-drive file=/home/block2.raw,if=none,id=drive-virtio-disk2,format=raw,cache=none,werror=stop,rerror=stop \
-device virtio-blk-pci,bus=downstream2,drive=drive-virtio-disk2,id=virtio-disk2,disable-legacy=on,disable-modern=off \
-device ioh3420,id=root.2,slot=3 \
-device virtio-net-pci,bus=root.2,netdev=tap10,mac=9a:6a:6b:6c:6d:6e -netdev tap,id=tap10 \
-device ioh3420,id=root.3,slot=4 \
-monitor stdio \

2.Hot-plug virtio block to the downstream3 port
{"execute":"__com.redhat_drive_add", "arguments":{"file":"/home/block4.qcow2","format":"qcow2","id":"test31"}}
{"return": {}}
{"execute":"device_add","arguments":{"driver":"virtio-blk-pci","drive":"test31","id":"test31","disable-legacy":"on","disable-modern":"off","bus":"downstream3"}}
{"return": {}}

3. In guest, check the virtio block disk

Actual results:
[root@localhost ~]# lsblk
Before hot-plug:
NAME          MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
vda           252:0    0   50G  0 disk 
├─vda1        252:1    0    1G  0 part /boot
└─vda2        252:2    0   49G  0 part 
  ├─rhel-root 253:0    0 45.1G  0 lvm  /
  └─rhel-swap 253:1    0  3.9G  0 lvm  [SWAP]
vdb           252:16   0    5G  0 disk 
vdc           252:32   0    8G  0 disk 

After hot-plug:
[root@localhost ~]# lsblk
NAME          MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
vda           252:0    0   50G  0 disk 
├─vda1        252:1    0    1G  0 part /boot
└─vda2        252:2    0   49G  0 part 
  ├─rhel-root 253:0    0 45.1G  0 lvm  /
  └─rhel-swap 253:1    0  3.9G  0 lvm  [SWAP]
vdb           252:16   0    5G  0 disk 
vdc           252:32   0    8G  0 disk 

But in hmp, you can check the disk through "info block"
(qemu) info block
drive-virtio-disk0 (#block129): /home/rhel73.img (qcow2)
    Cache mode:       writeback, direct

drive-virtio-disk1 (#block335): /home/block1.qcow2 (qcow2)
    Cache mode:       writeback, direct

drive-virtio-disk2 (#block582): /home/block2.raw (raw)
    Cache mode:       writeback, direct

test31 (#block721): /home/block4.qcow2 (qcow2)
    Cache mode:       writeback

You must reboot the guest and the disk can be found:
[root@localhost ~]# lsblk
NAME          MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
vda           252:0    0   50G  0 disk 
├─vda1        252:1    0    1G  0 part /boot
└─vda2        252:2    0   49G  0 part 
  ├─rhel-root 253:0    0 45.1G  0 lvm  /
  └─rhel-swap 253:1    0  3.9G  0 lvm  [SWAP]
vdb           252:16   0    8G  0 disk 
vdc           252:32   0    5G  0 disk 
vdd           252:48   0   20G  0 disk 


Expected results:
block disk can be found and didn't need to reboot the guest

Additional info:
The disk can be found and didn't need to reboot the guest if the downstream port which want to hot-plug is the last one

Following is the command which disk can be found right now after hot-plug operation
/usr/libexec/qemu-kvm \
-M q35 \
-cpu SandyBridge \
-nodefaults -rtc base=utc \
-m 4G \
-smp 2,sockets=2,cores=1,threads=1 \
-enable-kvm \
-name rhel7.3 \
-uuid 990ea161-6b67-47b2-b803-19fb01d30d12 \
-smbios type=1,manufacturer='Red Hat',product='RHEV Hypervisor',version=el6,serial=koTUXQrb,uuid=feebc8fd-f8b0-4e75-abc3-e63fcdb67170 \
-k en-us \
-serial unix:/tmp/serial0,server,nowait \
-boot menu=on \
-bios /usr/share/seabios/bios.bin \
-chardev file,path=/home/seabios.log,id=seabios \
-device isa-debugcon,chardev=seabios,iobase=0x402 \
-vga qxl \
-spice port=5932,disable-ticketing \
-qmp tcp:0:6666,server,nowait \
-device ioh3420,id=root.0,slot=1 \
-drive file=/home/rhel73.img,if=none,id=drive-virtio-disk0,format=qcow2,cache=none,werror=stop,rerror=stop \
-device virtio-blk-pci,bus=root.0,drive=drive-virtio-disk0,id=virtio-disk0,disable-legacy=on,disable-modern=off,bootindex=1 \
-device ioh3420,id=root.1,slot=2 \
-device x3130-upstream,bus=root.1,id=upstream1 \
-device xio3130-downstream,bus=upstream1,id=downstream1,chassis=1 \
-device xio3130-downstream,bus=upstream1,id=downstream2,chassis=2 \
-device xio3130-downstream,bus=upstream1,id=downstream3,chassis=3 \
-drive file=/home/block1.qcow2,if=none,id=drive-virtio-disk1,format=qcow2,cache=none,werror=stop,rerror=stop \
-device virtio-blk-pci,bus=downstream1,drive=drive-virtio-disk1,id=virtio-disk1,disable-legacy=on,disable-modern=off \
-drive file=/home/block2.raw,if=none,id=drive-virtio-disk2,format=raw,cache=none,werror=stop,rerror=stop \
-device virtio-blk-pci,bus=downstream2,drive=drive-virtio-disk2,id=virtio-disk2,disable-legacy=on,disable-modern=off \
-device ioh3420,id=root.2,slot=3 \
-device virtio-net-pci,bus=root.2,netdev=tap10,mac=9a:6a:6b:6c:6d:6e -netdev tap,id=tap10 \
-device ioh3420,id=root.3,slot=4 \
-monitor stdio \

Comment 2 jingzhao 2016-07-22 03:32:21 UTC

Virtio scsi also hit the issue

1. In qmp

{"execute":"device_add","arguments":{"driver":"virtio-scsi-pci","id":"scsi3","bus":"downstream3","disable-legacy":true,"disable-modern":false}}
{"return": {}}
{"execute":"__com.redhat_drive_add", "arguments": {"file":"/home/block2.raw","format":"raw","id":"drive_datadisk3"}}
{"return": {}}
{"execute":"device_add","arguments":{"driver":"scsi-hd","drive":"drive_datadisk3","id":"device_datadisk3","bus":"scsi3.0"}}
{"return": {}}

2. In hmp:

(qemu) info block
drive-virtio-disk0 (#block103): /home/q35.qcow2 (qcow2)
    Cache mode:       writeback, direct

satadisk0 (#block330): /home/satadisk0 (raw)
    Cache mode:       writeback

drive_datadisk0 (#block543): /home/block1.qcow2 (qcow2)
    Cache mode:       writeback

drive_datadisk1 (#block704): /home/block4.qcow2 (qcow2)
    Cache mode:       writeback

drive_datadisk3 (#block925): /home/block2.raw (raw)
    Cache mode:       writeback

3. In guest, disk didn't found

[root@localhost ~]# lsblk
NAME          MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda             8:0    0   1G  0 disk 
sdb             8:16   0   5G  0 disk 
vda           252:0    0  30G  0 disk 
├─vda1        252:1    0   1G  0 part /boot
└─vda2        252:2    0  29G  0 part 
  ├─rhel-root 253:0    0  26G  0 lvm  /
  └─rhel-swap 253:1    0   3G  0 lvm  [SWAP]

Comment 3 jingzhao 2016-07-22 04:01:56 UTC

(In reply to jingzhao from comment #2)
> Virtio scsi also hit the issue
> 
> 1. In qmp
> 
> {"execute":"device_add","arguments":{"driver":"virtio-scsi-pci","id":"scsi3",
> "bus":"downstream3","disable-legacy":true,"disable-modern":false}}
> {"return": {}}
> {"execute":"__com.redhat_drive_add", "arguments":
> {"file":"/home/block2.raw","format":"raw","id":"drive_datadisk3"}}
> {"return": {}}
> {"execute":"device_add","arguments":{"driver":"scsi-hd","drive":
> "drive_datadisk3","id":"device_datadisk3","bus":"scsi3.0"}}
> {"return": {}}
> 
> 2. In hmp:
> 
> (qemu) info block
> drive-virtio-disk0 (#block103): /home/q35.qcow2 (qcow2)
>     Cache mode:       writeback, direct
> 
> satadisk0 (#block330): /home/satadisk0 (raw)
>     Cache mode:       writeback
> 
> drive_datadisk0 (#block543): /home/block1.qcow2 (qcow2)
>     Cache mode:       writeback
> 
> drive_datadisk1 (#block704): /home/block4.qcow2 (qcow2)
>     Cache mode:       writeback
> 
> drive_datadisk3 (#block925): /home/block2.raw (raw)
>     Cache mode:       writeback
> 
> 3. In guest, disk didn't found
> 
> [root@localhost ~]# lsblk
> NAME          MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
> sda             8:0    0   1G  0 disk 
> sdb             8:16   0   5G  0 disk 
> vda           252:0    0  30G  0 disk 
> ├─vda1        252:1    0   1G  0 part /boot
> └─vda2        252:2    0  29G  0 part 
>   ├─rhel-root 253:0    0  26G  0 lvm  /
>   └─rhel-swap 253:1    0   3G  0 lvm  [SWAP]

Following is the qemu command:
/usr/libexec/qemu-kvm \
-M q35 \
-cpu SandyBridge \
-nodefaults -rtc base=utc \
-m 4G \
-smp 2,sockets=2,cores=1,threads=1 \
-enable-kvm \
-name rhel7.3 \
-uuid 990ea161-6b67-47b2-b803-19fb01d30d12 \
-smbios type=1,manufacturer='Red Hat',product='RHEV Hypervisor',version=el6,serial=koTUXQrb,uuid=feebc8fd-f8b0-4e75-abc3-e63fcdb67170 \
-k en-us \
-serial unix:/tmp/serial0,server,nowait \
-boot menu=on \
-bios /usr/share/seabios/bios.bin \
-chardev file,path=/home/seabios.log,id=seabios \
-device isa-debugcon,chardev=seabios,iobase=0x402 \
-qmp tcp::8887,server,nowait \
-vga qxl \
-spice port=5932,disable-ticketing \
-device ioh3420,id=root.0,slot=1 \
-drive file=/home/q35.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none,werror=stop,rerror=stop \
-device virtio-blk-pci,bus=root.0,drive=drive-virtio-disk0,id=virtio-disk0,disable-legacy=on,disable-modern=off,bootindex=1 \
-device ahci,id=ahci0 \
-drive file=/home/satadisk0,if=none,id=satadisk0,format=raw \
-device ide-drive,bus=ahci0.0,drive=satadisk0,id=stata0 \
-device ioh3420,id=root.1,slot=2 \
-device x3130-upstream,bus=root.1,id=upstream1 \
-device xio3130-downstream,bus=upstream1,id=downstream1,chassis=1 \
-device x3130-upstream,bus=downstream1,id=upstream1.1 \
-device xio3130-downstream,bus=upstream1.1,id=downstream1.1,chassis=2 \
-device xio3130-downstream,bus=upstream1,id=downstream2,chassis=3 \
-device x3130-upstream,bus=downstream2,id=upstream1.2 \
-device xio3130-downstream,bus=upstream1.2,id=downstream1.2,chassis=4 \
-device xio3130-downstream,bus=upstream1,id=downstream3,chassis=5 \
-device x3130-upstream,bus=downstream3,id=upstream1.3 \
-device xio3130-downstream,bus=upstream1.3,id=downstream1.3,chassis=6 \
-device x3130-upstream,bus=downstream1.3,id=upstream1.3.1 \
-device xio3130-downstream,bus=upstream1.3.1,id=downstream1.3.1,chassis=7 \
-device virtio-net-pci,bus=downstream1.2,netdev=tap10,mac=9a:6a:6b:6c:6d:6e,disable-legacy=on,disable-modern=off -netdev tap,id=tap10 \
-device ioh3420,id=root.2,slot=3 \
-device ioh3420,id=root.3,slot=4 \
-device x3130-upstream,bus=root.3,id=upstream2 \
-device xio3130-downstream,bus=upstream2,id=downstream4,chassis=8 \
-monitor stdio \

Comment 4 jingzhao 2016-07-22 04:12:11 UTC

(In reply to jingzhao from comment #3)
> (In reply to jingzhao from comment #2)
> > Virtio scsi also hit the issue
> > 
> > 1. In qmp
> > 
Modified the cmd
{"execute":"device_add","arguments":{"driver":"virtio-scsi-pci","id":"scsi3","bus":"downstream4","disable-legacy":true,"disable-modern":false}}
{"return": {}}
{"execute":"__com.redhat_drive_add", "arguments": {"file":"/home/block2.raw","format":"raw","id":"drive_datadisk3"}}
{"return": {}}
{"execute":"device_add","arguments":{"driver":"scsi-hd","drive":"drive_datadisk3","id":"device_datadisk3","bus":"scsi3.0"}}
{"return": {}}

> > 
> > 2. In hmp:
> > 
> > (qemu) info block
> > drive-virtio-disk0 (#block103): /home/q35.qcow2 (qcow2)
> >     Cache mode:       writeback, direct
> > 
> > satadisk0 (#block330): /home/satadisk0 (raw)
> >     Cache mode:       writeback
> > 
> > drive_datadisk0 (#block543): /home/block1.qcow2 (qcow2)
> >     Cache mode:       writeback
> > 
> > drive_datadisk1 (#block704): /home/block4.qcow2 (qcow2)
> >     Cache mode:       writeback
> > 
> > drive_datadisk3 (#block925): /home/block2.raw (raw)
> >     Cache mode:       writeback
> > 
> > 3. In guest, disk didn't found
> > 
> > [root@localhost ~]# lsblk
> > NAME          MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
> > sda             8:0    0   1G  0 disk 
> > sdb             8:16   0   5G  0 disk 
> > vda           252:0    0  30G  0 disk 
> > ├─vda1        252:1    0   1G  0 part /boot
> > └─vda2        252:2    0  29G  0 part 
> >   ├─rhel-root 253:0    0  26G  0 lvm  /
> >   └─rhel-swap 253:1    0   3G  0 lvm  [SWAP]
> 
> Following is the qemu command:
> /usr/libexec/qemu-kvm \
> -M q35 \
> -cpu SandyBridge \
> -nodefaults -rtc base=utc \
> -m 4G \
> -smp 2,sockets=2,cores=1,threads=1 \
> -enable-kvm \
> -name rhel7.3 \
> -uuid 990ea161-6b67-47b2-b803-19fb01d30d12 \
> -smbios type=1,manufacturer='Red Hat',product='RHEV
> Hypervisor',version=el6,serial=koTUXQrb,uuid=feebc8fd-f8b0-4e75-abc3-
> e63fcdb67170 \
> -k en-us \
> -serial unix:/tmp/serial0,server,nowait \
> -boot menu=on \
> -bios /usr/share/seabios/bios.bin \
> -chardev file,path=/home/seabios.log,id=seabios \
> -device isa-debugcon,chardev=seabios,iobase=0x402 \
> -qmp tcp::8887,server,nowait \
> -vga qxl \
> -spice port=5932,disable-ticketing \
> -device ioh3420,id=root.0,slot=1 \
> -drive
> file=/home/q35.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none,
> werror=stop,rerror=stop \
> -device
> virtio-blk-pci,bus=root.0,drive=drive-virtio-disk0,id=virtio-disk0,disable-
> legacy=on,disable-modern=off,bootindex=1 \
> -device ahci,id=ahci0 \
> -drive file=/home/satadisk0,if=none,id=satadisk0,format=raw \
> -device ide-drive,bus=ahci0.0,drive=satadisk0,id=stata0 \
> -device ioh3420,id=root.1,slot=2 \
> -device x3130-upstream,bus=root.1,id=upstream1 \
> -device xio3130-downstream,bus=upstream1,id=downstream1,chassis=1 \
> -device x3130-upstream,bus=downstream1,id=upstream1.1 \
> -device xio3130-downstream,bus=upstream1.1,id=downstream1.1,chassis=2 \
> -device xio3130-downstream,bus=upstream1,id=downstream2,chassis=3 \
> -device x3130-upstream,bus=downstream2,id=upstream1.2 \
> -device xio3130-downstream,bus=upstream1.2,id=downstream1.2,chassis=4 \
> -device xio3130-downstream,bus=upstream1,id=downstream3,chassis=5 \
> -device x3130-upstream,bus=downstream3,id=upstream1.3 \
> -device xio3130-downstream,bus=upstream1.3,id=downstream1.3,chassis=6 \
> -device x3130-upstream,bus=downstream1.3,id=upstream1.3.1 \
> -device xio3130-downstream,bus=upstream1.3.1,id=downstream1.3.1,chassis=7 \
> -device
> virtio-net-pci,bus=downstream1.2,netdev=tap10,mac=9a:6a:6b:6c:6d:6e,disable-
> legacy=on,disable-modern=off -netdev tap,id=tap10 \
> -device ioh3420,id=root.2,slot=3 \
> -device ioh3420,id=root.3,slot=4 \
> -device x3130-upstream,bus=root.3,id=upstream2 \
> -device xio3130-downstream,bus=upstream2,id=downstream4,chassis=8 \
> -monitor stdio \

Comment 5 Marcel Apfelbaum 2016-08-08 13:16:25 UTC

Hi,

Please see BZ 1352808, comment #3.
While the symptom is different, the root cause is the same.
The guest OS tries to deal with pci resource re-balancing and fails.

Maybe it worth opening a BZ for the pci guys from linux kernel team explaining how PCI resource re-balancing fails fails for the above scenario.

*** This bug has been marked as a duplicate of bug 1352808 ***

Note You need to log in before you can comment on or make changes to this bug.