Created attachment 1146294 [details] The seabios log when boot up guest Description of problem: fail to find vq for virtio-blk when boot up with 88 virtio-blk-pci data disks via PCIe-switch, and guest didn't boot up Version-Release number of selected component (if applicable): kernel-3.10.0-373.el7.x86_64 qemu-img-rhev-2.5.0-4.el7.x86_64 seabios-1.9.1-2.el7.x86_64 How reproducible: 3/3 Steps to Reproduce: 1.boot guest as the following via multi-pcie.sh shell script. sh multi-pcie.sh 88 cat multi-pcie.sh #!/bin/sh MACHINE=q35 SMP=4,cores=2,threads=2,sockets=1 MEM=2G GUEST_IMG=/home/rhel.img IMG_FORMAT=qcow2 CLI="gdb --args /usr/libexec/qemu-kvm -enable-kvm -M $MACHINE -smp $SMP -m $MEM -name vm1 -drive file=$GUEST_IMG,if=none,id=guest-img,format=$IMG_FORMAT,werror=stop,rerror=stop -device ide-hd,drive=guest-img,bus=ide.0,unit=0,id=os-disk,bootindex=0 -spice port=5931,disable-ticketing -vga qxl -monitor stdio -serial unix:/tmp/console,server,nowait -qmp tcp:0:6666,server,nowait -chardev file,path=/home/seabios.log,id=seabios -device isa-debugcon,chardev=seabios,iobase=0x402 -boot menu=on,reboot-timeout=8,strict=on" echo before juli while [ ${i:=0} -lt ${1:-0} ] do dstreamId=$((i)) ustreamId=$((dstreamId/32)) chassisId=$((dstreamId+1)) blkDiskId=$((dstreamId)) if [ $((dstreamId%32)) -eq 0 ] then CLI="$CLI -device ioh3420,bus=pcie.0,id=root.$ustreamId,slot=$ustreamId" CLI="$CLI -device x3130-upstream,bus=root.$ustreamId,id=upstream$ustreamId" fi qemu-img create -f qcow2 /home/disk/disk$blkDiskId 100M echo "juli$i" CLI="$CLI -device xio3130-downstream,bus=upstream$ustreamId,id=downstream$dstreamId,chassis=$chassisId" CLI="$CLI -drive file=/home/disk/disk$blkDiskId,if=none,id=disk$blkDiskId,format=qcow2 " CLI="$CLI -device virtio-blk-pci,scsi=off,drive=disk$blkDiskId,id=virtio-blk$blkDiskId,bus=downstream$dstreamId,disable-modern=off,disable-legacy=on" i=$((i+1)) done $CLI Actual results: guest didn't boot up and using isa-debugcon debug, got the following messages: All threads complete. Found 0 lpt ports Found 1 serial ports AHCI controller at 1f.2, iobase fb072000, irq 10 Searching bootorder for: /pci@i0cf8/*@1f,2/drive@0/disk@0 AHCI/0: registering: "AHCI/0: QEMU HARDDISK ATA-7 Hard-Disk (20480 MiBytes)" PS2 keyboard initialized found virtio-blk at 3:0 pci dev 3:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 3:0 found virtio-blk at 4:0 pci dev 4:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 4:0 found virtio-blk at 5:0 pci dev 5:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 5:0 found virtio-blk at 6:0 pci dev 6:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 6:0 found virtio-blk at 7:0 pci dev 7:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 7:0 found virtio-blk at 8:0 pci dev 8:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 8:0 found virtio-blk at 9:0 pci dev 9:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 9:0 found virtio-blk at a:0 pci dev a:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk a:0 found virtio-blk at b:0 pci dev b:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk b:0 found virtio-blk at c:0 pci dev c:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk c:0 found virtio-blk at d:0 pci dev d:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk d:0 found virtio-blk at e:0 pci dev e:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk e:0 found virtio-blk at f:0 pci dev f:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk f:0 found virtio-blk at 10:0 pci dev 10:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 10:0 found virtio-blk at 11:0 pci dev 11:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 11:0 found virtio-blk at 12:0 pci dev 12:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 12:0 found virtio-blk at 13:0 pci dev 13:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 13:0 found virtio-blk at 14:0 pci dev 14:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 14:0 found virtio-blk at 15:0 pci dev 15:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 15:0 found virtio-blk at 16:0 pci dev 16:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 16:0 found virtio-blk at 17:0 pci dev 17:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 17:0 found virtio-blk at 18:0 pci dev 18:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 18:0 found virtio-blk at 19:0 pci dev 19:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 19:0 found virtio-blk at 1a:0 pci dev 1a:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 1a:0 found virtio-blk at 1b:0 pci dev 1b:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 1b:0 found virtio-blk at 1c:0 pci dev 1c:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 1c:0 found virtio-blk at 1d:0 pci dev 1d:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 1d:0 found virtio-blk at 1e:0 pci dev 1e:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 1e:0 found virtio-blk at 1f:0 pci dev 1f:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 1f:0 found virtio-blk at 20:0 pci dev 20:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 20:0 found virtio-blk at 21:0 pci dev 21:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 21:0 found virtio-blk at 22:0 pci dev 22:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 22:0 found virtio-blk at 25:0 pci dev 25:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 25:0 found virtio-blk at 26:0 pci dev 26:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 26:0 found virtio-blk at 27:0 pci dev 27:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 27:0 found virtio-blk at 28:0 pci dev 28:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 28:0 found virtio-blk at 29:0 pci dev 29:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 29:0 found virtio-blk at 2a:0 pci dev 2a:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 2a:0 found virtio-blk at 2b:0 pci dev 2b:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 2b:0 found virtio-blk at 2c:0 pci dev 2c:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 2c:0 found virtio-blk at 2d:0 pci dev 2d:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 2d:0 found virtio-blk at 2e:0 pci dev 2e:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 2e:0 found virtio-blk at 2f:0 pci dev 2f:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 2f:0 found virtio-blk at 30:0 pci dev 30:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 30:0 found virtio-blk at 31:0 pci dev 31:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 31:0 found virtio-blk at 32:0 pci dev 32:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 32:0 found virtio-blk at 33:0 pci dev 33:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 33:0 found virtio-blk at 34:0 pci dev 34:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 34:0 found virtio-blk at 35:0 pci dev 35:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 35:0 found virtio-blk at 36:0 pci dev 36:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 36:0 found virtio-blk at 37:0 pci dev 37:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 37:0 found virtio-blk at 38:0 pci dev 38:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 38:0 found virtio-blk at 39:0 pci dev 39:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 39:0 found virtio-blk at 3a:0 pci dev 3a:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 3a:0 found virtio-blk at 3b:0 pci dev 3b:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 3b:0 found virtio-blk at 3c:0 pci dev 3c:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 3c:0 found virtio-blk at 3d:0 pci dev 3d:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 3d:0 found virtio-blk at 3e:0 pci dev 3e:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 3e:0 found virtio-blk at 3f:0 pci dev 3f:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 3f:0 found virtio-blk at 40:0 pci dev 40:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 40:0 found virtio-blk at 41:0 pci dev 41:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 41:0 found virtio-blk at 42:0 pci dev 42:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 42:0 found virtio-blk at 43:0 pci dev 43:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 43:0 found virtio-blk at 44:0 pci dev 44:0 using modern (1.0) virtio mode ERROR: queue already active fail to find vq for virtio-blk 44:0 found virtio-blk at 47:0 pci dev 47:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 47:0 found virtio-blk at 48:0 pci dev 48:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 48:0 found virtio-blk at 49:0 pci dev 49:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 49:0 found virtio-blk at 4a:0 pci dev 4a:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 4a:0 found virtio-blk at 4b:0 pci dev 4b:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 4b:0 found virtio-blk at 4c:0 pci dev 4c:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 4c:0 found virtio-blk at 4d:0 pci dev 4d:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 4d:0 found virtio-blk at 4e:0 pci dev 4e:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 4e:0 found virtio-blk at 4f:0 pci dev 4f:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 4f:0 found virtio-blk at 50:0 pci dev 50:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 50:0 found virtio-blk at 51:0 pci dev 51:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 51:0 found virtio-blk at 52:0 pci dev 52:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 52:0 found virtio-blk at 53:0 pci dev 53:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 53:0 found virtio-blk at 54:0 pci dev 54:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 54:0 found virtio-blk at 55:0 pci dev 55:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 55:0 found virtio-blk at 56:0 pci dev 56:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 56:0 found virtio-blk at 57:0 pci dev 57:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 57:0 found virtio-blk at 58:0 pci dev 58:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 58:0 found virtio-blk at 59:0 pci dev 59:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 59:0 found virtio-blk at 5a:0 pci dev 5a:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 5a:0 found virtio-blk at 5b:0 pci dev 5b:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 5b:0 found virtio-blk at 5c:0 pci dev 5c:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 5c:0 found virtio-blk at 5d:0 pci dev 5d:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 5d:0 found virtio-blk at 5e:0 pci dev 5e:0 using modern (1.0) virtio mode ERROR: queue size is 0 fail to find vq for virtio-blk 5e:0 All threads complete. Scan for option roms Running option rom at c980:0003 pmm call arg1=1 pmm call arg1=0 pmm call arg1=1 pmm call arg1=0 Searching bootorder for: /pci@i0cf8/*@2 Searching bootorder for: /rom@genroms/kvmvapic.bin Press ESC for boot menu. Expected results: guest boot up and seabios didn't have error message Additional info: Detailed seabios log please check the attachment
I've already opened a BZ for exactly the same issue: https://bugzilla.redhat.com/show_bug.cgi?id=1324959 Please try to reproduce without Q35 or switches, just by having the same number of virtio-blk-pci devices, then I suggest to close one of the BZs. Thank you, Marcel
(In reply to Marcel Apfelbaum from comment #2) > I've already opened a BZ for exactly the same issue: > https://bugzilla.redhat.com/show_bug.cgi?id=1324959 > > Please try to reproduce without Q35 or switches, just by having the same > number of virtio-blk-pci devices, then I suggest to close one of the BZs. > > Thank you, > Marcel Hi Marcel 1. only hit the warning issue (WARNING - Unable to allocate resource at init_virtio_blk:104!) when boot with "sh multi-pcie.sh 14" 2.reproduce without Q35, and hit the warning issue, no fail issue. Following is the cli： #!/bin/sh MACHINE=pc SMP=4,cores=2,threads=2,sockets=1 MEM=2G GUEST_IMG=/home/rhel.img IMG_FORMAT=qcow2 CLI="gdb --args /usr/libexec/qemu-kvm -enable-kvm -M $MACHINE -smp $SMP -m $MEM -name vm1 -drive file=$GUEST_IMG,if=none,id=guest-img,format=$IMG_FORMAT,werror=stop,rerror=stop -device ide-hd,drive=guest-img,bus=ide.0,unit=0,id=os-disk,bootindex=0 -spice port=5931,disable-ticketing -vga qxl -monitor stdio -serial unix:/tmp/console,server,nowait -qmp tcp:0:6666,server,nowait -chardev file,path=/home/seabios.log,id=seabios -device isa-debugcon,chardev=seabios,iobase=0x402 -boot menu=on,reboot-timeout=8,strict=on" echo before juli while [ ${i:=0} -lt ${1:-0} ] do dstreamId=$((i)) blkDiskId=$((dstreamId)) qemu-img create -f qcow2 /home/disk/disk$blkDiskId 100M echo "juli$i" CLI="$CLI -drive file=/home/disk/disk$blkDiskId,if=none,id=disk$blkDiskId,format=qcow2 " CLI="$CLI -device virtio-blk-pci,scsi=off,drive=disk$blkDiskId,id=virtio-blk$blkDiskId" i=$((i+1)) done $CLI And the seabios log: Searching bootorder for: /pci@i0cf8/*@e found virtio-blk at 0:f WARNING - Unable to allocate resource at init_virtio_blk:104! found virtio-blk at 0:10 WARNING - Unable to allocate resource at init_virtio_blk:104! found virtio-blk at 0:11 WARNING - Unable to allocate resource at init_virtio_blk:104! WARNING - Unable to allocate resource at init_atadrive:715!
(In reply to jingzhao from comment #3) > (In reply to Marcel Apfelbaum from comment #2) > > I've already opened a BZ for exactly the same issue: > > https://bugzilla.redhat.com/show_bug.cgi?id=1324959 > > > > Please try to reproduce without Q35 or switches, just by having the same > > number of virtio-blk-pci devices, then I suggest to close one of the BZs. > > > > Thank you, > > Marcel > > Hi Marcel > 1. only hit the warning issue (WARNING - Unable to allocate resource at > init_virtio_blk:104!) when boot with "sh multi-pcie.sh 14" > > 2.reproduce without Q35, and hit the warning issue, no fail issue. Hi, I understand, but did you use disable-modern=off,disable-legacy=on for Q35? Once you use the above parameters for virtio-blk-pci, it should come to the same outcome on both machines. Can you please try? Thanks, Marcel
(In reply to Marcel Apfelbaum from comment #4) > (In reply to jingzhao from comment #3) > > (In reply to Marcel Apfelbaum from comment #2) > > > I've already opened a BZ for exactly the same issue: > > > https://bugzilla.redhat.com/show_bug.cgi?id=1324959 > > > > > > Please try to reproduce without Q35 or switches, just by having the same > > > number of virtio-blk-pci devices, then I suggest to close one of the BZs. > > > > > > Thank you, > > > Marcel > > > > Hi Marcel > > 1. only hit the warning issue (WARNING - Unable to allocate resource at > > init_virtio_blk:104!) when boot with "sh multi-pcie.sh 14" > > > > 2.reproduce without Q35, and hit the warning issue, no fail issue. > > Hi, > > I understand, but did you use disable-modern=off,disable-legacy=on for Q35? ---Yes, I used the above parameters for Q35 > Once you use the above parameters for virtio-blk-pci, it should come to the > same outcome on both machines. > Can you please try? ----Yes, the same outcome on pc and q35 once used the above parameters(disable-modern=off,disable-legacy=on for Q35 and disable-modern=on,disable-legacy=off for pc machine type) > > Thanks, > Marcel
*** This bug has been marked as a duplicate of bug 1324959 ***