Hide Forgot
Description of problem: qemu-kvm core dump when boot with 128 virtio-blk-pci data disks via PCIe-switch. when boot with virtio-blk-pci data disks between 128 and 232, hit this issue too. BTW, boot guest with system disk not under pcie-switch downstream. So this is different with bug 987322 and bug 1055416. In this bug, can see seabios GUI and can boot guest. When booting guest kernel, qemu-kvm core dump. Version-Release number of selected component (if applicable): 3.10.0-67.el7.x86_64 qemu-kvm-rhev-1.5.3-37.el7.x86_64 seabios-bin-1.7.2.2-6.el7.x86_64 How reproducible: 100% Steps to Reproduce: 1.boot guest as the following via multi-pcie.sh shell script. # ./multi-pcie.sh 128 # cat script/multi-pcie.sh #!/bin/sh MACHINE=q35 SMP=4,cores=2,threads=2,sockets=1 MEM=2G GUEST_IMG=/home/juli/rhel7_64.raw IMG_FORMAT=raw CLI="gdb --args /usr/libexec/qemu-kvm -enable-kvm -M $MACHINE -smp $SMP -m $MEM -name vm1 -drive file=$GUEST_IMG,if=none,id=guest-img,format=$IMG_FORMAT,werror=stop,rerror=stop -device ide-hd,drive=guest-img,bus=ide.0,unit=0,id=os-disk,bootindex=1 -spice port=5931,disable-ticketing -vga qxl -monitor stdio -qmp tcp:0:6666,server,nowait -boot menu=on,reboot-timeout=8,strict=on -chardev stdio,id=log -device isa-debugcon,iobase=0x402,chardev=log" echo before juli while [ ${i:=0} -lt ${1:-0} ] do dstreamId=$((i)) ustreamId=$((dstreamId/32)) chassisId=$((dstreamId+1)) blkDiskId=$((dstreamId)) if [ $((dstreamId%32)) -eq 0 ] then CLI="$CLI -device ioh3420,bus=pcie.0,id=root.$ustreamId,slot=$ustreamId" CLI="$CLI -device x3130-upstream,bus=root.$ustreamId,id=upstream$ustreamId" fi # qemu-img create -f qcow2 /home/disk/disk$blkDiskId 100M echo "juli$i" CLI="$CLI -device xio3130-downstream,bus=upstream$ustreamId,id=downstream$dstreamId,chassis=$chassisId" CLI="$CLI -drive file=/home/disk/disk$blkDiskId,if=none,id=disk$blkDiskId,format=qcow2 " CLI="$CLI -device virtio-blk-pci,scsi=off,drive=disk$blkDiskId,id=virtio-blk$blkDiskId,bus=downstream$dstreamId" i=$((i+1)) done $CLI ---- 2. 3. Actual results: qemu-kvm core dump. gdb info as followings: qemu-kvm: /builddir/build/BUILD/qemu-1.5.3/exec.c:773: register_subpage: Assertion `existing->mr->subpage || existing->mr == &io_mem_unassigned' failed. Program received signal SIGABRT, Aborted. [Switching to Thread 0x7fffd53b1700 (LWP 95203)] 0x00007ffff2cb2979 in raise () from /lib64/libc.so.6 (gdb) bt #0 0x00007ffff2cb2979 in raise () from /lib64/libc.so.6 #1 0x00007ffff2cb4088 in abort () from /lib64/libc.so.6 #2 0x00007ffff2cab8e6 in __assert_fail_base () from /lib64/libc.so.6 #3 0x00007ffff2cab992 in __assert_fail () from /lib64/libc.so.6 #4 0x000055555578723c in register_subpage () #5 0x0000555555787462 in mem_add () #6 0x00005555557da74c in address_space_update_topology_pass.isra.5 () #7 0x00005555557db2dd in memory_region_transaction_commit () #8 0x00005555556c585c in pci_default_write_config () #9 0x00005555556c8ab9 in pci_bridge_write_config () #10 0x00005555556bf36a in xio3130_downstream_write_config () #11 0x00005555557d8d02 in access_with_adjusted_size () #12 0x00005555557da1d7 in memory_region_iorange_write () #13 0x00005555557d7ae5 in kvm_cpu_exec () #14 0x0000555555780055 in qemu_kvm_cpu_thread_fn () #15 0x00007ffff6055de3 in start_thread () from /lib64/libpthread.so.0 #16 0x00007ffff2d7326d in clone () from /lib64/libc.so.6 Expected results: qemu-kvm works well, no core dump. Additional info:
Can you please test it again with 7.3? (it may be fixed already)
(In reply to Amnon Ilan from comment #6) > Can you please test it again with 7.3? (it may be fixed already) Please provide the RHEL7.3 qemu-kvm-rhev build for QE testing. Best Regards, Junyi
I made one now: https://brewweb.devel.redhat.com/taskinfo?taskID=10560994 Thanks, Marcel
(In reply to Marcel Apfelbaum from comment #8) > I made one now: > > https://brewweb.devel.redhat.com/taskinfo?taskID=10560994 > > Thanks, > Marcel Hi Marcel Didn't reproduced the issue with kernel-3.10.0-350.el7.x86_64 and qemu-kvm-rhev-2.5.0-1.el7.test.x86_64, but hit another issue: guest didn't boot up with 12 virtio-blk-pci data disks via PCIe-switch. The reproduced steps: 1.boot guest as the following via multi-pcie.sh shell script. # ./multi-pcie.sh 12 # cat multi-pcie.sh #!/bin/sh MACHINE=q35 SMP=4,cores=2,threads=2,sockets=1 MEM=2G GUEST_IMG=/home/rhel.img IMG_FORMAT=raw CLI="gdb --args /usr/libexec/qemu-kvm -enable-kvm -M $MACHINE -smp $SMP -m $MEM -name vm1 -drive file=$GUEST_IMG,if=none,id=guest-img,format=$IMG_FORMAT,werror=stop,rerror=stop -device ide-hd,drive=guest-img,bus=ide.0,unit=0,id=os-disk,bootindex=0 -spice port=5931,disable-ticketing -vga qxl -monitor stdio -serial unix:/tmp/console,server,nowait -qmp tcp:0:6666,server,nowait -chardev file,path=/home/seabios.log,id=seabios -device isa-debugcon,chardev=seabios,iobase=0x402 -boot menu=on,reboot-timeout=8,strict=on " echo before juli while [ ${i:=0} -lt ${1:-0} ] do dstreamId=$((i)) ustreamId=$((dstreamId/32)) chassisId=$((dstreamId+1)) blkDiskId=$((dstreamId)) if [ $((dstreamId%32)) -eq 0 ] then CLI="$CLI -device ioh3420,bus=pcie.0,id=root.$ustreamId,slot=$ustreamId" CLI="$CLI -device x3130-upstream,bus=root.$ustreamId,id=upstream$ustreamId" fi qemu-img create -f qcow2 /home/disk/disk$blkDiskId 100M echo "juli$i" CLI="$CLI -device xio3130-downstream,bus=upstream$ustreamId,id=downstream$dstreamId,chassis=$chassisId" CLI="$CLI -drive file=/home/disk/disk$blkDiskId,if=none,id=disk$blkDiskId,format=qcow2 " CLI="$CLI -device virtio-blk-pci,scsi=off,drive=disk$blkDiskId,id=virtio-blk$blkDiskId,bus=downstream$dstreamId" i=$((i+1)) done $CLI Actual result: guest didn't boot up and didn't select the device which boot up, please check the pic through attachment Using isa-debugcon debug, got the following messages: [root@localhost 0226]# cat /home/seabios.log SeaBIOS (version seabios-1.7.5-11.el7) Running on QEMU (q35) Running on KVM RamSize: 0x80000000 [cmos] Relocating init from 0x000e3f59 to 0x7ffb1cd0 (size 57960) Found QEMU fw_cfg RamBlock: addr 0x0000000000000000 len 0x0000000080000000 [e820] Moving pm_base to 0x600 boot order: 1: /pci@i0cf8/pci8086,2922@1f,2/drive@0/disk@0 2: HALT CPU Mhz=3396 === PCI bus & bridge init === PCI: pci_bios_init_bus_rec bus = 0x0 PCI: pci_bios_init_bus_rec bdf = 0x18 PCI: primary bus = 0x0 PCI: secondary bus = 0xff -> 0x1 PCI: pci_bios_init_bus_rec bus = 0x1 PCI: pci_bios_init_bus_rec bdf = 0x100 PCI: primary bus = 0x0 -> 0x1 PCI: secondary bus = 0xff -> 0x2 PCI: pci_bios_init_bus_rec bus = 0x2 PCI: pci_bios_init_bus_rec bdf = 0x200 PCI: primary bus = 0x0 -> 0x2 PCI: secondary bus = 0xff -> 0x3 PCI: pci_bios_init_bus_rec bus = 0x3 PCI: subordinate bus = 0x0 -> 0x3 PCI: pci_bios_init_bus_rec bdf = 0x208 PCI: primary bus = 0x0 -> 0x2 PCI: secondary bus = 0xff -> 0x4 PCI: pci_bios_init_bus_rec bus = 0x4 PCI: subordinate bus = 0x0 -> 0x4 PCI: pci_bios_init_bus_rec bdf = 0x210 PCI: primary bus = 0x0 -> 0x2 PCI: secondary bus = 0xff -> 0x5 PCI: pci_bios_init_bus_rec bus = 0x5 PCI: subordinate bus = 0x0 -> 0x5 PCI: pci_bios_init_bus_rec bdf = 0x218 PCI: primary bus = 0x0 -> 0x2 PCI: secondary bus = 0xff -> 0x6 PCI: pci_bios_init_bus_rec bus = 0x6 PCI: subordinate bus = 0x0 -> 0x6 PCI: pci_bios_init_bus_rec bdf = 0x220 PCI: primary bus = 0x0 -> 0x2 PCI: secondary bus = 0xff -> 0x7 PCI: pci_bios_init_bus_rec bus = 0x7 PCI: subordinate bus = 0x0 -> 0x7 PCI: pci_bios_init_bus_rec bdf = 0x228 PCI: primary bus = 0x0 -> 0x2 PCI: secondary bus = 0xff -> 0x8 PCI: pci_bios_init_bus_rec bus = 0x8 PCI: subordinate bus = 0x0 -> 0x8 PCI: pci_bios_init_bus_rec bdf = 0x230 PCI: primary bus = 0x0 -> 0x2 PCI: secondary bus = 0xff -> 0x9 PCI: pci_bios_init_bus_rec bus = 0x9 PCI: subordinate bus = 0x0 -> 0x9 PCI: pci_bios_init_bus_rec bdf = 0x238 PCI: primary bus = 0x0 -> 0x2 PCI: secondary bus = 0xff -> 0xa PCI: pci_bios_init_bus_rec bus = 0xa PCI: subordinate bus = 0x0 -> 0xa PCI: pci_bios_init_bus_rec bdf = 0x240 PCI: primary bus = 0x0 -> 0x2 PCI: secondary bus = 0xff -> 0xb PCI: pci_bios_init_bus_rec bus = 0xb PCI: subordinate bus = 0x0 -> 0xb PCI: pci_bios_init_bus_rec bdf = 0x248 PCI: primary bus = 0x0 -> 0x2 PCI: secondary bus = 0xff -> 0xc PCI: pci_bios_init_bus_rec bus = 0xc PCI: subordinate bus = 0x0 -> 0xc PCI: pci_bios_init_bus_rec bdf = 0x250 PCI: primary bus = 0x0 -> 0x2 PCI: secondary bus = 0xff -> 0xd PCI: pci_bios_init_bus_rec bus = 0xd PCI: subordinate bus = 0x0 -> 0xd PCI: pci_bios_init_bus_rec bdf = 0x258 PCI: primary bus = 0x0 -> 0x2 PCI: secondary bus = 0xff -> 0xe PCI: pci_bios_init_bus_rec bus = 0xe PCI: subordinate bus = 0x0 -> 0xe PCI: subordinate bus = 0x0 -> 0xe PCI: subordinate bus = 0x0 -> 0xe === PCI device probing === Found 32 PCI devices (max PCI bus is 0e) === PCI new allocation pass #1 === PCI: check devices PCI: secondary bus 14 size 00001000 type io PCI: secondary bus 14 size 00200000 type mem PCI: secondary bus 14 size 00000000 type prefmem PCI: secondary bus 13 size 00001000 type io PCI: secondary bus 13 size 00200000 type mem PCI: secondary bus 13 size 00000000 type prefmem PCI: secondary bus 12 size 00001000 type io PCI: secondary bus 12 size 00200000 type mem PCI: secondary bus 12 size 00000000 type prefmem PCI: secondary bus 11 size 00001000 type io PCI: secondary bus 11 size 00200000 type mem PCI: secondary bus 11 size 00000000 type prefmem PCI: secondary bus 10 size 00001000 type io PCI: secondary bus 10 size 00200000 type mem PCI: secondary bus 10 size 00000000 type prefmem PCI: secondary bus 9 size 00001000 type io PCI: secondary bus 9 size 00200000 type mem PCI: secondary bus 9 size 00000000 type prefmem PCI: secondary bus 8 size 00001000 type io PCI: secondary bus 8 size 00200000 type mem PCI: secondary bus 8 size 00000000 type prefmem PCI: secondary bus 7 size 00001000 type io PCI: secondary bus 7 size 00200000 type mem PCI: secondary bus 7 size 00000000 type prefmem PCI: secondary bus 6 size 00001000 type io PCI: secondary bus 6 size 00200000 type mem PCI: secondary bus 6 size 00000000 type prefmem PCI: secondary bus 5 size 00001000 type io PCI: secondary bus 5 size 00200000 type mem PCI: secondary bus 5 size 00000000 type prefmem PCI: secondary bus 4 size 00001000 type io PCI: secondary bus 4 size 00200000 type mem PCI: secondary bus 4 size 00000000 type prefmem PCI: secondary bus 3 size 00001000 type io PCI: secondary bus 3 size 00200000 type mem PCI: secondary bus 3 size 00000000 type prefmem PCI: secondary bus 2 size 0000c000 type io PCI: secondary bus 2 size 01800000 type mem PCI: secondary bus 2 size 00000000 type prefmem PCI: secondary bus 1 size 0000c000 type io PCI: secondary bus 1 size 01800000 type mem PCI: secondary bus 1 size 00000000 type prefmem === PCI new allocation pass #2 === PCI: IO: 1000 - d0bf PCI: 32: 00000000c0000000 - 00000000fec00000 PCI: map device bdf=00:02.0 bar 1, addr 0000d000, size 00000040 [io] PCI: map device bdf=00:1f.3 bar 4, addr 0000d040, size 00000040 [io] PCI: map device bdf=00:01.0 bar 3, addr 0000d080, size 00000020 [io] PCI: map device bdf=00:1f.2 bar 4, addr 0000d0a0, size 00000020 [io] PCI: map device bdf=00:01.0 bar 0, addr f4000000, size 04000000 [mem] PCI: map device bdf=00:01.0 bar 1, addr f8000000, size 04000000 [mem] PCI: map device bdf=00:02.0 bar 6, addr fd800000, size 00040000 [mem] PCI: map device bdf=00:02.0 bar 0, addr fd840000, size 00020000 [mem] PCI: map device bdf=00:01.0 bar 6, addr fd860000, size 00010000 [mem] PCI: map device bdf=00:01.0 bar 2, addr fd870000, size 00002000 [mem] PCI: map device bdf=00:1f.2 bar 5, addr fd872000, size 00001000 [mem] PCI: map device bdf=03:00.0 bar 0, addr 0000c000, size 00000040 [io] PCI: map device bdf=03:00.0 bar 1, addr fd600000, size 00001000 [mem] PCI: map device bdf=04:00.0 bar 0, addr 0000b000, size 00000040 [io] PCI: map device bdf=04:00.0 bar 1, addr fd400000, size 00001000 [mem] PCI: map device bdf=05:00.0 bar 0, addr 0000a000, size 00000040 [io] PCI: map device bdf=05:00.0 bar 1, addr fd200000, size 00001000 [mem] PCI: map device bdf=06:00.0 bar 0, addr 00009000, size 00000040 [io] PCI: map device bdf=06:00.0 bar 1, addr fd000000, size 00001000 [mem] PCI: map device bdf=07:00.0 bar 0, addr 00008000, size 00000040 [io] PCI: map device bdf=07:00.0 bar 1, addr fce00000, size 00001000 [mem] PCI: map device bdf=08:00.0 bar 0, addr 00007000, size 00000040 [io] PCI: map device bdf=08:00.0 bar 1, addr fcc00000, size 00001000 [mem] PCI: map device bdf=09:00.0 bar 0, addr 00006000, size 00000040 [io] PCI: map device bdf=09:00.0 bar 1, addr fca00000, size 00001000 [mem] PCI: map device bdf=0a:00.0 bar 0, addr 00005000, size 00000040 [io] PCI: map device bdf=0a:00.0 bar 1, addr fc800000, size 00001000 [mem] PCI: map device bdf=0b:00.0 bar 0, addr 00004000, size 00000040 [io] PCI: map device bdf=0b:00.0 bar 1, addr fc600000, size 00001000 [mem] PCI: map device bdf=0c:00.0 bar 0, addr 00003000, size 00000040 [io] PCI: map device bdf=0c:00.0 bar 1, addr fc400000, size 00001000 [mem] PCI: map device bdf=0d:00.0 bar 0, addr 00002000, size 00000040 [io] PCI: map device bdf=0d:00.0 bar 1, addr fc200000, size 00001000 [mem] PCI: map device bdf=0e:00.0 bar 0, addr 00001000, size 00000040 [io] PCI: map device bdf=0e:00.0 bar 1, addr fc000000, size 00001000 [mem] PCI: init bdf=00:00.0 id=8086:29c0 PCI: init bdf=00:01.0 id=1b36:0100 PCI: init bdf=00:02.0 id=8086:100e PCI: init bdf=00:03.0 id=8086:3420 PCI: init bdf=00:1f.0 id=8086:2918 Q35 LPC init: elcr=00 0c Using pmtimer, ioport 0x608 PCI: init bdf=00:1f.2 id=8086:2922 PCI: init bdf=00:1f.3 id=8086:2930 PCI: init bdf=01:00.0 id=104c:8232 PCI: init bdf=02:00.0 id=104c:8233 PCI: init bdf=02:01.0 id=104c:8233 PCI: init bdf=02:02.0 id=104c:8233 PCI: init bdf=02:03.0 id=104c:8233 PCI: init bdf=02:04.0 id=104c:8233 PCI: init bdf=02:05.0 id=104c:8233 PCI: init bdf=02:06.0 id=104c:8233 PCI: init bdf=02:07.0 id=104c:8233 PCI: init bdf=02:08.0 id=104c:8233 PCI: init bdf=02:09.0 id=104c:8233 PCI: init bdf=02:0a.0 id=104c:8233 PCI: init bdf=02:0b.0 id=104c:8233 PCI: init bdf=03:00.0 id=1af4:1001 PCI: init bdf=04:00.0 id=1af4:1001 PCI: init bdf=05:00.0 id=1af4:1001 PCI: init bdf=06:00.0 id=1af4:1001 PCI: init bdf=07:00.0 id=1af4:1001 PCI: init bdf=08:00.0 id=1af4:1001 PCI: init bdf=09:00.0 id=1af4:1001 PCI: init bdf=0a:00.0 id=1af4:1001 PCI: init bdf=0b:00.0 id=1af4:1001 PCI: init bdf=0c:00.0 id=1af4:1001 PCI: init bdf=0d:00.0 id=1af4:1001 PCI: init bdf=0e:00.0 id=1af4:1001 PCI: Using 00:01.0 for primary VGA Found 4 cpu(s) max supported 4 cpu(s) Copying PIR from 0x7ffbfcef to 0x000f2150 Copying MPTABLE from 0x00006e64/7ffa8330 to 0x000f2060 Copying SMBIOS entry point from 0x00006e64 to 0x000f1e70 Scan for VGA option rom Running option rom at c000:0003 Start SeaVGABIOS (version seabios-1.7.5-11.el7) enter vga_post: a=00000008 b=0000ffff c=00000000 d=0000ffff ds=0000 es=f000 ss=0000 si=00000000 di=00007200 bp=00000000 sp=00006dc6 cs=f000 ip=d313 f=0000 VBE DISPI: bdf 00:01.0, bar 0 VBE DISPI: lfb_addr=f4000000, size 16 MB Turning on vga text mode console set VGA mode 3 SeaBIOS (version seabios-1.7.5-11.el7) All threads complete. Found 0 lpt ports Found 1 serial ports AHCI controller at 1f.2, iobase fd872000, irq 10 Searching bootorder for: /pci@i0cf8/*@1f,2/drive@0/disk@0 AHCI/0: registering: "AHCI/0: QEMU HARDDISK ATA-7 Hard-Disk (100 GiBytes)" found virtio-blk at 3:0 Searching bootorder for: /pci@i0cf8/pci-bridge@3/pci-bridge@0/pci-bridge@0/*@0 found virtio-blk at 4:0 Searching bootorder for: /pci@i0cf8/pci-bridge@3/pci-bridge@0/pci-bridge@1/*@0 found virtio-blk at 5:0 Searching bootorder for: /pci@i0cf8/pci-bridge@3/pci-bridge@0/pci-bridge@2/*@0 found virtio-blk at 6:0 Searching bootorder for: /pci@i0cf8/pci-bridge@3/pci-bridge@0/pci-bridge@3/*@0 found virtio-blk at 7:0 Searching bootorder for: /pci@i0cf8/pci-bridge@3/pci-bridge@0/pci-bridge@4/*@0 found virtio-blk at 8:0 Searching bootorder for: /pci@i0cf8/pci-bridge@3/pci-bridge@0/pci-bridge@5/*@0 found virtio-blk at 9:0 Searching bootorder for: /pci@i0cf8/pci-bridge@3/pci-bridge@0/pci-bridge@6/*@0 found virtio-blk at a:0 WARNING - Unable to allocate resource at vp_find_vq:35! fail to find vq for virtio-blk a:0 found virtio-blk at b:0 WARNING - Unable to allocate resource at vp_find_vq:35! fail to find vq for virtio-blk b:0 found virtio-blk at c:0 WARNING - Unable to allocate resource at vp_find_vq:35! fail to find vq for virtio-blk c:0 found virtio-blk at d:0 WARNING - Unable to allocate resource at vp_find_vq:35! fail to find vq for virtio-blk d:0 found virtio-blk at e:0 WARNING - Unable to allocate resource at vp_find_vq:35! fail to find vq for virtio-blk e:0 PS2 keyboard initialized All threads complete. Scan for option roms Running option rom at c900:0003 pmm call arg1=1 pmm call arg1=0 pmm call arg1=1 pmm call arg1=0 Searching bootorder for: /pci@i0cf8/*@2 Searching bootorder for: /rom@genroms/kvmvapic.bin Press ESC for boot menu.
Created attachment 1131787 [details] the screen of seabios
Hi Jin, Can you please close this BZ and open a new one? Thank you, Marcel
Verified it on kernel-3.10.0-470.el7.x86_64 and qemu-img-rhev-2.6.0-14.el7.x86_64
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2016-2673.html