| Summary: | qemu will core dumped when disable PCI option ROM with rombar=0 for vfio-pci (Broadcom BCM57810 card) | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Sibiao Luo <sluo> |
| Component: | qemu-kvm | Assignee: | Bandan Das <bdas> |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Virtualization Bugs <virt-bugs> |
| Severity: | high | Docs Contact: | |
| Priority: | medium | ||
| Version: | 7.0 | CC: | acathrow, alex.williamson, chayang, hhuang, juzhang, michen, qzhang, sluo, virt-maint, xfu |
| Target Milestone: | rc | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2013-12-16 04:40:48 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
(In reply to Sibiao Luo from comment #0) > Description of problem: > boot guest with a PF vfio-pci assgined and specified rombar=0, qemu will > quit and core qumped. > BTW, also tried if specified rombar=1 that QEMU work well. > I am not able to reproduce this with qemu-kvm-1.5.3-21. I would actually expect it to not work with rombar=1 due to reasons mentioned in bug 1005629. What's the firmware version on the Broadcom card ? (In reply to Bandan Das from comment #1) > (In reply to Sibiao Luo from comment #0) > > Description of problem: > > boot guest with a PF vfio-pci assgined and specified rombar=0, qemu will > > quit and core qumped. > > BTW, also tried if specified rombar=1 that QEMU work well. > > > I am not able to reproduce this with qemu-kvm-1.5.3-21. I would actually > expect it to not work with rombar=1 due to reasons mentioned in bug 1005629. > What's the firmware version on the Broadcom card ? Boot guest with a PF vfio-pci assgined and specified rombar=1 that QEMU work well and guest can boot up successfully. Just specified rombar=0 will core dumped. My firmware-version is bc 7.8.22. # ethtool -i p2p2 driver: bnx2x version: 1.78.17-0 firmware-version: bc 7.8.22 bus-info: 0000:08:00.1 supports-statistics: yes supports-test: yes supports-eeprom-access: yes supports-register-dump: yes supports-priv-flags: yes # modinfo be2net filename: /lib/modules/3.10.0-57.el7.x86_64/kernel/drivers/net/ethernet/emulex/benet/be2net.ko license: GPL author: Emulex Corporation description: Emulex OneConnect 10Gbps NIC Driver 4.9.224.0r version: 4.9.224.0r srcversion: 0D545EF0B52BCC867E4168E alias: pci:v000010DFd00000728sv*sd*bc*sc*i* alias: pci:v000010DFd00000720sv*sd*bc*sc*i* alias: pci:v000010DFd0000E228sv*sd*bc*sc*i* alias: pci:v000010DFd0000E220sv*sd*bc*sc*i* alias: pci:v000019A2d00000710sv*sd*bc*sc*i* alias: pci:v000019A2d00000700sv*sd*bc*sc*i* alias: pci:v000019A2d00000221sv*sd*bc*sc*i* alias: pci:v000019A2d00000211sv*sd*bc*sc*i* depends: intree: Y vermagic: 3.10.0-57.el7.x86_64 SMP mod_unload modversions signer: Red Hat Enterprise Linux kernel signing key sig_key: 15:56:8C:5B:A2:B3:4B:15:2F:B4:AE:B1:B5:1B:F7:AA:25:05:11:78 sig_hashalgo: sha256 parm: num_vfs:Number of PCI VFs to initialize (uint) parm: rx_frag_size:Size of a fragment that holds rcvd data. (ushort) |
Description of problem: boot guest with a PF vfio-pci assgined and specified rombar=0, qemu will quit and core qumped. BTW, also tried if specified rombar=1 that QEMU work well. Version-Release number of selected component (if applicable): host info: 3.10.0-57.el7.x86_64 qemu-kvm-1.5.3-20.el7.x86_64 seabios-1.7.2.2-4.el7.x86_64 guest info: 3.10.0-57.el7.x86_64 How reproducible: 100% Steps to Reproduce: 1.Load vfio-pci, vfio, vfio_iommu_type1 modules. # lsmod | grep vfio vfio_pci 36474 0 vfio_iommu_type1 17636 0 vfio 20777 2 vfio_iommu_type1,vfio_pci 2.Check what other devices are in the same group as PF, unbind all of them and bind to vfio-pci. # lspci | grep -i BCM57810 08:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM57810 10 Gigabit Ethernet (rev 10) 08:00.1 Ethernet controller: Broadcom Corporation NetXtreme II BCM57810 10 Gigabit Ethernet (rev 10) # readlink /sys/bus/pci/devices/0000:08:00.0/iommu_group ../../../../kernel/iommu_groups/14 # readlink /sys/bus/pci/devices/0000:08:00.1/iommu_group ../../../../kernel/iommu_groups/14 # lspci -n -s 0000:08:00.0 | awk '{ print $3 }' 14e4:168e # echo "14e4 168e" > /sys/bus/pci/drivers/vfio-pci/new_id # echo 0000:08:00.0 > /sys/bus/pci/devices/0000\:08\:00.0/driver/unbind # echo 0000:08:00.0 > /sys/bus/pci/drivers/vfio-pci/bind # lspci -n -s 0000:08:00.1 | awk '{ print $3 }' 14e4:168e # echo "14e4 168e" >> /sys/bus/pci/drivers/vfio-pci/new_id # echo 0000:08:00.1 >> /sys/bus/pci/devices/0000\:08\:00.1/driver/unbind # echo 0000:08:00.1 >> /sys/bus/pci/drivers/vfio-pci/bind 3.Do vfio-pci with PF to guest and specified rombar=0. e.g:...-device vfio-pci,host=08:00.0,id=sluo_guest_nic_pf0,rombar=0 Actual results: after step 3, qemu quit and core dumped. QEMU 1.5.3 monitor - type 'help' for more information (qemu) c (qemu) Segmentation fault (core dumped) Core was generated by `/usr/libexec/qemu-kvm -M pc -S -cpu host -enable-kvm -m 2048 -smp 2,sockets=2,c'. Program terminated with signal 11, Segmentation fault. #0 0x00007f8a8bfcbb7e in qemu_set_irq (irq=0x7f, level=0) at hw/core/irq.c:38 38 irq->handler(irq->opaque, irq->n, level); (gdb) bt #0 0x00007f8a8bfcbb7e in qemu_set_irq (irq=0x7f, level=0) at hw/core/irq.c:38 #1 0x00007f8a8c10497b in vfio_disable_intx_kvm (vdev=vdev@entry=0x7f8a8e89f180) at /usr/src/debug/qemu-1.5.3/hw/misc/vfio.c:434 #2 0x00007f8a8c105b4b in vfio_update_irq (pdev=0x7f8a8e89f180) at /usr/src/debug/qemu-1.5.3/hw/misc/vfio.c:477 #3 0x00007f8a8c01643b in pci_bus_fire_intx_routing_notifier (bus=0x7f8a8e808ba0) at hw/pci/pci.c:1236 #4 0x00007f8a8c010115 in piix3_write_config (dev=0x7f8a8e80ae60, address=<optimized out>, val=<optimized out>, len=<optimized out>) at hw/pci-host/piix.c:408 #5 0x00007f8a8c11d252 in access_with_adjusted_size (addr=addr@entry=2, value=value@entry=0x7f8a7e58bb58, size=1, access_size_min=<optimized out>, access_size_max=<optimized out>, access=access@entry=0x7f8a8c11d810 <memory_region_write_accessor>, opaque=opaque@entry=0x7f8a8e808a28) at /usr/src/debug/qemu-1.5.3/memory.c:364 #6 0x00007f8a8c11e727 in memory_region_iorange_write (iorange=<optimized out>, offset=2, width=1, data=139) at /usr/src/debug/qemu-1.5.3/memory.c:439 #7 0x00007f8a8c11bdf2 in kvm_handle_io (count=1, size=1, direction=1, data=<optimized out>, port=3326) at /usr/src/debug/qemu-1.5.3/kvm-all.c:1497 #8 kvm_cpu_exec (env=env@entry=0x7f8a8e7edea0) at /usr/src/debug/qemu-1.5.3/kvm-all.c:1649 #9 0x00007f8a8c0c72c5 in qemu_kvm_cpu_thread_fn (arg=0x7f8a8e7edea0) at /usr/src/debug/qemu-1.5.3/cpus.c:793 #10 0x00007f8a89ed7de3 in start_thread () from /lib64/libpthread.so.0 #11 0x00007f8a86fce26d in clone () from /lib64/libc.so.6 (gdb) bt full #0 0x00007f8a8bfcbb7e in qemu_set_irq (irq=0x7f, level=0) at hw/core/irq.c:38 No locals. #1 0x00007f8a8c10497b in vfio_disable_intx_kvm (vdev=vdev@entry=0x7f8a8e89f180) at /usr/src/debug/qemu-1.5.3/hw/misc/vfio.c:434 irqfd = {fd = 25, gsi = 11, flags = 1, resamplefd = 0, pad = '\000' <repeats 15 times>} #2 0x00007f8a8c105b4b in vfio_update_irq (pdev=0x7f8a8e89f180) at /usr/src/debug/qemu-1.5.3/hw/misc/vfio.c:477 vdev = 0x7f8a8e89f180 route = {mode = PCI_INTX_DISABLED, irq = -1} #3 0x00007f8a8c01643b in pci_bus_fire_intx_routing_notifier (bus=0x7f8a8e808ba0) at hw/pci/pci.c:1236 dev = <optimized out> sec = <optimized out> #4 0x00007f8a8c010115 in piix3_write_config (dev=0x7f8a8e80ae60, address=<optimized out>, val=<optimized out>, len=<optimized out>) at hw/pci-host/piix.c:408 piix3 = 0x7f8a8e80ae60 address = <optimized out> val = <optimized out> len = <optimized out> dev = 0x7f8a8e80ae60 #5 0x00007f8a8c11d252 in access_with_adjusted_size (addr=addr@entry=2, value=value@entry=0x7f8a7e58bb58, size=1, access_size_min=<optimized out>, access_size_max=<optimized out>, access=access@entry=0x7f8a8c11d810 <memory_region_write_accessor>, opaque=opaque@entry=0x7f8a8e808a28) at /usr/src/debug/qemu-1.5.3/memory.c:364 access_mask = 255 access_size = 1 i = <optimized out> #6 0x00007f8a8c11e727 in memory_region_iorange_write (iorange=<optimized out>, offset=2, width=1, data=139) at /usr/src/debug/qemu-1.5.3/memory.c:439 mrio = <optimized out> mr = 0x7f8a8e808a28 __PRETTY_FUNCTION__ = "memory_region_iorange_write" #7 0x00007f8a8c11bdf2 in kvm_handle_io (count=1, size=1, direction=1, data=<optimized out>, port=3326) at /usr/src/debug/qemu-1.5.3/kvm-all.c:1497 i = 0 ptr = 0x7f8a8be99000 <Address 0x7f8a8be99000 out of bounds> #8 kvm_cpu_exec (env=env@entry=0x7f8a8e7edea0) at /usr/src/debug/qemu-1.5.3/kvm-all.c:1649 cpu = 0x7f8a8e7edd90 __func__ = "kvm_cpu_exec" run = 0x7f8a8be98000 ret = <optimized out> run_ret = <optimized out> #9 0x00007f8a8c0c72c5 in qemu_kvm_cpu_thread_fn (arg=0x7f8a8e7edea0) at /usr/src/debug/qemu-1.5.3/cpus.c:793 cpu = 0x7f8a8e7edd90 __func__ = "qemu_kvm_cpu_thread_fn" r = <optimized out> #10 0x00007f8a89ed7de3 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. #11 0x00007f8a86fce26d in clone () from /lib64/libc.so.6 No symbol table info available. (gdb) Expected results: it should no any qemu core dumped. Additional info: # /usr/libexec/qemu-kvm -M pc -S -cpu host -enable-kvm -m 2048 -smp 2,sockets=2,cores=1,threads=1 -no-kvm-pit-reinjection -usb -device usb-tablet,id=input0 -name sluo -uuid 990ea161-6b67-47b2-b803-19fb01d30d30 -rtc base=localtime,clock=host,driftfix=slew -device virtio-serial-pci,id=virtio-serial0,max_ports=16,vectors=0,bus=pci.0,addr=0x3 -chardev socket,id=channel1,path=/tmp/helloworld1,server,nowait -device virtserialport,chardev=channel1,name=com.redhat.rhevm.vdsm,bus=virtio-serial0.0,id=port1 -chardev socket,id=channel2,path=/tmp/helloworld2,server,nowait -device virtserialport,chardev=channel2,name=com.redhat.rhevm.vdsm,bus=virtio-serial0.0,id=port2 -drive file=/home/RHEL-7.0-20131127.1_Server_x86_64.qcow2,if=none,id=drive-disk,cache=none,format=qcow2,aio=native,werror=stop,rerror=stop -device virtio-blk-pci,vectors=0,bus=pci.0,addr=0x4,scsi=off,drive=drive-disk,id=system-disk,bootindex=1 -net none -global PIIX4_PM.disable_s3=0 -global PIIX4_PM.disable_s4=0 -k en-us -boot menu=on -qmp tcp:0:4444,server,nowait -serial unix:/tmp/ttyS0,server,nowait -vnc :1 -spice disable-ticketing,port=5931 -monitor stdio -device vfio-pci,host=08:00.0,id=sluo_guest_nic_pf0,rombar=0