Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1038375

Summary: qemu will core dumped when disable PCI option ROM with rombar=0 for vfio-pci (Broadcom BCM57810 card)
Product: Red Hat Enterprise Linux 7 Reporter: Sibiao Luo <sluo>
Component: qemu-kvmAssignee: Bandan Das <bdas>
Status: CLOSED CURRENTRELEASE QA Contact: Virtualization Bugs <virt-bugs>
Severity: high Docs Contact:
Priority: medium    
Version: 7.0CC: acathrow, alex.williamson, chayang, hhuang, juzhang, michen, qzhang, sluo, virt-maint, xfu
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-12-16 04:40:48 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Sibiao Luo 2013-12-05 02:19:52 UTC 
Description of problem:
boot guest with a PF vfio-pci assgined and specified rombar=0, qemu will quit and core qumped.
BTW, also tried if specified rombar=1 that QEMU work well.

Version-Release number of selected component (if applicable):
host info:
3.10.0-57.el7.x86_64
qemu-kvm-1.5.3-20.el7.x86_64
seabios-1.7.2.2-4.el7.x86_64
guest info:
3.10.0-57.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1.Load vfio-pci, vfio, vfio_iommu_type1 modules.
# lsmod | grep vfio
vfio_pci               36474  0 
vfio_iommu_type1       17636  0 
vfio                   20777  2 vfio_iommu_type1,vfio_pci

2.Check what other devices are in the same group as PF, unbind all of them and bind to vfio-pci.
# lspci | grep -i BCM57810
08:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM57810 10 Gigabit Ethernet (rev 10)
08:00.1 Ethernet controller: Broadcom Corporation NetXtreme II BCM57810 10 Gigabit Ethernet (rev 10)
# readlink /sys/bus/pci/devices/0000:08:00.0/iommu_group
../../../../kernel/iommu_groups/14
# readlink /sys/bus/pci/devices/0000:08:00.1/iommu_group
../../../../kernel/iommu_groups/14
# lspci -n -s 0000:08:00.0 | awk '{ print $3 }'
14e4:168e
# echo "14e4 168e" > /sys/bus/pci/drivers/vfio-pci/new_id
# echo 0000:08:00.0 > /sys/bus/pci/devices/0000\:08\:00.0/driver/unbind 
# echo 0000:08:00.0 > /sys/bus/pci/drivers/vfio-pci/bind
# lspci -n -s 0000:08:00.1 | awk '{ print $3 }'
14e4:168e
# echo "14e4 168e" >> /sys/bus/pci/drivers/vfio-pci/new_id
# echo 0000:08:00.1 >> /sys/bus/pci/devices/0000\:08\:00.1/driver/unbind 
# echo 0000:08:00.1 >> /sys/bus/pci/drivers/vfio-pci/bind

3.Do vfio-pci with PF to guest and specified rombar=0.
e.g:...-device vfio-pci,host=08:00.0,id=sluo_guest_nic_pf0,rombar=0

Actual results:
after step 3, qemu quit and core dumped.
QEMU 1.5.3 monitor - type 'help' for more information
(qemu) c
(qemu) Segmentation fault (core dumped)

Core was generated by `/usr/libexec/qemu-kvm -M pc -S -cpu host -enable-kvm -m 2048 -smp 2,sockets=2,c'.
Program terminated with signal 11, Segmentation fault.
#0  0x00007f8a8bfcbb7e in qemu_set_irq (irq=0x7f, level=0) at hw/core/irq.c:38
38	    irq->handler(irq->opaque, irq->n, level);
(gdb) bt
#0  0x00007f8a8bfcbb7e in qemu_set_irq (irq=0x7f, level=0) at hw/core/irq.c:38
#1  0x00007f8a8c10497b in vfio_disable_intx_kvm (vdev=vdev@entry=0x7f8a8e89f180)
    at /usr/src/debug/qemu-1.5.3/hw/misc/vfio.c:434
#2  0x00007f8a8c105b4b in vfio_update_irq (pdev=0x7f8a8e89f180) at /usr/src/debug/qemu-1.5.3/hw/misc/vfio.c:477
#3  0x00007f8a8c01643b in pci_bus_fire_intx_routing_notifier (bus=0x7f8a8e808ba0) at hw/pci/pci.c:1236
#4  0x00007f8a8c010115 in piix3_write_config (dev=0x7f8a8e80ae60, address=<optimized out>, val=<optimized out>, 
    len=<optimized out>) at hw/pci-host/piix.c:408
#5  0x00007f8a8c11d252 in access_with_adjusted_size (addr=addr@entry=2, value=value@entry=0x7f8a7e58bb58, size=1, 
    access_size_min=<optimized out>, access_size_max=<optimized out>, 
    access=access@entry=0x7f8a8c11d810 <memory_region_write_accessor>, opaque=opaque@entry=0x7f8a8e808a28)
    at /usr/src/debug/qemu-1.5.3/memory.c:364
#6  0x00007f8a8c11e727 in memory_region_iorange_write (iorange=<optimized out>, offset=2, width=1, data=139)
    at /usr/src/debug/qemu-1.5.3/memory.c:439
#7  0x00007f8a8c11bdf2 in kvm_handle_io (count=1, size=1, direction=1, data=<optimized out>, port=3326)
    at /usr/src/debug/qemu-1.5.3/kvm-all.c:1497
#8  kvm_cpu_exec (env=env@entry=0x7f8a8e7edea0) at /usr/src/debug/qemu-1.5.3/kvm-all.c:1649
#9  0x00007f8a8c0c72c5 in qemu_kvm_cpu_thread_fn (arg=0x7f8a8e7edea0) at /usr/src/debug/qemu-1.5.3/cpus.c:793
#10 0x00007f8a89ed7de3 in start_thread () from /lib64/libpthread.so.0
#11 0x00007f8a86fce26d in clone () from /lib64/libc.so.6
(gdb) bt full
#0  0x00007f8a8bfcbb7e in qemu_set_irq (irq=0x7f, level=0) at hw/core/irq.c:38
No locals.
#1  0x00007f8a8c10497b in vfio_disable_intx_kvm (vdev=vdev@entry=0x7f8a8e89f180)
    at /usr/src/debug/qemu-1.5.3/hw/misc/vfio.c:434
        irqfd = {fd = 25, gsi = 11, flags = 1, resamplefd = 0, pad = '\000' <repeats 15 times>}
#2  0x00007f8a8c105b4b in vfio_update_irq (pdev=0x7f8a8e89f180) at /usr/src/debug/qemu-1.5.3/hw/misc/vfio.c:477
        vdev = 0x7f8a8e89f180
        route = {mode = PCI_INTX_DISABLED, irq = -1}
#3  0x00007f8a8c01643b in pci_bus_fire_intx_routing_notifier (bus=0x7f8a8e808ba0) at hw/pci/pci.c:1236
        dev = <optimized out>
        sec = <optimized out>
#4  0x00007f8a8c010115 in piix3_write_config (dev=0x7f8a8e80ae60, address=<optimized out>, val=<optimized out>, 
    len=<optimized out>) at hw/pci-host/piix.c:408
        piix3 = 0x7f8a8e80ae60
        address = <optimized out>
        val = <optimized out>
        len = <optimized out>
        dev = 0x7f8a8e80ae60
#5  0x00007f8a8c11d252 in access_with_adjusted_size (addr=addr@entry=2, value=value@entry=0x7f8a7e58bb58, size=1, 
    access_size_min=<optimized out>, access_size_max=<optimized out>, 
    access=access@entry=0x7f8a8c11d810 <memory_region_write_accessor>, opaque=opaque@entry=0x7f8a8e808a28)
    at /usr/src/debug/qemu-1.5.3/memory.c:364
        access_mask = 255
        access_size = 1
        i = <optimized out>
#6  0x00007f8a8c11e727 in memory_region_iorange_write (iorange=<optimized out>, offset=2, width=1, data=139)
    at /usr/src/debug/qemu-1.5.3/memory.c:439
        mrio = <optimized out>
        mr = 0x7f8a8e808a28
        __PRETTY_FUNCTION__ = "memory_region_iorange_write"
#7  0x00007f8a8c11bdf2 in kvm_handle_io (count=1, size=1, direction=1, data=<optimized out>, port=3326)
    at /usr/src/debug/qemu-1.5.3/kvm-all.c:1497
        i = 0
        ptr = 0x7f8a8be99000 <Address 0x7f8a8be99000 out of bounds>
#8  kvm_cpu_exec (env=env@entry=0x7f8a8e7edea0) at /usr/src/debug/qemu-1.5.3/kvm-all.c:1649
        cpu = 0x7f8a8e7edd90
        __func__ = "kvm_cpu_exec"
        run = 0x7f8a8be98000
        ret = <optimized out>
        run_ret = <optimized out>
#9  0x00007f8a8c0c72c5 in qemu_kvm_cpu_thread_fn (arg=0x7f8a8e7edea0) at /usr/src/debug/qemu-1.5.3/cpus.c:793
        cpu = 0x7f8a8e7edd90
        __func__ = "qemu_kvm_cpu_thread_fn"
        r = <optimized out>
#10 0x00007f8a89ed7de3 in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#11 0x00007f8a86fce26d in clone () from /lib64/libc.so.6
No symbol table info available.
(gdb)

Expected results:
it should no any qemu core dumped.

Additional info:
# /usr/libexec/qemu-kvm -M pc -S -cpu host -enable-kvm -m 2048 -smp 2,sockets=2,cores=1,threads=1 -no-kvm-pit-reinjection -usb -device usb-tablet,id=input0 -name sluo -uuid 990ea161-6b67-47b2-b803-19fb01d30d30 -rtc base=localtime,clock=host,driftfix=slew -device virtio-serial-pci,id=virtio-serial0,max_ports=16,vectors=0,bus=pci.0,addr=0x3 -chardev socket,id=channel1,path=/tmp/helloworld1,server,nowait -device virtserialport,chardev=channel1,name=com.redhat.rhevm.vdsm,bus=virtio-serial0.0,id=port1 -chardev socket,id=channel2,path=/tmp/helloworld2,server,nowait -device virtserialport,chardev=channel2,name=com.redhat.rhevm.vdsm,bus=virtio-serial0.0,id=port2 -drive file=/home/RHEL-7.0-20131127.1_Server_x86_64.qcow2,if=none,id=drive-disk,cache=none,format=qcow2,aio=native,werror=stop,rerror=stop -device virtio-blk-pci,vectors=0,bus=pci.0,addr=0x4,scsi=off,drive=drive-disk,id=system-disk,bootindex=1 -net none -global PIIX4_PM.disable_s3=0 -global PIIX4_PM.disable_s4=0 -k en-us -boot menu=on -qmp tcp:0:4444,server,nowait -serial unix:/tmp/ttyS0,server,nowait -vnc :1 -spice disable-ticketing,port=5931 -monitor stdio -device vfio-pci,host=08:00.0,id=sluo_guest_nic_pf0,rombar=0
Comment 1 Bandan Das 2013-12-11 23:30:52 UTC 
(In reply to Sibiao Luo from comment #0)
> Description of problem:
> boot guest with a PF vfio-pci assgined and specified rombar=0, qemu will
> quit and core qumped.
> BTW, also tried if specified rombar=1 that QEMU work well.
> 
I am not able to reproduce this with qemu-kvm-1.5.3-21. I would actually expect it to not work with rombar=1 due to reasons mentioned in bug 1005629. What's the firmware version on the Broadcom card ?
Comment 2 Sibiao Luo 2013-12-12 02:15:13 UTC 
(In reply to Bandan Das from comment #1)
> (In reply to Sibiao Luo from comment #0)
> > Description of problem:
> > boot guest with a PF vfio-pci assgined and specified rombar=0, qemu will
> > quit and core qumped.
> > BTW, also tried if specified rombar=1 that QEMU work well.
> > 
> I am not able to reproduce this with qemu-kvm-1.5.3-21. I would actually
> expect it to not work with rombar=1 due to reasons mentioned in bug 1005629.
> What's the firmware version on the Broadcom card ?
Boot guest with a PF vfio-pci assgined and specified rombar=1 that QEMU work well and guest can boot up successfully. Just specified rombar=0 will core dumped.

My firmware-version is bc 7.8.22.
# ethtool -i p2p2
driver: bnx2x
version: 1.78.17-0
firmware-version: bc 7.8.22
bus-info: 0000:08:00.1
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes

# modinfo be2net
filename:       /lib/modules/3.10.0-57.el7.x86_64/kernel/drivers/net/ethernet/emulex/benet/be2net.ko
license:        GPL
author:         Emulex Corporation
description:    Emulex OneConnect 10Gbps NIC Driver 4.9.224.0r
version:        4.9.224.0r
srcversion:     0D545EF0B52BCC867E4168E
alias:          pci:v000010DFd00000728sv*sd*bc*sc*i*
alias:          pci:v000010DFd00000720sv*sd*bc*sc*i*
alias:          pci:v000010DFd0000E228sv*sd*bc*sc*i*
alias:          pci:v000010DFd0000E220sv*sd*bc*sc*i*
alias:          pci:v000019A2d00000710sv*sd*bc*sc*i*
alias:          pci:v000019A2d00000700sv*sd*bc*sc*i*
alias:          pci:v000019A2d00000221sv*sd*bc*sc*i*
alias:          pci:v000019A2d00000211sv*sd*bc*sc*i*
depends:        
intree:         Y
vermagic:       3.10.0-57.el7.x86_64 SMP mod_unload modversions 
signer:         Red Hat Enterprise Linux kernel signing key
sig_key:        15:56:8C:5B:A2:B3:4B:15:2F:B4:AE:B1:B5:1B:F7:AA:25:05:11:78
sig_hashalgo:   sha256
parm:           num_vfs:Number of PCI VFs to initialize (uint)
parm:           rx_frag_size:Size of a fragment that holds rcvd data. (ushort)