RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1151306 - Qemu coredumpd after reboot a mq enabled but host-tap deleted guest.
Summary: Qemu coredumpd after reboot a mq enabled but host-tap deleted guest.
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: qemu-kvm-rhev
Version: 7.1
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: rc
: ---
Assignee: jason wang
QA Contact: Virtualization Bugs
jason wang
URL:
Whiteboard:
: 1242383 1265903 (view as bug list)
Depends On:
Blocks: 1401400
TreeView+ depends on / blocked
 
Reported: 2014-10-10 05:09 UTC by Qian Guo
Modified: 2016-12-26 05:47 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-12-26 04:29:37 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 1004275 0 medium CLOSED Qemu core dumpd, after deleting the tap in host 2021-02-22 00:41:40 UTC

Internal Links: 1004275

Description Qian Guo 2014-10-10 05:09:28 UTC 
Description of problem:
Eanbled mq in guest, and then delete the tap device in host, then reboot guest, qemu coredumped

Version-Release number of selected component (if applicable):
qemu-kvm-rhev-2.1.2-1.el7.x86_64
# uname -r
3.10.0-183.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1.Boot guest with mq
/usr/libexec/qemu-kvm -cpu SandyBridge -enable-kvm -m 4G -smp 4,sockets=1,cores=4,threads=1 -name test -rtc base=localtime,clock=host,driftfix=slew -k en-us -boot menu=on -spice disable-ticketing,port=5901 -vga qxl -usb -device usb-tablet -monitor stdio -drive file=/home/rhel7u1cp1.qcow2,if=none,id=drive-system-disk,media=disk,format=qcow2,aio=native,werror=stop,rerror=stop -device virtio-blk-pci,drive=drive-system-disk,id=system-disk,addr=0x3 -qmp unix:/tmp/q1,server,nowait -netdev tap,id=hostnet0,vhost=on,queues=4,script=/etc/qemu-ifup -device virtio-net-pci,mq=on,vectors=10,netdev=hostnet0,id=vnet0,mac=52:54:00:1a:2b:02 -serial unix:/tmp/s2,server,nowait

2.In guest enable mq
# ethtool -l eth0
Channel parameters for eth0:
Pre-set maximums:
RX:		0
TX:		0
Other:		0
Combined:	4
Current hardware settings:
RX:		0
TX:		0
Other:		0
Combined:	4


3.Delete tap device in host
# ip link delete tap0

4.Reboot guest.

Actual results:

Then when guest try to boot

qemu coredumped
(gdb) bt
#0  0x00007ffff1ea1989 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1  0x00007ffff1ea3098 in __GI_abort () at abort.c:90
#2  0x00007ffff1e9a8f6 in __assert_fail_base (fmt=0x7ffff1fea3e8 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", 
    assertion=assertion@entry=0x55555589dfbd "!r", 
    file=file@entry=0x55555589dcb8 "/builddir/build/BUILD/qemu-2.1.2/hw/net/virtio-net.c", line=line@entry=437, 
    function=function@entry=0x55555589e5d0 <__PRETTY_FUNCTION__.28404> "virtio_net_set_queues") at assert.c:92
#3  0x00007ffff1e9a9a2 in __GI___assert_fail (assertion=assertion@entry=0x55555589dfbd "!r", 
    file=file@entry=0x55555589dcb8 "/builddir/build/BUILD/qemu-2.1.2/hw/net/virtio-net.c", line=line@entry=437, 
    function=function@entry=0x55555589e5d0 <__PRETTY_FUNCTION__.28404> "virtio_net_set_queues") at assert.c:101
#4  0x000055555567618a in virtio_net_set_queues (n=0x5555563fac38) at /usr/src/debug/qemu-2.1.2/hw/net/virtio-net.c:437
#5  0x00005555556768ab in virtio_net_set_multiqueue (multiqueue=1, n=0x5555563fac38)
    at /usr/src/debug/qemu-2.1.2/hw/net/virtio-net.c:1336
#6  virtio_net_set_features (vdev=<optimized out>, features=955252707) at /usr/src/debug/qemu-2.1.2/hw/net/virtio-net.c:522
#7  0x000055555567e04b in virtio_set_features (vdev=0x5555563fac38, val=955252707)
    at /usr/src/debug/qemu-2.1.2/hw/virtio/virtio.c:971
#8  0x000055555565479a in access_with_adjusted_size (addr=addr@entry=4, value=value@entry=0x7fffe8fbcaf0, size=size@entry=4, 
    access_size_min=<optimized out>, access_size_max=<optimized out>, access=0x555555654910 <memory_region_write_accessor>, 
    mr=0x5555563faaa8) at /usr/src/debug/qemu-2.1.2/memory.c:481
#9  0x0000555555659367 in memory_region_dispatch_write (size=4, data=955252707, addr=4, mr=0x5555563faaa8)
    at /usr/src/debug/qemu-2.1.2/memory.c:1143
#10 io_mem_write (mr=mr@entry=0x5555563faaa8, addr=4, val=<optimized out>, size=4) at /usr/src/debug/qemu-2.1.2/memory.c:1976
#11 0x0000555555624723 in address_space_rw (as=0x555555cb0480 <address_space_io>, addr=addr@entry=49220, 
    buf=0x7ffff7ec1000 "\343\377\357\070", len=len@entry=4, is_write=is_write@entry=true) at /usr/src/debug/qemu-2.1.2/exec.c:2088
#12 0x0000555555653c20 in kvm_handle_io (count=1, size=4, direction=<optimized out>, data=<optimized out>, port=49220)
    at /usr/src/debug/qemu-2.1.2/kvm-all.c:1600
#13 kvm_cpu_exec (cpu=cpu@entry=0x55555637c9c0) at /usr/src/debug/qemu-2.1.2/kvm-all.c:1737
#14 0x0000555555642dc2 in qemu_kvm_cpu_thread_fn (arg=0x55555637c9c0) at /usr/src/debug/qemu-2.1.2/cpus.c:874
#15 0x00007ffff6ba2df3 in start_thread (arg=0x7fffe8fbd700) at pthread_create.c:308
#16 0x00007ffff1f623dd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113
(gdb) bt ful
#0  0x00007ffff1ea1989 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
        resultvar = 0
        pid = 14549
        selftid = 14570
#1  0x00007ffff1ea3098 in __GI_abort () at abort.c:90
        save_stage = 2
        act = {__sigaction_handler = {sa_handler = 0x7fffffffe4c2, sa_sigaction = 0x7fffffffe4c2}, sa_mask = {__val = {
              140737253373233, 93824995679416, 437, 140737102203072, 140737252016723, 4, 140737102203040, 98398095760, 
              18446744073709550960, 8, 0, 0, 0, 21474836480, 140737354051584, 140737253385192}}, sa_flags = 1435099069, 
          sa_restorer = 0x55555589e5d0 <__PRETTY_FUNCTION__.28404>}
        sigs = {__val = {32, 0 <repeats 15 times>}}
#2  0x00007ffff1e9a8f6 in __assert_fail_base (fmt=0x7ffff1fea3e8 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", 
    assertion=assertion@entry=0x55555589dfbd "!r", 
    file=file@entry=0x55555589dcb8 "/builddir/build/BUILD/qemu-2.1.2/hw/net/virtio-net.c", line=line@entry=437, 
    function=function@entry=0x55555589e5d0 <__PRETTY_FUNCTION__.28404> "virtio_net_set_queues") at assert.c:92
        str = 0x7fffe0000b50 ""
        total = 4096
#3  0x00007ffff1e9a9a2 in __GI___assert_fail (assertion=assertion@entry=0x55555589dfbd "!r", 
    file=file@entry=0x55555589dcb8 "/builddir/build/BUILD/qemu-2.1.2/hw/net/virtio-net.c", line=line@entry=437, 
    function=function@entry=0x55555589e5d0 <__PRETTY_FUNCTION__.28404> "virtio_net_set_queues") at assert.c:101
No locals.
#4  0x000055555567618a in virtio_net_set_queues (n=0x5555563fac38) at /usr/src/debug/qemu-2.1.2/hw/net/virtio-net.c:437
        i = 1
#5  0x00005555556768ab in virtio_net_set_multiqueue (multiqueue=1, n=0x5555563fac38)
    at /usr/src/debug/qemu-2.1.2/hw/net/virtio-net.c:1336
        vdev = 0x5555563fac38
        i = <optimized out>
        max = <optimized out>
#6  virtio_net_set_features (vdev=<optimized out>, features=955252707) at /usr/src/debug/qemu-2.1.2/hw/net/virtio-net.c:522
        n = 0x5555563fac38
        __func__ = "virtio_net_set_features"
        i = <optimized out>
#7  0x000055555567e04b in virtio_set_features (vdev=0x5555563fac38, val=955252707)
    at /usr/src/debug/qemu-2.1.2/hw/virtio/virtio.c:971
---Type <return> to continue, or q <return> to quit---
        qbus = <optimized out>
        __func__ = "virtio_set_features"
        vbusk = 0x55555614edf0
        k = 0x5555561563d0
        supported_features = <optimized out>
        bad = false
#8  0x000055555565479a in access_with_adjusted_size (addr=addr@entry=4, value=value@entry=0x7fffe8fbcaf0, size=size@entry=4, 
    access_size_min=<optimized out>, access_size_max=<optimized out>, access=0x555555654910 <memory_region_write_accessor>, 
    mr=0x5555563faaa8) at /usr/src/debug/qemu-2.1.2/memory.c:481
        access_mask = 4294967295
        access_size = 4
        i = <optimized out>
#9  0x0000555555659367 in memory_region_dispatch_write (size=4, data=955252707, addr=4, mr=0x5555563faaa8)
    at /usr/src/debug/qemu-2.1.2/memory.c:1143
No locals.
#10 io_mem_write (mr=mr@entry=0x5555563faaa8, addr=4, val=<optimized out>, size=4) at /usr/src/debug/qemu-2.1.2/memory.c:1976
No locals.
#11 0x0000555555624723 in address_space_rw (as=0x555555cb0480 <address_space_io>, addr=addr@entry=49220, 
    buf=0x7ffff7ec1000 "\343\377\357\070", len=len@entry=4, is_write=is_write@entry=true) at /usr/src/debug/qemu-2.1.2/exec.c:2088
        l = 4
        ptr = <optimized out>
        val = 955252707
        addr1 = 4
        mr = 0x5555563faaa8
        error = false
#12 0x0000555555653c20 in kvm_handle_io (count=1, size=4, direction=<optimized out>, data=<optimized out>, port=49220)
    at /usr/src/debug/qemu-2.1.2/kvm-all.c:1600
        i = 0
        ptr = <optimized out>
#13 kvm_cpu_exec (cpu=cpu@entry=0x55555637c9c0) at /usr/src/debug/qemu-2.1.2/kvm-all.c:1737
        run = 0x7ffff7ec0000
        ret = <optimized out>
        run_ret = <optimized out>
#14 0x0000555555642dc2 in qemu_kvm_cpu_thread_fn (arg=0x55555637c9c0) at /usr/src/debug/qemu-2.1.2/cpus.c:874
---Type <return> to continue, or q <return> to quit---
        cpu = 0x55555637c9c0
        r = <optimized out>
#15 0x00007ffff6ba2df3 in start_thread (arg=0x7fffe8fbd700) at pthread_create.c:308
        __res = <optimized out>
        pd = 0x7fffe8fbd700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140737102206720, -4482216436985367725, 1, 140737102207424, 140737102206720, 
                140737488345680, 4482249389558068051, 4482236741904495443}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 
              0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
        pagesize_m1 = <optimized out>
        sp = <optimized out>
        freesize = <optimized out>
#16 0x00007ffff1f623dd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113
No locals.


Expected results:

works well
Additional info:

If did not enable mq inside guest, won't hit this issue.
Comment 2 Qian Guo 2014-10-10 05:28:02 UTC 
When qemu-kvm coredumpd, it prints 
"
qemu-kvm: could not disable queue
qemu-kvm: /builddir/build/BUILD/qemu-2.1.2/hw/net/virtio-net.c:437: virtio_net_set_queues: Assertion `!r' failed.
"
first

So this should be a copy from bug 1004275 and just filed to the qemu-kvm-rhev component.
Comment 3 jason wang 2014-10-11 05:38:50 UTC 
Less priority, it's hard to recover from host mis-configuration.
Comment 4 Ronen Hod 2014-10-21 11:26:03 UTC 
Deferring.
QEMU cannot recover from a non-functioning (deleted) interface. Need to think what can be done.
Comment 5 jason wang 2014-11-07 08:27:44 UTC 
More investigation: this could be fixed by both side:
- Qemu, warn instead of abort(), this will help the case that SELinux prevent qemu from adding/removing queues.
- Kernel, do not allow the device to be deleted before file descriptors/sockets were gone. This may prevent tap to be deleted when qemu is running. This seems a regression when multiqueue was introduced.
Comment 6 jason wang 2014-11-27 06:55:16 UTC 
Confirm with Michael:

This could be addressed after virtio 1.0 is merged. It has a specific device status field (DEVICE_NEEDS_RESET). So we qemu detects misconfig like this bug, it can set DEVICE_NEEDS_RESET filed and issue an config change interrupt for driver to disable the device.
Comment 8 jason wang 2014-12-15 05:32:28 UTC 
Lower severity as host mis-configuration.
Comment 10 jason wang 2015-08-10 05:27:58 UTC 
Low priority will try to fix it in 7.3.
Comment 11 jason wang 2015-09-24 09:17:59 UTC 
*** Bug 1265903 has been marked as a duplicate of this bug. ***
Comment 16 jason wang 2016-12-26 05:47:33 UTC 
*** Bug 1242383 has been marked as a duplicate of this bug. ***
Note You need to log in before you can comment on or make changes to this bug.