Bug 1746224

Summary: qemu coredump: qemu-kvm: block/create.c:68: qmp_blockdev_create: Assertion `drv' failed
Product: Red Hat Enterprise Linux 7 Reporter: Xu Tian <xutian>
Component: qemu-kvm-rhevAssignee: Philippe Mathieu-Daudé <philmd>
Status: CLOSED ERRATA QA Contact: aihua liang <aliang>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.8CC: coli, jinzhao, juzhang, philmd, virt-maint
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: qemu-kvm-rhev-2.12.0-36.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1746267 (view as bug list) Environment:
Last Closed: 2020-03-31 14:34:56 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1746267    

Description Xu Tian 2019-08-28 02:31:03 UTC
Description of problem:

qemu core dump when call qmp command "blockdev-create" to create block device with nfs driver.

see backtrack below:


(gdb) bt
#0  0x00007fe11ab6f377 in raise () at /lib64/libc.so.6
#1  0x00007fe11ab70a68 in abort () at /lib64/libc.so.6
#2  0x00007fe11ab68196 in __assert_fail_base () at /lib64/libc.so.6
#3  0x00007fe11ab68242 in  () at /lib64/libc.so.6
#4  0x000055d3c7b8a580 in qmp_blockdev_create (job_id=0x55d3ca65eef8 "mkfile", options=0x55d3ca103dc0, errp=errp@entry=0x7ffce8fe6ef8) at block/create.c:68
#5  0x000055d3c79e6ab0 in qmp_marshal_blockdev_create (args=<optimized out>, ret=<optimized out>, errp=0x7ffce8fe6f68) at qapi/qapi-commands-block-core.c:1226
#6  0x000055d3c7c0226a in qmp_dispatch (errp=0x7ffce8fe6f60, request=0x7ffce8fe6f60, cmds=<optimized out>) at qapi/qmp-dispatch.c:111
#7  0x000055d3c7c0226a in qmp_dispatch (cmds=<optimized out>, request=request@entry=0x55d3cbf7a800) at qapi/qmp-dispatch.c:160
#8  0x000055d3c78fa181 in monitor_qmp_dispatch_one (req_obj=<optimized out>) at /usr/src/debug/qemu-2.12.0/monitor.c:4102
#9  0x000055d3c78fa3e5 in monitor_qmp_bh_dispatcher (data=<optimized out>) at /usr/src/debug/qemu-2.12.0/monitor.c:4160
#10 0x000055d3c7c0c1b1 in aio_bh_poll (bh=0x55d3c9fb3b30) at util/async.c:90
#11 0x000055d3c7c0c1b1 in aio_bh_poll (ctx=ctx@entry=0x55d3c9f6cdc0) at util/async.c:118
#12 0x000055d3c7c0f260 in aio_dispatch (ctx=0x55d3c9f6cdc0) at util/aio-posix.c:440
#13 0x000055d3c7c0c08e in aio_ctx_dispatch (source=<optimized out>, callback=<optimized out>, user_data=<optimized out>) at util/async.c:261
#14 0x00007fe1213aa099 in g_main_context_dispatch () at /lib64/libglib-2.0.so.0
#15 0x000055d3c7c0e557 in main_loop_wait () at util/main-loop.c:215
#16 0x000055d3c7c0e557 in main_loop_wait (timeout=<optimized out>) at util/main-loop.c:238
#17 0x000055d3c7c0e557 in main_loop_wait (nonblocking=nonblocking@entry=0) at util/main-loop.c:497
#18 0x000055d3c78ae227 in main () at vl.c:2013
#19 0x000055d3c78ae227 in main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>) at vl.c:4776
(gdb) 
 


Version-Release number of selected component (if applicable):

qemu-kvm-rhev-2.12.0-33.el7.x86_64


How reproducible:

100%

Steps to Reproduce:
1. launch a VM

MALLOC_PERTURB_=1 /usr/libexec/qemu-kvm -S -name 'avocado-vt-vm1' -machine pc -nodefaults -device VGA,bus=pci.0,addr=0x2 -chardev socket,id=qmp_id_qmpmonitor1,path=/var/tmp/avocado_Gz1E9w/monitor-qmpmonitor1-20190828-101405-0W6NTpow,server,nowait -mon chardev=qmp_id_qmpmonitor1,mode=control -chardev socket,id=qmp_id_catch_monitor,path=/var/tmp/avocado_Gz1E9w/monitor-catch_monitor-20190828-101405-0W6NTpow,server,nowait -mon chardev=qmp_id_catch_monitor,mode=control -device pvpanic,ioport=0x505,id=idYFQaPO -chardev socket,path=/var/tmp/avocado_Gz1E9w/serial-serial0-20190828-101405-0W6NTpow,nowait,id=chardev_serial0,server -device isa-serial,id=serial0,chardev=chardev_serial0 -chardev socket,id=seabioslog_id_20190828-101405-0W6NTpow,path=/var/tmp/avocado_Gz1E9w/seabios-20190828-101405-0W6NTpow,server,nowait -device isa-debugcon,chardev=seabioslog_id_20190828-101405-0W6NTpow,iobase=0x402 -device nec-usb-xhci,id=usb1,bus=pci.0,addr=0x3 -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pci.0,addr=0x4 -blockdev node-name=file_image1,driver=file,filename=/home/kvm_autotest_root/images/rhel810-64-virtio-scsi.qcow2,cache.direct=on,cache.no-flush=off -blockdev node-name=drive_image1,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_image1 -device scsi-hd,id=image1,drive=drive_image1,write-cache=on -blockdev node-name=file_image2,driver=file,filename=/home/kvm_autotest_root/images/data.qcow2,cache.direct=on,cache.no-flush=off -blockdev node-name=drive_image2,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_image2 -device scsi-hd,id=image2,drive=drive_image2,write-cache=on -device virtio-net-pci,mac=9a:0a:41:06:7a:1a,id=idVJMvWD,netdev=idDCWjBK,bus=pci.0,addr=0x5 -netdev tap,id=idDCWjBK,vhost=on,vhostfd=20,fd=17 -m 3072 -smp 4,maxcpus=4,cores=2,threads=1,sockets=2 -cpu 'Skylake-Client',+kvm_pv_unhalt -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 -vnc :0 -rtc base=utc,clock=host,driftfix=slew -boot menu=off,strict=off,order=cdn,once=c -enable-kvm

2. send QMP cmd

{'execute': 'blockdev-create', 'arguments': {'job-id': 'mkfile', 'options': {'size': 2147483648, 'driver': 'nfs', 'location': {'path': '/home/kvm_autotest_root/images', 'group': 0, 'user': 0, 'server': {'host': '127.0.0.1', 'type': 'inet'}}}}, 'id': 'hiVGLisR'}


Actual results:

Qemu coredump,  stdout is "qemu-kvm: block/create.c:68: qmp_blockdev_create: Assertion `drv' failed."

Expected results:

Qemu not coredump and block device created

Additional info:

Comment 2 Xu Tian 2019-08-28 02:35:29 UTC
It's not nfs server side issue, nfs server works in local host.

root$ showmount  -e 127.0.0.1
Export list for 127.0.0.1:
/home/kvm_autotest_root/images *

thanks,
Xu

Comment 3 Xu Tian 2019-08-28 02:42:22 UTC
Full track:

(gdb) bt full
#0  0x00007fe11ab6f377 in raise () at /lib64/libc.so.6
#1  0x00007fe11ab70a68 in abort () at /lib64/libc.so.6
#2  0x00007fe11ab68196 in __assert_fail_base () at /lib64/libc.so.6
#3  0x00007fe11ab68242 in  () at /lib64/libc.so.6
#4  0x000055d3c7b8a580 in qmp_blockdev_create (job_id=0x55d3ca65eef8 "mkfile", options=0x55d3ca103dc0, errp=errp@entry=0x7ffce8fe6ef8) at block/create.c:68
        s = <optimized out>
        fmt = <optimized out>
        drv = 0x0
        __PRETTY_FUNCTION__ = "qmp_blockdev_create"
        __func__ = "qmp_blockdev_create"
#5  0x000055d3c79e6ab0 in qmp_marshal_blockdev_create (args=<optimized out>, ret=<optimized out>, errp=0x7ffce8fe6f68) at qapi/qapi-commands-block-core.c:1226
        err = 0x0
        v = 0x55d3c9f5b860
        arg = {job_id = 0x55d3ca65eef8 "mkfile", options = 0x55d3ca103dc0}
#6  0x000055d3c7c0226a in qmp_dispatch (errp=0x7ffce8fe6f60, request=0x7ffce8fe6f60, cmds=<optimized out>) at qapi/qmp-dispatch.c:111
        local_err = 0x0
        command = 0x55d3cb573ee0 "blockdev-create"
        dict = <optimized out>
        ret = 0x0
        args = 0x55d3cbfd6000
        cmd = 0x55d3c9fb2ed0
        err = 0x0
        rsp = <optimized out>
#7  0x000055d3c7c0226a in qmp_dispatch (cmds=<optimized out>, request=request@entry=0x55d3cbf7a800) at qapi/qmp-dispatch.c:160
        err = 0x0
        rsp = <optimized out>
#8  0x000055d3c78fa181 in monitor_qmp_dispatch_one (req_obj=<optimized out>) at /usr/src/debug/qemu-2.12.0/monitor.c:4102
        mon = 0x55d3ca4d0160
        old_mon = 0x0
        req = 0x55d3cbf7a800
        rsp = 0x0
        id = 0x55d3ca4195c0
        need_resume = true
#9  0x000055d3c78fa3e5 in monitor_qmp_bh_dispatcher (data=<optimized out>) at /usr/src/debug/qemu-2.12.0/monitor.c:4160
        req_obj = 0x55d3ca9af080
#10 0x000055d3c7c0c1b1 in aio_bh_poll (bh=0x55d3c9fb3b30) at util/async.c:90
        bh = <optimized out>
        bhp = <optimized out>
        next = 0x55d3ca0b8300
        ret = 1
        deleted = false
#11 0x000055d3c7c0c1b1 in aio_bh_poll (ctx=ctx@entry=0x55d3c9f6cdc0) at util/async.c:118
        bh = <optimized out>
        bhp = <optimized out>
        next = 0x55d3ca0b8300
        ret = 1
        deleted = false
#12 0x000055d3c7c0f260 in aio_dispatch (ctx=0x55d3c9f6cdc0) at util/aio-posix.c:440
#13 0x000055d3c7c0c08e in aio_ctx_dispatch (source=<optimized out>, callback=<optimized out>, user_data=<optimized out>) at util/async.c:261
        ctx = <optimized out>
#14 0x00007fe1213aa099 in g_main_context_dispatch () at /lib64/libglib-2.0.so.0
#15 0x000055d3c7c0e557 in main_loop_wait () at util/main-loop.c:215
        context = 0x55d3c9f67ef0
        pfds = <optimized out>
        context = 0x55d3c9f67ef0
        ret = 1
        ret = 1
        timeout = 4294967295
        timeout_ns = 2339415669
#16 0x000055d3c7c0e557 in main_loop_wait (timeout=<optimized out>) at util/main-loop.c:238
        context = 0x55d3c9f67ef0
        ret = 1
        ret = 1
        timeout = 4294967295
        timeout_ns = 2339415669
#17 0x000055d3c7c0e557 in main_loop_wait (nonblocking=nonblocking@entry=0) at util/main-loop.c:497
        ret = 1
        timeout = 4294967295
        timeout_ns = 2339415669
#18 0x000055d3c78ae227 in main () at vl.c:2013
        i = <optimized out>
        snapshot = <optimized out>
        linux_boot = <optimized out>
        initrd_filename = <optimized out>
        kernel_filename = <optimized out>
        kernel_cmdline = <optimized out>
        boot_order = <optimized out>
        boot_once = 0x55d3c9fda5a0 "c"
        ds = <optimized out>
        opts = <optimized out>
        machine_opts = <optimized out>
        icount_opts = <optimized out>
        accel_opts = <optimized out>
        olist = <optimized out>
        optind = 62
        optarg = 0x0
        loadvm = <optimized out>
        machine_class = 0x0
        cpu_model = <optimized out>
        vga_model = 0x0
        qtest_chrdev = <optimized out>
        qtest_log = <optimized out>
        pid_file = <optimized out>
        incoming = <optimized out>
        userconfig = <optimized out>
        nographic = <optimized out>
        display_remote = <optimized out>
        log_mask = <optimized out>
        log_file = <optimized out>
        trace_file = <optimized out>
        maxram_size = <optimized out>
        ram_slots = <optimized out>
        vmstate_dump_file = <optimized out>
        main_loop_err = 0x0
        err = 0x0
        list_data_dirs = <optimized out>
        dir = <optimized out>
        dirs = 0x0
        bdo_queue = {sqh_first = 0x0, sqh_last = 0x7ffce8fe7240}
        __func__ = "main"
        __FUNCTION__ = "main"
#19 0x000055d3c78ae227 in main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>) at vl.c:4776
        i = <optimized out>
        snapshot = <optimized out>
        linux_boot = <optimized out>
        initrd_filename = <optimized out>
        kernel_filename = <optimized out>
        kernel_cmdline = <optimized out>
        boot_order = <optimized out>
        boot_once = 0x55d3c9fda5a0 "c"
        ds = <optimized out>
        opts = <optimized out>
        machine_opts = <optimized out>
        icount_opts = <optimized out>
        accel_opts = <optimized out>
        olist = <optimized out>
        optind = 62
        optarg = 0x0
        loadvm = <optimized out>
        machine_class = 0x0
        cpu_model = <optimized out>
        vga_model = 0x0
        qtest_chrdev = <optimized out>
        qtest_log = <optimized out>
        pid_file = <optimized out>
        incoming = <optimized out>
        userconfig = <optimized out>
        nographic = <optimized out>
        display_remote = <optimized out>
        log_mask = <optimized out>
        log_file = <optimized out>
        trace_file = <optimized out>
        maxram_size = <optimized out>
        ram_slots = <optimized out>
        vmstate_dump_file = <optimized out>
        main_loop_err = 0x0
        err = 0x0
        list_data_dirs = <optimized out>
        dir = <optimized out>
        dirs = 0x0
        bdo_queue = {sqh_first = 0x0, sqh_last = 0x7ffce8fe7240}
        __func__ = "main"
        __FUNCTION__ = "main"

Comment 4 John Ferlan 2019-08-28 21:28:29 UTC
Was this done usin

Comment 7 aihua liang 2019-09-25 06:10:31 UTC
Test on qemu-kvm-rhev-2.12.0-36.el7, don't hit this issue any more, will set bug's status to "Verified" after its "ON_QA".

 Test steps:
   1.Start guest with qemu cmds:
      /usr/libexec/qemu-kvm \
    -name 'avocado-vt-vm1' \
    -machine pc  \
    -nodefaults \
    -device VGA,bus=pci.0,addr=0x2  \
    -chardev socket,id=qmp_id_qmpmonitor1,path=/var/tmp/monitor-qmpmonitor1-20190624-022456-Uc0cnuxs,server,nowait \
    -mon chardev=qmp_id_qmpmonitor1,mode=control  \
    -chardev socket,id=qmp_id_catch_monitor,path=/var/tmp/monitor-catch_monitor-20190624-022456-Uc0cnuxs,server,nowait \
    -mon chardev=qmp_id_catch_monitor,mode=control \
    -device pvpanic,ioport=0x505,id=idOSU1la  \
    -chardev socket,id=serial_id_serial0,path=/var/tmp/serial-serial0-20190624-022456-Uc0cnuxs,server,nowait \
    -device isa-serial,chardev=serial_id_serial0  \
    -chardev socket,id=seabioslog_id_20190624-022456-Uc0cnuxs,path=/var/tmp/seabios-20190624-022456-Uc0cnuxs,server,nowait \
    -device isa-debugcon,chardev=seabioslog_id_20190624-022456-Uc0cnuxs,iobase=0x402 \
    -device nec-usb-xhci,id=usb1,bus=pci.0,addr=0x3 \
    -object iothread,id=iothread0 \
    -drive id=drive_image1,if=none,snapshot=off,aio=threads,cache=none,file=/home/kvm_autotest_root/images/rhel77-64-virtio-scsi.qcow2 \
    -device virtio-blk-pci,id=image1,drive=drive_image1,bootindex=1,bus=pci.0,iothread=iothread0 \
    -device virtio-net-pci,mac=9a:01:02:03:04:05,id=idk78PtS,vectors=4,netdev=idd7g5ik,bus=pci.0,addr=0x5  \
    -netdev tap,id=idd7g5ik,vhost=on \
    -m 7168  \
    -smp 4,maxcpus=4,cores=2,threads=1,sockets=2 \
    -cpu 'Penryn',+kvm_pv_unhalt \
    -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1  \
    -vnc :0  \
    -rtc base=utc,clock=host,driftfix=slew  \
    -boot menu=off,strict=off,order=cdn,once=c \
    -enable-kvm \
    -monitor stdio \

  2. Create a node with driver "nfs"
   {'execute':'blockdev-create','arguments':{'options': {'driver':'nfs','size':32212254720,'location':{'path':'/mnt/nfs','server':{'host':'10.66.8.210','type':'inet'}}},'job-id':'j1'}}
   {"error": {"class": "GenericError", "desc": "Block driver 'nfs' not found or not supported"}}

Comment 9 aihua liang 2019-09-30 02:06:05 UTC
As comment7, set bug's status to "Verified".

Comment 11 errata-xmlrpc 2020-03-31 14:34:56 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:1216