Bug 1786027 - qemu crashed when attaching vf to guest with non-existing rom file
Summary: qemu crashed when attaching vf to guest with non-existing rom file
Keywords:
Status: CLOSED DUPLICATE of bug 1782678
Alias: None
Product: Red Hat Enterprise Linux Advanced Virtualization
Classification: Red Hat
Component: qemu-kvm
Version: 8.1
Hardware: Unspecified
OS: Unspecified
medium
unspecified
Target Milestone: rc
: ---
Assignee: Amnon Ilan
QA Contact: Yanghang Liu
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-12-23 06:47 UTC by yafu
Modified: 2020-07-28 00:54 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-06-29 18:57:04 UTC
Type: Bug
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description yafu 2019-12-23 06:47:20 UTC
Description of problem:
qemu crashed when attaching vf to guest with non-existing rom file


Version-Release number of selected component (if applicable):
qemu-kvm-4.1.0-20.module+el8.1.1+5309+6d656f05.x86_64
libvirt-5.6.0-10.el8.x86_64


How reproducible:
100%


Steps to Reproduce:
1.Start a guest as follows:
#virsh dumpxml r8
...
<os>
    <type arch='x86_64' machine='pc-i440fx-rhel7.6.0'>hvm</type>
  </os>
...
<device>
<disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/nfs/r8.qcow2'/>
      <target dev='vda' bus='virtio'/>
      <boot order='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </disk>
...

2.Prepare a vf interface xml with non-existing rom file:
<interface type='hostdev' managed='yes'>
     <mac address='42:4c:43:50:3c:10'/>
     <source>
       <address type='pci' domain='0x0000' bus='0x04' slot='0x10' function='0x0'/>
     </source>
     <boot order='2'/> 
   <rom bar='on' file='/usr/share/ipxe/808610ca.rom'/> 
   </interface>


3.Attach the vf to guest:
# virsh attach-device r8 vf.xml 
error: Failed to attach device from vf.xml
error: internal error: child reported (status=125): Kernel does not provide mount namespace: No such file or directory


Actual results:
qemu crashed when attaching vf to guest with non-existing rom file

Expected results:
qemu should not crash and report error info.

Additional info:
Backtrace of guest:
Thread 1 "qemu-kvm" received signal SIGSEGV, Segmentation fault.
0x000055bad6ff814d in notifier_remove (notifier=notifier@entry=0x55bada342a28) at util/notify.c:31
31	    QLIST_REMOVE(notifier, node);
(gdb) t a a bt

Thread 6 (Thread 0x7ff3761ff700 (LWP 25385)):
#0  0x00007ff3857ac211 in __GI___poll (fds=0x7ff3140031e0, nfds=2, timeout=2147483647) at ../sysdeps/unix/sysv/linux/poll.c:29
#1  0x00007ff38a56e9b6 in g_main_context_poll (priority=<optimized out>, n_fds=2, fds=0x7ff3140031e0, timeout=<optimized out>, context=0x55badacda720) at gmain.c:4203
#2  g_main_context_iterate (context=0x55badacda720, block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>) at gmain.c:3897
#3  0x00007ff38a56ed72 in g_main_loop_run (loop=0x7ff314002530) at gmain.c:4098
#4  0x00007ff38795c47b in red_worker_main () from target:/lib64/libspice-server.so.1
#5  0x00007ff385a862de in start_thread (arg=<optimized out>) at pthread_create.c:486
#6  0x00007ff3857b7133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 5 (Thread 0x7ff376ffd700 (LWP 25383)):
#0  0x00007ff3857adb2b in ioctl () at ../sysdeps/unix/syscall-template.S:78
#1  0x000055bad6ce4749 in kvm_vcpu_ioctl (cpu=<optimized out>, type=<optimized out>) at /usr/src/debug/qemu-kvm-4.1.0-20.module+el8.1.1+5309+6d656f05.x86_64/accel/kvm/kvm-all.c:2429
#2  0x000055bad6ce4809 in kvm_cpu_exec (cpu=<optimized out>) at /usr/src/debug/qemu-kvm-4.1.0-20.module+el8.1.1+5309+6d656f05.x86_64/accel/kvm/kvm-all.c:2266
#3  0x000055bad6cc99fe in qemu_kvm_cpu_thread_fn (arg=0x55bad98fdd50) at /usr/src/debug/qemu-kvm-4.1.0-20.module+el8.1.1+5309+6d656f05.x86_64/cpus.c:1285
#4  0x000055bad6feb954 in qemu_thread_start (args=0x55bad99217a0) at util/qemu-thread-posix.c:502
#5  0x00007ff385a862de in start_thread (arg=<optimized out>) at pthread_create.c:486
#6  0x00007ff3857b7133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 4 (Thread 0x7ff3777fe700 (LWP 25382)):
#0  __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:103
#1  0x00007ff385a88af9 in __GI___pthread_mutex_lock (mutex=mutex@entry=0x55bad784c780 <qemu_global_mutex>) at ../nptl/pthread_mutex_lock.c:80
#2  0x000055bad6feba3d in qemu_mutex_lock_impl (mutex=0x55bad784c780 <qemu_global_mutex>, file=0x55bad7085588 "/builddir/build/BUILD/qemu-4.1.0/exec.c", line=3333) at util/qemu-thread-posix.c:66
#3  0x000055bad6cc982e in qemu_mutex_lock_iothread_impl (file=<optimized out>, line=<optimized out>) at /usr/src/debug/qemu-kvm-4.1.0-20.module+el8.1.1+5309+6d656f05.x86_64/cpus.c:1859
#4  0x000055bad6c81cf9 in prepare_mmio_access (mr=<optimized out>, mr=<optimized out>) at /usr/src/debug/qemu-kvm-4.1.0-20.module+el8.1.1+5309+6d656f05.x86_64/exec.c:3333
#5  0x000055bad6c82d10 in flatview_write_continue (fv=0x55bad9c6c420, addr=49280, attrs=..., buf=0x7ff38b12f000 "", len=1, addr1=<optimized out>, l=<optimized out>, mr=0x55badaba2f90)
    at /usr/src/debug/qemu-kvm-4.1.0-20.module+el8.1.1+5309+6d656f05.x86_64/exec.c:3364
#6  0x000055bad6c82ec6 in flatview_write (fv=0x55bad9c6c420, addr=49280, attrs=..., buf=0x7ff38b12f000 "", len=1) at /usr/src/debug/qemu-kvm-4.1.0-20.module+el8.1.1+5309+6d656f05.x86_64/exec.c:3408
#7  0x000055bad6c8725a in address_space_write (as=<optimized out>, addr=<optimized out>, attrs=..., buf=<optimized out>, len=<optimized out>)
    at /usr/src/debug/qemu-kvm-4.1.0-20.module+el8.1.1+5309+6d656f05.x86_64/exec.c:3498
#8  0x000055bad6ce4a34 in kvm_cpu_exec (cpu=<optimized out>) at /usr/src/debug/qemu-kvm-4.1.0-20.module+el8.1.1+5309+6d656f05.x86_64/accel/kvm/kvm-all.c:2060
#9  0x000055bad6cc99fe in qemu_kvm_cpu_thread_fn (arg=0x55bad98b1e60) at /usr/src/debug/qemu-kvm-4.1.0-20.module+el8.1.1+5309+6d656f05.x86_64/cpus.c:1285
#10 0x000055bad6feb954 in qemu_thread_start (args=0x55bad98d3c50) at util/qemu-thread-posix.c:502
#11 0x00007ff385a862de in start_thread (arg=<optimized out>) at pthread_create.c:486
#12 0x00007ff3857b7133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 3 (Thread 0x7ff377fff700 (LWP 25381)):
#0  0x00007ff3857ac211 in __GI___poll (fds=0x7ff36c0023e0, nfds=3, timeout=-1) at ../sysdeps/unix/sysv/linux/poll.c:29
#1  0x00007ff38a56e9b6 in g_main_context_poll (priority=<optimized out>, n_fds=3, fds=0x7ff36c0023e0, timeout=<optimized out>, context=0x55bad989ab50) at gmain.c:4203
#2  g_main_context_iterate (context=0x55bad989ab50, block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>) at gmain.c:3897
#3  0x00007ff38a56ed72 in g_main_loop_run (loop=0x55bad989ac90) at gmain.c:4098
#4  0x000055bad6dcb6b1 in iothread_run (opaque=0x55bad9813c00) at iothread.c:82
#5  0x000055bad6feb954 in qemu_thread_start (args=0x55bad989acb0) at util/qemu-thread-posix.c:502
#6  0x00007ff385a862de in start_thread (arg=<optimized out>) at pthread_create.c:486
#7  0x00007ff3857b7133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 2 (Thread 0x7ff37ed37700 (LWP 25375)):
#0  __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:103
#1  0x00007ff385a88af9 in __GI___pthread_mutex_lock (mutex=mutex@entry=0x55bad784c780 <qemu_global_mutex>) at ../nptl/pthread_mutex_lock.c:80
--Type <RET> for more, q to quit, c to continue without paging--
#2  0x000055bad6feba3d in qemu_mutex_lock_impl (mutex=0x55bad784c780 <qemu_global_mutex>, file=0x55bad71922b0 "util/rcu.c", line=268) at util/qemu-thread-posix.c:66
#3  0x000055bad6cc982e in qemu_mutex_lock_iothread_impl (file=file@entry=0x55bad71922b0 "util/rcu.c", line=line@entry=268) at /usr/src/debug/qemu-kvm-4.1.0-20.module+el8.1.1+5309+6d656f05.x86_64/cpus.c:1859
#4  0x000055bad6ffdd96 in call_rcu_thread (opaque=<optimized out>) at util/rcu.c:268
#5  0x000055bad6feb954 in qemu_thread_start (args=0x55bad97575a0) at util/qemu-thread-posix.c:502
#6  0x00007ff385a862de in start_thread (arg=<optimized out>) at pthread_create.c:486
#7  0x00007ff3857b7133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 1 (Thread 0x7ff38b0f9ec0 (LWP 25340)):
#0  0x000055bad6ff814d in notifier_remove (notifier=notifier@entry=0x55bada342a28) at util/notify.c:31
#1  0x000055bad6ce1679 in kvm_irqchip_remove_change_notifier (n=n@entry=0x55bada342a28) at /usr/src/debug/qemu-kvm-4.1.0-20.module+el8.1.1+5309+6d656f05.x86_64/accel/kvm/kvm-all.c:1353
#2  0x000055bad6d29fa8 in vfio_exitfn (pdev=<optimized out>) at /usr/src/debug/qemu-kvm-4.1.0-20.module+el8.1.1+5309+6d656f05.x86_64/hw/vfio/pci.c:3097
#3  0x000055bad6e68b4b in pci_qdev_unrealize (dev=<optimized out>, errp=errp@entry=0x0) at hw/pci/pci.c:1127
#4  0x000055bad6e6ac4b in pci_qdev_realize (qdev=0x55bada341de0, errp=<optimized out>) at hw/pci/pci.c:2142
#5  0x000055bad6e062a4 in device_set_realized (obj=<optimized out>, value=<optimized out>, errp=0x7ffca0ef0318) at hw/core/qdev.c:858
#6  0x000055bad6f17fcb in property_set_bool (obj=0x55bada341de0, v=<optimized out>, name=<optimized out>, opaque=0x55bad98535f0, errp=0x7ffca0ef0318) at qom/object.c:2079
#7  0x000055bad6f1c493 in object_property_set_qobject (obj=0x55bada341de0, value=<optimized out>, name=0x55bad70ccb3d "realized", errp=0x7ffca0ef0318) at qom/qom-qobject.c:26
#8  0x000055bad6f19cf9 in object_property_set_bool (obj=0x55bada341de0, value=<optimized out>, name=0x55bad70ccb3d "realized", errp=0x7ffca0ef0318) at qom/object.c:1337
#9  0x000055bad6dcd51d in qdev_device_add (opts=opts@entry=0x55bad9bfda90, errp=errp@entry=0x7ffca0ef03f0) at qdev-monitor.c:665
#10 0x000055bad6dcd9cb in qmp_device_add (qdict=<optimized out>, ret_data=<optimized out>, errp=0x7ffca0ef0438) at qdev-monitor.c:790
#11 0x000055bad6f9f24c in do_qmp_dispatch (errp=0x7ffca0ef0430, allow_oob=<optimized out>, request=<optimized out>, cmds=0x55bad787efe0 <qmp_commands>) at qapi/qmp-dispatch.c:131
#12 qmp_dispatch (cmds=0x55bad787efe0 <qmp_commands>, request=<optimized out>, allow_oob=<optimized out>) at qapi/qmp-dispatch.c:174
#13 0x000055bad6ec1751 in monitor_qmp_dispatch (mon=0x55bad989a620, req=<optimized out>) at monitor/qmp.c:120
#14 0x000055bad6ec1d9a in monitor_qmp_bh_dispatcher (data=<optimized out>) at monitor/qmp.c:209
#15 0x000055bad6fe6326 in aio_bh_call (bh=0x55bad97c7b20) at util/async.c:117
#16 aio_bh_poll (ctx=ctx@entry=0x55bad97c66d0) at util/async.c:117
#17 0x000055bad6fe9764 in aio_dispatch (ctx=0x55bad97c66d0) at util/aio-posix.c:459
#18 0x000055bad6fe6202 in aio_ctx_dispatch (source=<optimized out>, callback=<optimized out>, user_data=<optimized out>) at util/async.c:260
#19 0x00007ff38a56e67d in g_main_dispatch (context=0x55bad9850a80) at gmain.c:3176
#20 g_main_context_dispatch (context=context@entry=0x55bad9850a80) at gmain.c:3829
#21 0x000055bad6fe8818 in glib_pollfds_poll () at util/main-loop.c:218
#22 os_host_main_loop_wait (timeout=<optimized out>) at util/main-loop.c:241
#23 main_loop_wait (nonblocking=<optimized out>) at util/main-loop.c:517
#24 0x000055bad6dd0dd9 in main_loop () at vl.c:1809
#25 0x000055bad6c7e3d3 in main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>) at vl.c:4508

Comment 1 Yanghang Liu 2019-12-23 11:21:26 UTC
This bug can be reproduced in RHEL8.2-AV
qemu-kvm-4.2.0-4.scrmod+el8.2.0+5264+b06bacec.x86_64

Comment 3 Ademar Reis 2020-02-05 23:11:25 UTC
QEMU has been recently split into sub-components and as a one-time operation to avoid breakage of tools, we are setting the QEMU sub-component of this BZ to "General". Please review and change the sub-component if necessary the next time you review this BZ. Thanks

Comment 6 Eduardo Habkost 2020-06-25 21:31:30 UTC
Looks like the bug fixed by:

commit 0446f8121723b134ca1d1ed0b73e96d4a0a8689d
Author: Peter Xu <peterx>
Date:   Mon Jan 6 13:34:45 2020 -0700

   vfio/pci: Don't remove irqchip notifier if not registered
   
   The kvm irqchip notifier is only registered if the device supports
   INTx, however it's unconditionally removed.  If the assigned device
   does not support INTx, this will cause QEMU to crash when unplugging
   the device from the system.  Change it to conditionally remove the
   notifier only if the notify hook is setup.
   
   CC: Eduardo Habkost <ehabkost>
   CC: David Gibson <david.id.au>
   CC: Alex Williamson <alex.williamson>
   Cc: qemu-stable # v4.2
   Reported-by: yanghliu
   Debugged-by: Eduardo Habkost <ehabkost>
   Fixes: c5478fea27ac ("vfio/pci: Respond to KVM irqchip change notifier")
   Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1782678
   Signed-off-by: Peter Xu <peterx>
   Reviewed-by: David Gibson <david.id.au>
   Reviewed-by: Greg Kurz <groug>
   Signed-off-by: Alex Williamson <alex.williamson>

Comment 7 Eduardo Habkost 2020-06-25 21:34:43 UTC
Please retest using qemu-kvm-4.1.0-23.module+el8.1.1+6238+f5d69f68.3 or newer (8.1.z) or qemu-kvm-4.2.0-6.module+el8.2.0+5451+991cea0d or newer (8.2.0), to confirm this is a duplicate of bug 1782678.

Comment 8 yafu 2020-06-29 03:51:12 UTC
(In reply to Eduardo Habkost from comment #7)
> Please retest using qemu-kvm-4.1.0-23.module+el8.1.1+6238+f5d69f68.3 or
> newer (8.1.z) or qemu-kvm-4.2.0-6.module+el8.2.0+5451+991cea0d or newer
> (8.2.0), to confirm this is a duplicate of bug 1782678.

Can not reproduce the issue with qemu-kvm-4.2.0-27.module+el8.2.1+7092+9d345e72.x86_64.

Comment 9 Eduardo Habkost 2020-06-29 18:57:04 UTC
(In reply to yafu from comment #8)
> (In reply to Eduardo Habkost from comment #7)
> > Please retest using qemu-kvm-4.1.0-23.module+el8.1.1+6238+f5d69f68.3 or
> > newer (8.1.z) or qemu-kvm-4.2.0-6.module+el8.2.0+5451+991cea0d or newer
> > (8.2.0), to confirm this is a duplicate of bug 1782678.
> 
> Can not reproduce the issue with
> qemu-kvm-4.2.0-27.module+el8.2.1+7092+9d345e72.x86_64.

Thanks!  Marking as duplicate.

*** This bug has been marked as a duplicate of bug 1782678 ***


Note You need to log in before you can comment on or make changes to this bug.