Bug 1916097 - nodedev-list cause libvirt crash on host with grid host driver installed
Summary: nodedev-list cause libvirt crash on host with grid host driver installed
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux Advanced Virtualization
Classification: Red Hat
Component: libvirt
Version: 8.4
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: rc
: 8.4
Assignee: Jonathon Jongsma
QA Contact: yafu
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-01-14 08:05 UTC by Guo, Zhiyi
Modified: 2021-06-23 19:13 UTC (History)
14 users (show)

Fixed In Version: libvirt-7.0.0-1.el8
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-05-25 06:46:34 UTC
Type: Bug
Target Upstream Version: 7.0.0
Embargoed:


Attachments (Terms of Use)

Description Guo, Zhiyi 2021-01-14 08:05:37 UTC
Description of problem:
nodedev-list cause libivrt crash on host with grid host driver installed

Version-Release number of selected component (if applicable):
libvirt-client-6.10.0-1.module+el8.4.0+8898+a84e86e1.x86_64
GRID 12.0 Beta driver

How reproducible:
100%

Steps to Reproduce:
1.On vGPU host has grid driver installed, execute "virsh nodedev-list"
2.
3.

Actual results:
Libvirt crash with trace:
Jan 14 02:56:33 dell-per7425-01.lab.eng.pek2.redhat.com systemd-coredump[13056]: Process 12979 (libvirtd) of user 0 dumped core.
                                                                                 
                                                                                 Stack trace of thread 12982:
                                                                                 #0  0x00007fee583e9508 virNodeDeviceGetMdevTypesCaps (libvirt.so.0)
                                                                                 #1  0x00007fee583ee6fe virNodeDeviceGetPCIDynamicCaps (libvirt.so.0)
                                                                                 #2  0x00007fee583ee806 virNodeDeviceUpdateCaps (libvirt.so.0)
                                                                                 #3  0x00007fee583ef909 virNodeDeviceObjListExportCallback (libvirt.so.0)
                                                                                 #4  0x00007fee58308dee virHashForEach (libvirt.so.0)
                                                                                 #5  0x00007fee583f0415 virNodeDeviceObjListExport (libvirt.so.0)
                                                                                 #6  0x00007fee58530fee virConnectListAllNodeDevices (libvirt.so.0)
                                                                                 #7  0x00005602a7c310f8 remoteDispatchConnectListAllNodeDevicesHelper (libvirtd)
                                                                                 #8  0x00007fee584153e7 virNetServerProgramDispatch (libvirt.so.0)
                                                                                 #9  0x00007fee5841a706 virNetServerHandleJob (libvirt.so.0)
                                                                                 #10 0x00007fee58357a1f virThreadPoolWorker (libvirt.so.0)
                                                                                 #11 0x00007fee5835708b virThreadHelper (libvirt.so.0)
                                                                                 #12 0x00007fee5464914a start_thread (libpthread.so.0)
                                                                                 #13 0x00007fee56df7db3 __clone (libc.so.6)
                                                                                 
                                                                                 Stack trace of thread 12986:
                                                                                 #0  0x00007fee5464f2fc pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0)
                                                                                 #1  0x00007fee58356f3a virCondWait (libvirt.so.0)
                                                                                 #2  0x00007fee58357aa8 virThreadPoolWorker (libvirt.so.0)
                                                                                 #3  0x00007fee5835708b virThreadHelper (libvirt.so.0)
                                                                                 #4  0x00007fee5464914a start_thread (libpthread.so.0)
                                                                                 #5  0x00007fee56df7db3 __clone (libc.so.6)
                                                                                 
                                                                                 Stack trace of thread 12984:
                                                                                 #0  0x00007fee5464f2fc pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0)
                                                                                 #1  0x00007fee58356f3a virCondWait (libvirt.so.0)
                                                                                 #2  0x00007fee58357aeb virThreadPoolWorker (libvirt.so.0)
                                                                                 #3  0x00007fee5835708b virThreadHelper (libvirt.so.0)
                                                                                 #4  0x00007fee5464914a start_thread (libpthread.so.0)
                                                                                 #5  0x00007fee56df7db3 __clone (libc.so.6)
                                                                                 
                                                                                 Stack trace of thread 12990:
                                                                                 #0  0x00007fee5464f2fc pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0)
                                                                                 #1  0x00007fee58356f3a virCondWait (libvirt.so.0)
                                                                                 #2  0x00007fee58357aeb virThreadPoolWorker (libvirt.so.0)
                                                                                 #3  0x00007fee5835708b virThreadHelper (libvirt.so.0)
                                                                                 #4  0x00007fee5464914a start_thread (libpthread.so.0)
                                                                                 #5  0x00007fee56df7db3 __clone (libc.so.6)
                                                                                 
                                                                                 Stack trace of thread 12994:
                                                                                 #0  0x00007fee5464f2fc pthread_cond_wait@@GLIBC_2.3.2 (libpthread.so.0)
                                                                                 #1  0x00007fee58356f3a virCondWait (libvirt.so.0)
                                                                                 #2  0x00007fee58357aeb virThreadPoolWorker (libvirt.so.0)
                                                                                 #3  0x00007fee5835708b virThreadHelper (libvirt.so.0)
                                                                                 #4  0x00007fee5464914a start_thread (libpthread.so.0)
                                                                                 #5  0x00007fee56df7db3 __clone (libc.so.6)
...

Expected results:
No crash happen

Additional info:
I think this should be fixed by commit 4c4d0e2da07b5a035b26a0ff13ec27070f7c7b1a which will be included with libvirt 7.0.0
commit 4c4d0e2da07b5a035b26a0ff13ec27070f7c7b1a
Author: Jonathon Jongsma <jjongsma>
Date:   Wed Dec 2 11:52:39 2020 -0600

    conf: Fix segfault when parsing mdev types
    
    Commit f1b0890 introduced a potential crash due to incorrect operator
    precedence when accessing an element from a pointer to an array.
    
    Backtrace below:
    
      #0  virNodeDeviceGetMdevTypesCaps (sysfspath=0x7fff801661e0 "/sys/devices/pci0000:00/0000:00:02.0", mdev_types=0x7fff801c9b40, nmdev_types=0x7fff801c9b48) at ../src/conf/no
      #1  0x00007ffff7caf53d in virNodeDeviceGetPCIDynamicCaps (sysfsPath=0x7fff801661e0 "/sys/devices/pci0000:00/0000:00:02.0", pci_dev=0x7fff801c9ac8) at ../src/conf/node_devic
      #2  0x00007ffff7cae38f in virNodeDeviceUpdateCaps (def=0x7fff80168a10) at ../src/conf/node_device_conf.c:2342
      #3  0x00007ffff7cb11c0 in virNodeDeviceObjMatch (obj=0x7fff84002e50, flags=0) at ../src/conf/virnodedeviceobj.c:850
      #4  0x00007ffff7cb153d in virNodeDeviceObjListExportCallback (payload=0x7fff84002e50, name=0x7fff801cbc20 "pci_0000_00_02_0", opaque=0x7fffe2ffc6a0) at ../src/conf/virnoded
      #5  0x00007ffff7b69146 in virHashForEach (table=0x7fff9814b700 = {...}, iter=0x7ffff7cb149e <virNodeDeviceObjListExportCallback>, opaque=0x7fffe2ffc6a0) at ../src/util/virh
      #6  0x00007ffff7cb1694 in virNodeDeviceObjListExport (conn=0x7fff98013170, devs=0x7fff98154430, devices=0x7fffe2ffc798, filter=0x7ffff7cf47a1 <virConnectListAllNodeDevicesC
              at ../src/conf/virnodedeviceobj.c:943
      #7  0x00007fffe00694b2 in nodeConnectListAllNodeDevices (conn=0x7fff98013170, devices=0x7fffe2ffc798, flags=0) at ../src/node_device/node_device_driver.c:228
      #8  0x00007ffff7e703aa in virConnectListAllNodeDevices (conn=0x7fff98013170, devices=0x7fffe2ffc798, flags=0) at ../src/libvirt-nodedev.c:130
      #9  0x000055555557f796 in remoteDispatchConnectListAllNodeDevices (server=0x555555627080, client=0x5555556bf050, msg=0x5555556c0000, rerr=0x7fffe2ffc8a0, args=0x7fffd400847
              at src/remote/remote_daemon_dispatch_stubs.h:1613
      #10 0x000055555557f6f9 in remoteDispatchConnectListAllNodeDevicesHelper (server=0x555555627080, client=0x5555556bf050, msg=0x5555556c0000, rerr=0x7fffe2ffc8a0, args=0x7fffd
              at src/remote/remote_daemon_dispatch_stubs.h:1591
      #11 0x00007ffff7ce9542 in virNetServerProgramDispatchCall (prog=0x555555690c10, server=0x555555627080, client=0x5555556bf050, msg=0x5555556c0000) at ../src/rpc/virnetserver
      #12 0x00007ffff7ce90bd in virNetServerProgramDispatch (prog=0x555555690c10, server=0x555555627080, client=0x5555556bf050, msg=0x5555556c0000) at ../src/rpc/virnetserverprog
      #13 0x00007ffff7cf042b in virNetServerProcessMsg (srv=0x555555627080, client=0x5555556bf050, prog=0x555555690c10, msg=0x5555556c0000) at ../src/rpc/virnetserver.c:137
      #14 0x00007ffff7cf04eb in virNetServerHandleJob (jobOpaque=0x5555556b66b0, opaque=0x555555627080) at ../src/rpc/virnetserver.c:154
      #15 0x00007ffff7bd912f in virThreadPoolWorker (opaque=0x55555562bc70) at ../src/util/virthreadpool.c:163
      #16 0x00007ffff7bd8645 in virThreadHelper (data=0x55555562bc90) at ../src/util/virthread.c:233
      #17 0x00007ffff6d90432 in start_thread () at /lib64/libpthread.so.0
      #18 0x00007ffff75c5913 in clone () at /lib64/libc.so.6
    
    Signed-off-by: Jonathon Jongsma <jjongsma>
    Reviewed-by: Ján Tomko <jtomko>
    Signed-off-by: Ján Tomko <jtomko>

Comment 3 Guo, Zhiyi 2021-01-19 07:04:27 UTC
Test against libvirt-client-7.0.0-1.module+el8.4.0+9464+3e71831a.x86_64, not able to reproduce this issue anymore

Zhiyi

Comment 4 yafu 2021-01-20 03:59:21 UTC
Verified with libvirt-daemon-7.0.0-1.module+el8.4.0+9464+3e71831a.x86_64.

Comment 10 errata-xmlrpc 2021-05-25 06:46:34 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (virt:av bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:2098


Note You need to log in before you can comment on or make changes to this bug.