Bug 1324757
| Summary: | libvirtd crashed if destroy then start a guest which have redirdev device | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Luyao Huang <lhuang> |
| Component: | libvirt | Assignee: | Peter Krempa <pkrempa> |
| Status: | CLOSED ERRATA | QA Contact: | Virtualization Bugs <virt-bugs> |
| Severity: | high | Docs Contact: | |
| Priority: | high | ||
| Version: | 7.3 | CC: | dyuan, fjin, pkrempa, rbalakri, yanyang |
| Target Milestone: | rc | ||
| Target Release: | --- | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | libvirt-1.3.3-2.el7 | Doc Type: | Bug Fix |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2016-11-03 18:41:17 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Fixed upstream:
commit 03e8d5fb54c7c897225ed9ea56d83b894930f144
Author: Peter Krempa <pkrempa>
Date: Thu Apr 7 12:50:15 2016 +0200
qemu: perf: Fix crash/memory corruption on failed VM start
The new perf code didn't bother to clear a pointer in 'priv' causing a
double free or other memory corruption goodness if a VM failed to start.
Clear the pointer after freeing the memory.
*** Bug 1326990 has been marked as a duplicate of this bug. *** Verify this bug with libvirt-2.0.0-4.el7.x86_64:
0. use valgrind start libvirtd
1. prepare a guest which have memory device
# virsh dumpxml r7
<memory unit='KiB'>2097152</memory>
<currentMemory unit='KiB'>1048576</currentMemory>
<vcpu placement='static' current='6'>10</vcpu>
<cpu>
<numa>
<cell id='0' cpus='0-4' memory='524288' unit='KiB'/>
<cell id='1' cpus='5-9' memory='524288' unit='KiB'/>
</numa>
</cpu>
<redirdev bus='usb' type='pty'>
</redirdev>
<redirdev bus='usb' type='spicevmc'>
</redirdev>
<memory model='dimm'>
<target>
<size unit='KiB'>1048576</size>
<node>1</node>
</target>
</memory>
2. start guest:
# virsh start r7
Domain r7 started
3. edit guest to make it failed to start in next time:
# virsh edit r7
<memory model='dimm'>
<target>
<size unit='KiB'>1048576</size>
<node>3</node>
</target>
</memory>
Domain r7 XML configuration edited.
4. destroy guest and start guest again:
# virsh destroy r7
Domain r7 destroyed
# virsh start r7
error: Failed to start domain r7
error: unsupported configuration: can't add memory backend for guest node '3' as the guest has only '2' NUMA nodes configured
5. check valgrind report there is no invalid memory access in report
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHSA-2016-2577.html |
Description of problem: libvirtd crashed if destroy then start a guest which have redirdev device Version-Release number of selected component (if applicable): libvirt-1.3.3-1.el7.x86_64 How reproducible: 100% Steps to Reproduce: 1.prepare a guest like this: # virsh dumpxml rhel7.0-rhel <cpu> <numa> <cell id='0' cpus='0,2' memory='512000' unit='KiB'/> <cell id='1' cpus='1,3' memory='512000' unit='KiB'/> </numa> </cpu> <redirdev bus='usb' type='pty'> </redirdev> <redirdev bus='usb' type='spicevmc'> </redirdev> <memory model='dimm'> <target> <size unit='KiB'>141072</size> <node>0</node> </target> </memory> 2. start guest # virsh start rhel7.0-rhel Domain rhel7.0-rhel started 3. edit guest to make memory device target node to 2 which is not valid, guest will fail to start # virsh edit rhel7.0-rhel Domain rhel7.0-rhel XML configuration edited. 4. destroy and restart guest # virsh destroy rhel7.0-rhel Domain rhel7.0-rhel destroyed # virsh start rhel7.0-rhel error: Disconnected from qemu:///system due to I/O error error: Failed to start domain rhel7.0-rhel error: End of file while reading data: Input/output error Actual results: libvirtd crashed Expected results: report error Additional info: valgrind: ==13800== Thread 4: ==13800== Invalid read of size 1 ==13800== at 0x54AFF0E: virPerfFree (virperf.c:87) ==13800== by 0x1E660A3F: qemuProcessStop (qemu_process.c:5961) ==13800== by 0x1E6628EC: qemuProcessStart (qemu_process.c:5609) ==13800== by 0x1E6BF147: qemuDomainObjStart.constprop.47 (qemu_driver.c:7195) ==13800== by 0x1E6BF885: qemuDomainCreateWithFlags (qemu_driver.c:7249) ==13800== by 0x55B578B: virDomainCreate (libvirt-domain.c:6787) ==13800== by 0x14669A: remoteDispatchDomainCreate (remote_dispatch.h:4063) ==13800== by 0x14669A: remoteDispatchDomainCreateHelper (remote_dispatch.h:4039) ==13800== by 0x561F2E1: virNetServerProgramDispatchCall (virnetserverprogram.c:437) ==13800== by 0x561F2E1: virNetServerProgramDispatch (virnetserverprogram.c:307) ==13800== by 0x561A49C: virNetServerProcessMsg (virnetserver.c:137) ==13800== by 0x561A49C: virNetServerHandleJob (virnetserver.c:158) ==13800== by 0x55111C4: virThreadPoolWorker (virthreadpool.c:145) ==13800== by 0x55106E7: virThreadHelper (virthread.c:206) ==13800== by 0x807EDC4: start_thread (in /usr/lib64/libpthread-2.17.so) ==13800== Address 0x231ddd28 is 8 bytes inside a block of size 16 free'd ==13800== at 0x4C2AD17: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==13800== by 0x54AC2A9: virFree (viralloc.c:582) ==13800== by 0x54AFF1D: virPerfFree (virperf.c:91) ==13800== by 0x1E660A3F: qemuProcessStop (qemu_process.c:5961) ==13800== by 0x1E6B06A7: qemuDomainDestroyFlags (qemu_driver.c:2291) ==13800== by 0x55A2EAB: virDomainDestroy (libvirt-domain.c:479) ==13800== by 0x1462AA: remoteDispatchDomainDestroy (remote_dispatch.h:4456) ==13800== by 0x1462AA: remoteDispatchDomainDestroyHelper (remote_dispatch.h:4432) ==13800== by 0x561F2E1: virNetServerProgramDispatchCall (virnetserverprogram.c:437) ==13800== by 0x561F2E1: virNetServerProgramDispatch (virnetserverprogram.c:307) ==13800== by 0x561A49C: virNetServerProcessMsg (virnetserver.c:137) ==13800== by 0x561A49C: virNetServerHandleJob (virnetserver.c:158) ==13800== by 0x55111C4: virThreadPoolWorker (virthreadpool.c:145) ==13800== by 0x55106E7: virThreadHelper (virthread.c:206) ==13800== by 0x807EDC4: start_thread (in /usr/lib64/libpthread-2.17.so) ==13800== ==13800== Invalid free() / delete / delete[] / realloc() ==13800== at 0x4C2AD17: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==13800== by 0x54AC2A9: virFree (viralloc.c:582) ==13800== by 0x54AFF1D: virPerfFree (virperf.c:91) ==13800== by 0x1E660A3F: qemuProcessStop (qemu_process.c:5961) ==13800== by 0x1E6628EC: qemuProcessStart (qemu_process.c:5609) ==13800== by 0x1E6BF147: qemuDomainObjStart.constprop.47 (qemu_driver.c:7195) ==13800== by 0x1E6BF885: qemuDomainCreateWithFlags (qemu_driver.c:7249) ==13800== by 0x55B578B: virDomainCreate (libvirt-domain.c:6787) ==13800== by 0x14669A: remoteDispatchDomainCreate (remote_dispatch.h:4063) ==13800== by 0x14669A: remoteDispatchDomainCreateHelper (remote_dispatch.h:4039) ==13800== by 0x561F2E1: virNetServerProgramDispatchCall (virnetserverprogram.c:437) ==13800== by 0x561F2E1: virNetServerProgramDispatch (virnetserverprogram.c:307) ==13800== by 0x561A49C: virNetServerProcessMsg (virnetserver.c:137) ==13800== by 0x561A49C: virNetServerHandleJob (virnetserver.c:158) ==13800== by 0x55111C4: virThreadPoolWorker (virthreadpool.c:145) ==13800== Address 0x231ddd20 is 0 bytes inside a block of size 16 free'd ==13800== at 0x4C2AD17: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==13800== by 0x54AC2A9: virFree (viralloc.c:582) ==13800== by 0x54AFF1D: virPerfFree (virperf.c:91) ==13800== by 0x1E660A3F: qemuProcessStop (qemu_process.c:5961) ==13800== by 0x1E6B06A7: qemuDomainDestroyFlags (qemu_driver.c:2291) ==13800== by 0x55A2EAB: virDomainDestroy (libvirt-domain.c:479) ==13800== by 0x1462AA: remoteDispatchDomainDestroy (remote_dispatch.h:4456) ==13800== by 0x1462AA: remoteDispatchDomainDestroyHelper (remote_dispatch.h:4432) ==13800== by 0x561F2E1: virNetServerProgramDispatchCall (virnetserverprogram.c:437) ==13800== by 0x561F2E1: virNetServerProgramDispatch (virnetserverprogram.c:307) ==13800== by 0x561A49C: virNetServerProcessMsg (virnetserver.c:137) ==13800== by 0x561A49C: virNetServerHandleJob (virnetserver.c:158) ==13800== by 0x55111C4: virThreadPoolWorker (virthreadpool.c:145) ==13800== by 0x55106E7: virThreadHelper (virthread.c:206) ==13800== by 0x807EDC4: start_thread (in /usr/lib64/libpthread-2.17.so) ==13800== gdb: Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7f906404a700 (LWP 14088)] 0x00007f9070fffaad in malloc_consolidate () from /lib64/libc.so.6 (gdb) bt #0 0x00007f9070fffaad in malloc_consolidate () from /lib64/libc.so.6 #1 0x00007f9071000f06 in _int_free () from /lib64/libc.so.6 #2 0x00007f9073cea2aa in virFree (ptrptr=ptrptr@entry=0x7f9064049608) at util/viralloc.c:582 #3 0x00007f9073d6dd9f in virDomainRedirdevDefFree (def=0x7f903801b340) at conf/domain_conf.c:2239 #4 0x00007f9073d7f2e4 in virDomainDefFree (def=0x7f9038014af0) at conf/domain_conf.c:2567 #5 0x00007f905b055bc0 in qemuProcessStop (driver=driver@entry=0x7f9050128a60, vm=vm@entry=0x7f90501ea450, reason=reason@entry=VIR_DOMAIN_SHUTOFF_FAILED, asyncJob=asyncJob@entry=QEMU_ASYNC_JOB_START, flags=flags@entry=2) at qemu/qemu_process.c:6030 #6 0x00007f905b0578ed in qemuProcessStart (conn=conn@entry=0x7f9044000ac0, driver=driver@entry=0x7f9050128a60, vm=vm@entry=0x7f90501ea450, asyncJob=asyncJob@entry=QEMU_ASYNC_JOB_START, migrateFrom=migrateFrom@entry=0x0, migrateFd=migrateFd@entry=-1, migratePath=migratePath@entry=0x0, snapshot=snapshot@entry=0x0, vmop=vmop@entry=VIR_NETDEV_VPORT_PROFILE_OP_CREATE, flags=flags@entry=1) at qemu/qemu_process.c:5609 #7 0x00007f905b0b4148 in qemuDomainObjStart (conn=0x7f9044000ac0, driver=driver@entry=0x7f9050128a60, vm=0x7f90501ea450, flags=flags@entry=0, asyncJob=QEMU_ASYNC_JOB_START) at qemu/qemu_driver.c:7195 #8 0x00007f905b0b4886 in qemuDomainCreateWithFlags (dom=0x7f904c002dd0, flags=0) at qemu/qemu_driver.c:7249 #9 0x00007f9073df378c in virDomainCreate (domain=domain@entry=0x7f904c002dd0) at libvirt-domain.c:6787 #10 0x00007f9074a6269b in remoteDispatchDomainCreate (server=0x7f9076498ab0, msg=0x7f90764b0300, args=<optimized out>, rerr=0x7f9064049c30, client=0x7f90764b0370) at remote_dispatch.h:4063 #11 remoteDispatchDomainCreateHelper (server=0x7f9076498ab0, client=0x7f90764b0370, msg=0x7f90764b0300, rerr=0x7f9064049c30, args=<optimized out>, ret=0x7f904c000d50) at remote_dispatch.h:4039 #12 0x00007f9073e5d2e2 in virNetServerProgramDispatchCall (msg=0x7f90764b0300, client=0x7f90764b0370, server=0x7f9076498ab0, prog=0x7f90764ac2a0) at rpc/virnetserverprogram.c:437 #13 virNetServerProgramDispatch (prog=0x7f90764ac2a0, server=server@entry=0x7f9076498ab0, client=0x7f90764b0370, msg=0x7f90764b0300) at rpc/virnetserverprogram.c:307 #14 0x00007f9073e5849d in virNetServerProcessMsg (msg=<optimized out>, prog=<optimized out>, client=<optimized out>, srv=0x7f9076498ab0) at rpc/virnetserver.c:137 #15 virNetServerHandleJob (jobOpaque=<optimized out>, opaque=0x7f9076498ab0) at rpc/virnetserver.c:158 #16 0x00007f9073d4f1c5 in virThreadPoolWorker (opaque=opaque@entry=0x7f907648d440) at util/virthreadpool.c:145 #17 0x00007f9073d4e6e8 in virThreadHelper (data=<optimized out>) at util/virthread.c:206 #18 0x00007f9071353dc5 in start_thread () from /lib64/libpthread.so.0 #19 0x00007f907107a28d in clone () from /lib64/libc.so.6