Red Hat Bugzilla – Bug 856489
Modify target type of channel element from 'virtio' to 'guestfwd' will cause libvirtd crash
Last modified: 2013-02-21 02:23:30 EST
Description： Modify target type of channel element from 'virtio' to 'guestfwd' then do migration will cause libvirtd crash Version # rpm -qa libvirt qemu-kvm kernel libvirt-0.10.1-1.el6.x86_64 qemu-kvm-0.12.1.2-2.308.el6.x86_64 kernel-2.6.32-298.el6.x86_64 How reproducible: 100% Step: 1.Prepare migration environment. 2.Start a guest with the disk in nfs server, which has a channel element like: " <channel type='pty'> <target type='virtio' name='arbitrary.virtio.serial.port.name'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> " 3.# virsh dumpxml test > test.xml 4.Edit test.xml which was just saved. From " <target type='virtio' name='arbitrary.virtio.serial.port.name'/> " to " <target type='guestfwd' name='arbitrary.virtio.serial.port.name'/> " 5. Do migration with --xml option: # virsh migrate --live --xml test.xml test qemu+ssh://${target_ip}/system --verbose error: internal error received hangup / error event on socket error: Failed to reconnect to the hypervisor 6. # service libvirtd status libvirtd dead but pid file exists Actual result As steps Expected result Should give a configuration error like : "Target channel type %s does not match source %s" or like "do not supported target type guestfwd of channel type='pty'" or something like that. But should keep libvirtd running normally. Additional info We have another way to reproduce the bug: 1.A shutoff guest with a channel <channel type='spicevmc'> <target type='virtio' name='com.redhat.spice.0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> 2. change <target type='virtio'..../> to <target type='guestfwd'.../> 3. # virsh edit test-abi error: End of file while reading data: Input/output error Failed. Try again? [y,n,f,?]: error: Failed to reconnect to the hypervisor 4. # service libvirtd status libvirtd dead but pid file exists
GDB backtrace: Program received signal SIGSEGV, Segmentation fault. 0x000000312ca7a96c in free () from /lib64/libc.so.6 (gdb) bt #0 0x000000312ca7a96c in free () from /lib64/libc.so.6 #1 0x00007f93d8fb48b9 in virFree (ptrptr=0x7f93b800ee78) at util/memory.c:309 #2 0x00007f93d8fe8846 in virDomainChrDefFree (def=0x7f93b800ee70) at conf/domain_conf.c:1276 #3 0x00007f93d8ff40ca in virDomainChrDefParseXML (caps=0x7f93c4017f30, vmdef=0x7f93b800dee0, node=<value optimized out>, flags=2) at conf/domain_conf.c:5634 #4 0x00007f93d90048f9 in virDomainDefParseXML (caps=0x7f93c4017f30, xml=0x7f93b80030e0, root=0x7f93b8003440, ctxt=<value optimized out>, expectedVirtTypes=<value optimized out>, flags=2) at conf/domain_conf.c:9388 #5 0x00007f93d9007aa4 in virDomainDefParseNode (caps=0x7f93c4017f30, xml=0x7f93b80030e0, root=0x7f93b8003440, expectedVirtTypes=15, flags=2) at conf/domain_conf.c:9961 #6 0x00007f93d9007c0f in virDomainDefParse (xmlStr=<value optimized out>, filename=<value optimized out>, caps=0x7f93c4017f30, expectedVirtTypes=15, flags=2) at conf/domain_conf.c:9911 #7 0x00007f93cbdb145d in qemudDomainDefine (conn=<value optimized out>, xml=0x7f93b80009a0 "<domain type='kvm'>\n <name>foo</name>\n <uuid>d93dae01-6865-d193-39f3-c62773b83417</uuid>\n <memory unit='KiB'>1048576</memory>\n <currentMemory unit='KiB'>1048576</currentMemory>\n <vcpu placement='"...) at qemu/qemu_driver.c:5527 #8 0x00007f93d904f336 in virDomainDefineXML (conn=0x7f93c0000bd0, xml=0x7f93b80009a0 "<domain type='kvm'>\n <name>foo</name>\n <uuid>d93dae01-6865-d193-39f3-c62773b83417</uuid>\n <memory unit='KiB'>1048576</memory>\n <currentMemory unit='KiB'>1048576</currentMemory>\n <vcpu placement='"...) at libvirt.c:7822 #9 0x000000000042973c in remoteDispatchDomainDefineXML (server=<value optimized out>, client=0x19f1050, msg=<value optimized out>, rerr=0x7f93d08a7bc0, args=0x7f93b8000970, ret=0x7f93b80008c0) at remote_dispatch.h:1098 #10 remoteDispatchDomainDefineXMLHelper (server=<value optimized out>, client=0x19f1050, msg=<value optimized out>, rerr=0x7f93d08a7bc0, args=0x7f93b8000970, ret=0x7f93b80008c0) at remote_dispatch.h:1078 #11 0x00007f93d90afb6d in virNetServerProgramDispatchCall (prog=0x19ec330, server=0x19e2e60, client=0x19f1050, msg=0x19fb430) at rpc/virnetserverprogram.c:424 #12 virNetServerProgramDispatch (prog=0x19ec330, server=0x19e2e60, client=0x19f1050, msg=0x19fb430) at rpc/virnetserverprogram.c:297 #13 0x00007f93d90ac0be in virNetServerProcessMsg (srv=<value optimized out>, client=0x19f1050, prog=<value optimized out>, msg=0x19fb430) at rpc/virnetserver.c:170 #14 0x00007f93d90ac84c in virNetServerHandleJob (jobOpaque=<value optimized out>, opaque=0x19e2e60) at rpc/virnetserver.c:191 #15 0x00007f93d8fbfe1e in virThreadPoolWorker (opaque=<value optimized out>) at util/threadpool.c:144 #16 0x00007f93d8fbf406 in virThreadHelper (data=<value optimized out>) at util/threads-pthread.c:161 #17 0x000000312ce07851 in start_thread () from /lib64/libpthread.so.0 #18 0x000000312cae767d in clone () from /lib64/libc.so.6 (gdb) frame 2 #2 0x00007f93d8fe8846 in virDomainChrDefFree (def=0x7f93b800ee70) at conf/domain_conf.c:1276 1276 VIR_FREE(def->target.name);
Valgrind memory error detection: ==21747== Invalid free() / delete / delete[] ==21747== at 0x4A0595D: free (vg_replace_malloc.c:366) ==21747== by 0x4E8B818: virFree (memory.c:309) ==21747== by 0x4EBF72C: virDomainChrDefFree (domain_conf.c:1277) ==21747== by 0x4ECAF79: virDomainChrDefParseXML (domain_conf.c:5617) ==21747== by 0x4EDB45A: virDomainDefParseXML (domain_conf.c:9170) ==21747== by 0x4EDDD33: virDomainDefParseNode (domain_conf.c:9723) ==21747== by 0x4EDDE9E: virDomainDefParse (domain_conf.c:9673) ==21747== by 0x1273FADC: qemudDomainDefine (qemu_driver.c:5527) ==21747== by 0x4F255C5: virDomainDefineXML (libvirt.c:7822) ==21747== by 0x42973B: remoteDispatchDomainDefineXMLHelper (remote_dispatch.h:1098) ==21747== by 0x4F85DFC: virNetServerProgramDispatch (virnetserverprogram.c:424) ==21747== by 0x4F8234D: virNetServerProcessMsg (virnetserver.c:170) ==21747== Address 0xffffffff is not stack'd, malloc'd or (recently) free'd
Patch for upstream and wait for review: https://www.redhat.com/archives/libvir-list/2012-September/msg00934.html
In POST: commit 9ed534f081da8ed31a0539c0f039082710e2404a Author: Alex Jia <ajia@redhat.com> Date: Thu Sep 13 23:36:17 2012 +0800 conf: avoid libvirt crash with empty address guestfwd channel The 'def->target.addr' hasn't been initialized in virDomainChrDefNew() and its value is always '0xffffffff', in addition, the following test scenario hasn't also include 'address' element in channel XML block, so the branch 'if (addrStr == NULL)' is hit in virDomainChrDefParseTargetXML(), the programming jumps to 'error' label to release relevant resources, and the statement 'if (VIR_ALLOC(def->target.addr) < 0)' hasn't been executed then the virDomainChrDefFree() will free 'def->target.addr'(0xffffffff) via VIR_FREE(), which results in libvirt crash, to use valgrind can also find a 'Invalid free() / delete / delete[]' error. This patch just adjusts codes order to initialize 'def->target.addr' firstly. With this patch, libvirt hasn't crash and can get a expected error message " XML error: guestfwd channel does not define a target address".
can reproduce this with build :libvirt-0.10.1-1.el6.x86_64 verify with : libvirt-0.10.2-0rc1.el6.x86_64 step: 1 . prepare a guest with xml <channel type='pty'> <target type='virtio' name='arbitrary.virtio.serial.port.name'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> 2 . start the guest and edit the xml #virsh edit $guest change type='virtio' to 'guestfwd' and save error: XML error: guestfwd channel does not define a target address Failed. Try again? [y,n,f,?]: the error msg as expect. check libvirtd , #service libvirtd status libvirtd (pid 15843) is running... no crash, verification passed.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHSA-2013-0276.html