Bug 856489

Summary: Modify target type of channel element from 'virtio' to 'guestfwd' will cause libvirtd crash
Product: Red Hat Enterprise Linux 6 Reporter: EricLee <bili>
Component: libvirtAssignee: Alex Jia <ajia>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: high Docs Contact:
Priority: high    
Version: 6.4CC: acathrow, ajia, dyasny, dyuan, lsu, mzhan, rwu, veillard, weizhan, whuang, yupzhang, zpeng
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: libvirt-0.10.2-0rc1.el6 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-02-21 07:23:30 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description EricLee 2012-09-12 07:00:30 UTC
Description:
Modify target type of channel element from 'virtio' to 'guestfwd' then do migration will cause libvirtd crash

Version
# rpm -qa libvirt qemu-kvm kernel
libvirt-0.10.1-1.el6.x86_64
qemu-kvm-0.12.1.2-2.308.el6.x86_64
kernel-2.6.32-298.el6.x86_64

How reproducible:
100%

Step:

1.Prepare migration environment.

2.Start a guest with the disk in nfs server, which has a channel element like:
"    <channel type='pty'>
      <target type='virtio' name='arbitrary.virtio.serial.port.name'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
"

3.# virsh dumpxml test > test.xml

4.Edit test.xml which was just saved.
From
"      <target type='virtio' name='arbitrary.virtio.serial.port.name'/>
"
to
"      <target type='guestfwd' name='arbitrary.virtio.serial.port.name'/>
"

5. Do migration with --xml option:
# virsh migrate --live --xml test.xml test qemu+ssh://${target_ip}/system --verbose
error: internal error received hangup / error event on socket
error: Failed to reconnect to the hypervisor

6. # service libvirtd status
libvirtd dead but pid file exists

Actual result
As steps

Expected result
Should give a configuration error like :
"Target channel type %s does not match source %s"
or like
"do not supported target type guestfwd of channel type='pty'"
or something like that.
But should keep libvirtd running normally.

Additional info
We have another way to reproduce the bug:
1.A shutoff guest with a channel
<channel type='spicevmc'>
      <target type='virtio' name='com.redhat.spice.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>

2.
change  <target type='virtio'..../> to  <target type='guestfwd'.../>

3.
# virsh edit test-abi
error: End of file while reading data: Input/output error
Failed. Try again? [y,n,f,?]:
error: Failed to reconnect to the hypervisor

4.
# service libvirtd status
libvirtd dead but pid file exists

Comment 2 Alex Jia 2012-09-13 16:25:44 UTC
GDB backtrace:

Program received signal SIGSEGV, Segmentation fault.
0x000000312ca7a96c in free () from /lib64/libc.so.6
(gdb) bt
#0  0x000000312ca7a96c in free () from /lib64/libc.so.6
#1  0x00007f93d8fb48b9 in virFree (ptrptr=0x7f93b800ee78) at util/memory.c:309
#2  0x00007f93d8fe8846 in virDomainChrDefFree (def=0x7f93b800ee70) at conf/domain_conf.c:1276
#3  0x00007f93d8ff40ca in virDomainChrDefParseXML (caps=0x7f93c4017f30, vmdef=0x7f93b800dee0, node=<value optimized out>, flags=2)
    at conf/domain_conf.c:5634
#4  0x00007f93d90048f9 in virDomainDefParseXML (caps=0x7f93c4017f30, xml=0x7f93b80030e0, root=0x7f93b8003440, ctxt=<value optimized out>, 
    expectedVirtTypes=<value optimized out>, flags=2) at conf/domain_conf.c:9388
#5  0x00007f93d9007aa4 in virDomainDefParseNode (caps=0x7f93c4017f30, xml=0x7f93b80030e0, root=0x7f93b8003440, expectedVirtTypes=15, 
    flags=2) at conf/domain_conf.c:9961
#6  0x00007f93d9007c0f in virDomainDefParse (xmlStr=<value optimized out>, filename=<value optimized out>, caps=0x7f93c4017f30, 
    expectedVirtTypes=15, flags=2) at conf/domain_conf.c:9911
#7  0x00007f93cbdb145d in qemudDomainDefine (conn=<value optimized out>, 
    xml=0x7f93b80009a0 "<domain type='kvm'>\n  <name>foo</name>\n  <uuid>d93dae01-6865-d193-39f3-c62773b83417</uuid>\n  <memory unit='KiB'>1048576</memory>\n  <currentMemory unit='KiB'>1048576</currentMemory>\n  <vcpu placement='"...) at qemu/qemu_driver.c:5527
#8  0x00007f93d904f336 in virDomainDefineXML (conn=0x7f93c0000bd0, 
    xml=0x7f93b80009a0 "<domain type='kvm'>\n  <name>foo</name>\n  <uuid>d93dae01-6865-d193-39f3-c62773b83417</uuid>\n  <memory unit='KiB'>1048576</memory>\n  <currentMemory unit='KiB'>1048576</currentMemory>\n  <vcpu placement='"...) at libvirt.c:7822
#9  0x000000000042973c in remoteDispatchDomainDefineXML (server=<value optimized out>, client=0x19f1050, msg=<value optimized out>, 
    rerr=0x7f93d08a7bc0, args=0x7f93b8000970, ret=0x7f93b80008c0) at remote_dispatch.h:1098
#10 remoteDispatchDomainDefineXMLHelper (server=<value optimized out>, client=0x19f1050, msg=<value optimized out>, rerr=0x7f93d08a7bc0, 
    args=0x7f93b8000970, ret=0x7f93b80008c0) at remote_dispatch.h:1078
#11 0x00007f93d90afb6d in virNetServerProgramDispatchCall (prog=0x19ec330, server=0x19e2e60, client=0x19f1050, msg=0x19fb430)
    at rpc/virnetserverprogram.c:424
#12 virNetServerProgramDispatch (prog=0x19ec330, server=0x19e2e60, client=0x19f1050, msg=0x19fb430) at rpc/virnetserverprogram.c:297
#13 0x00007f93d90ac0be in virNetServerProcessMsg (srv=<value optimized out>, client=0x19f1050, prog=<value optimized out>, msg=0x19fb430)
    at rpc/virnetserver.c:170
#14 0x00007f93d90ac84c in virNetServerHandleJob (jobOpaque=<value optimized out>, opaque=0x19e2e60) at rpc/virnetserver.c:191
#15 0x00007f93d8fbfe1e in virThreadPoolWorker (opaque=<value optimized out>) at util/threadpool.c:144
#16 0x00007f93d8fbf406 in virThreadHelper (data=<value optimized out>) at util/threads-pthread.c:161
#17 0x000000312ce07851 in start_thread () from /lib64/libpthread.so.0
#18 0x000000312cae767d in clone () from /lib64/libc.so.6
(gdb) frame 2
#2  0x00007f93d8fe8846 in virDomainChrDefFree (def=0x7f93b800ee70) at conf/domain_conf.c:1276
1276	            VIR_FREE(def->target.name);

Comment 3 Alex Jia 2012-09-13 16:29:17 UTC
Valgrind memory error detection:

==21747== Invalid free() / delete / delete[]
==21747==    at 0x4A0595D: free (vg_replace_malloc.c:366)
==21747==    by 0x4E8B818: virFree (memory.c:309)
==21747==    by 0x4EBF72C: virDomainChrDefFree (domain_conf.c:1277)
==21747==    by 0x4ECAF79: virDomainChrDefParseXML (domain_conf.c:5617)
==21747==    by 0x4EDB45A: virDomainDefParseXML (domain_conf.c:9170)
==21747==    by 0x4EDDD33: virDomainDefParseNode (domain_conf.c:9723)
==21747==    by 0x4EDDE9E: virDomainDefParse (domain_conf.c:9673)
==21747==    by 0x1273FADC: qemudDomainDefine (qemu_driver.c:5527)
==21747==    by 0x4F255C5: virDomainDefineXML (libvirt.c:7822)
==21747==    by 0x42973B: remoteDispatchDomainDefineXMLHelper (remote_dispatch.h:1098)
==21747==    by 0x4F85DFC: virNetServerProgramDispatch (virnetserverprogram.c:424)
==21747==    by 0x4F8234D: virNetServerProcessMsg (virnetserver.c:170)
==21747==  Address 0xffffffff is not stack'd, malloc'd or (recently) free'd

Comment 4 Alex Jia 2012-09-13 16:30:07 UTC
Patch for upstream and wait for review:

https://www.redhat.com/archives/libvir-list/2012-September/msg00934.html

Comment 6 Alex Jia 2012-09-13 16:45:41 UTC
In POST:

commit 9ed534f081da8ed31a0539c0f039082710e2404a
Author: Alex Jia <ajia>
Date:   Thu Sep 13 23:36:17 2012 +0800

    conf: avoid libvirt crash with empty address guestfwd channel
    
    The 'def->target.addr' hasn't been initialized in virDomainChrDefNew() and
    its value is always '0xffffffff', in addition, the following test scenario
    hasn't also include 'address' element in channel XML block, so the branch
    'if (addrStr == NULL)' is hit in virDomainChrDefParseTargetXML(), the
    programming jumps to 'error' label to release relevant resources, and the
    statement 'if (VIR_ALLOC(def->target.addr) < 0)' hasn't been executed then
    the virDomainChrDefFree() will free 'def->target.addr'(0xffffffff) via
    VIR_FREE(), which results in libvirt crash, to use valgrind can also
    find a 'Invalid free() / delete / delete[]' error. This patch just adjusts
    codes order to initialize 'def->target.addr' firstly.
    
    With this patch, libvirt hasn't crash and can get a expected error message "
    XML error: guestfwd channel does not define a target address".

Comment 8 zhe peng 2012-09-19 08:09:37 UTC
can reproduce this with build :libvirt-0.10.1-1.el6.x86_64

verify with : libvirt-0.10.2-0rc1.el6.x86_64

step:
 1 . prepare a guest with xml
 <channel type='pty'>
      <target type='virtio' name='arbitrary.virtio.serial.port.name'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
 2 . start the guest and edit the xml
  #virsh edit $guest
  change type='virtio' to 'guestfwd' and save
 error: XML error: guestfwd channel does not define a target address
 Failed. Try again? [y,n,f,?]:
 the error msg as expect.
  check libvirtd ,
 #service libvirtd status
 libvirtd (pid  15843) is running...
 no crash, verification passed.

Comment 9 errata-xmlrpc 2013-02-21 07:23:30 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2013-0276.html