856489 – Modify target type of channel element from 'virtio' to 'guestfwd' will cause libvirtd crash

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 856489 - Modify target type of channel element from 'virtio' to 'guestfwd' will cause libvirtd crash

Summary: Modify target type of channel element from 'virtio' to 'guestfwd' will cause ...

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat Enterprise Linux 6
Classification:	Red Hat
Component:	libvirt
Sub Component:
Version:	6.4
Hardware:	x86_64
OS:	Linux
Priority:	high
Severity:	high
Target Milestone:	rc
Target Release:	---
Assignee:	Alex Jia
QA Contact:	Virtualization Bugs
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2012-09-12 07:00 UTC by EricLee
Modified:	2013-02-21 07:23 UTC (History)
CC List:	12 users (show)
Fixed In Version:	libvirt-0.10.2-0rc1.el6
Doc Type:	Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed:	2013-02-21 07:23:30 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Red Hat Product Errata	RHSA-2013:0276	0	normal	SHIPPED_LIVE	Moderate: libvirt security, bug fix, and enhancement update	2013-02-20 21:18:26 UTC

Description EricLee 2012-09-12 07:00:30 UTC

Description：
Modify target type of channel element from 'virtio' to 'guestfwd' then do migration will cause libvirtd crash

Version
# rpm -qa libvirt qemu-kvm kernel
libvirt-0.10.1-1.el6.x86_64
qemu-kvm-0.12.1.2-2.308.el6.x86_64
kernel-2.6.32-298.el6.x86_64

How reproducible:
100%

Step:

1.Prepare migration environment.

2.Start a guest with the disk in nfs server, which has a channel element like:
"    <channel type='pty'>
      <target type='virtio' name='arbitrary.virtio.serial.port.name'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
"

3.# virsh dumpxml test > test.xml

4.Edit test.xml which was just saved.
From
"      <target type='virtio' name='arbitrary.virtio.serial.port.name'/>
"
to
"      <target type='guestfwd' name='arbitrary.virtio.serial.port.name'/>
"

5. Do migration with --xml option:
# virsh migrate --live --xml test.xml test qemu+ssh://${target_ip}/system --verbose
error: internal error received hangup / error event on socket
error: Failed to reconnect to the hypervisor

6. # service libvirtd status
libvirtd dead but pid file exists

Actual result
As steps

Expected result
Should give a configuration error like :
"Target channel type %s does not match source %s"
or like
"do not supported target type guestfwd of channel type='pty'"
or something like that.
But should keep libvirtd running normally.

Additional info
We have another way to reproduce the bug:
1.A shutoff guest with a channel
<channel type='spicevmc'>
      <target type='virtio' name='com.redhat.spice.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>

2.
change  <target type='virtio'..../> to  <target type='guestfwd'.../>

3.
# virsh edit test-abi
error: End of file while reading data: Input/output error
Failed. Try again? [y,n,f,?]:
error: Failed to reconnect to the hypervisor

4.
# service libvirtd status
libvirtd dead but pid file exists

Comment 2 Alex Jia 2012-09-13 16:25:44 UTC

GDB backtrace:

Program received signal SIGSEGV, Segmentation fault.
0x000000312ca7a96c in free () from /lib64/libc.so.6
(gdb) bt
#0  0x000000312ca7a96c in free () from /lib64/libc.so.6
#1  0x00007f93d8fb48b9 in virFree (ptrptr=0x7f93b800ee78) at util/memory.c:309
#2  0x00007f93d8fe8846 in virDomainChrDefFree (def=0x7f93b800ee70) at conf/domain_conf.c:1276
#3  0x00007f93d8ff40ca in virDomainChrDefParseXML (caps=0x7f93c4017f30, vmdef=0x7f93b800dee0, node=<value optimized out>, flags=2)
    at conf/domain_conf.c:5634
#4  0x00007f93d90048f9 in virDomainDefParseXML (caps=0x7f93c4017f30, xml=0x7f93b80030e0, root=0x7f93b8003440, ctxt=<value optimized out>, 
    expectedVirtTypes=<value optimized out>, flags=2) at conf/domain_conf.c:9388
#5  0x00007f93d9007aa4 in virDomainDefParseNode (caps=0x7f93c4017f30, xml=0x7f93b80030e0, root=0x7f93b8003440, expectedVirtTypes=15, 
    flags=2) at conf/domain_conf.c:9961
#6  0x00007f93d9007c0f in virDomainDefParse (xmlStr=<value optimized out>, filename=<value optimized out>, caps=0x7f93c4017f30, 
    expectedVirtTypes=15, flags=2) at conf/domain_conf.c:9911
#7  0x00007f93cbdb145d in qemudDomainDefine (conn=<value optimized out>, 
    xml=0x7f93b80009a0 "<domain type='kvm'>\n  <name>foo</name>\n  <uuid>d93dae01-6865-d193-39f3-c62773b83417</uuid>\n  <memory unit='KiB'>1048576</memory>\n  <currentMemory unit='KiB'>1048576</currentMemory>\n  <vcpu placement='"...) at qemu/qemu_driver.c:5527
#8  0x00007f93d904f336 in virDomainDefineXML (conn=0x7f93c0000bd0, 
    xml=0x7f93b80009a0 "<domain type='kvm'>\n  <name>foo</name>\n  <uuid>d93dae01-6865-d193-39f3-c62773b83417</uuid>\n  <memory unit='KiB'>1048576</memory>\n  <currentMemory unit='KiB'>1048576</currentMemory>\n  <vcpu placement='"...) at libvirt.c:7822
#9  0x000000000042973c in remoteDispatchDomainDefineXML (server=<value optimized out>, client=0x19f1050, msg=<value optimized out>, 
    rerr=0x7f93d08a7bc0, args=0x7f93b8000970, ret=0x7f93b80008c0) at remote_dispatch.h:1098
#10 remoteDispatchDomainDefineXMLHelper (server=<value optimized out>, client=0x19f1050, msg=<value optimized out>, rerr=0x7f93d08a7bc0, 
    args=0x7f93b8000970, ret=0x7f93b80008c0) at remote_dispatch.h:1078
#11 0x00007f93d90afb6d in virNetServerProgramDispatchCall (prog=0x19ec330, server=0x19e2e60, client=0x19f1050, msg=0x19fb430)
    at rpc/virnetserverprogram.c:424
#12 virNetServerProgramDispatch (prog=0x19ec330, server=0x19e2e60, client=0x19f1050, msg=0x19fb430) at rpc/virnetserverprogram.c:297
#13 0x00007f93d90ac0be in virNetServerProcessMsg (srv=<value optimized out>, client=0x19f1050, prog=<value optimized out>, msg=0x19fb430)
    at rpc/virnetserver.c:170
#14 0x00007f93d90ac84c in virNetServerHandleJob (jobOpaque=<value optimized out>, opaque=0x19e2e60) at rpc/virnetserver.c:191
#15 0x00007f93d8fbfe1e in virThreadPoolWorker (opaque=<value optimized out>) at util/threadpool.c:144
#16 0x00007f93d8fbf406 in virThreadHelper (data=<value optimized out>) at util/threads-pthread.c:161
#17 0x000000312ce07851 in start_thread () from /lib64/libpthread.so.0
#18 0x000000312cae767d in clone () from /lib64/libc.so.6
(gdb) frame 2
#2  0x00007f93d8fe8846 in virDomainChrDefFree (def=0x7f93b800ee70) at conf/domain_conf.c:1276
1276	            VIR_FREE(def->target.name);

Comment 3 Alex Jia 2012-09-13 16:29:17 UTC

Valgrind memory error detection:

==21747== Invalid free() / delete / delete[]
==21747==    at 0x4A0595D: free (vg_replace_malloc.c:366)
==21747==    by 0x4E8B818: virFree (memory.c:309)
==21747==    by 0x4EBF72C: virDomainChrDefFree (domain_conf.c:1277)
==21747==    by 0x4ECAF79: virDomainChrDefParseXML (domain_conf.c:5617)
==21747==    by 0x4EDB45A: virDomainDefParseXML (domain_conf.c:9170)
==21747==    by 0x4EDDD33: virDomainDefParseNode (domain_conf.c:9723)
==21747==    by 0x4EDDE9E: virDomainDefParse (domain_conf.c:9673)
==21747==    by 0x1273FADC: qemudDomainDefine (qemu_driver.c:5527)
==21747==    by 0x4F255C5: virDomainDefineXML (libvirt.c:7822)
==21747==    by 0x42973B: remoteDispatchDomainDefineXMLHelper (remote_dispatch.h:1098)
==21747==    by 0x4F85DFC: virNetServerProgramDispatch (virnetserverprogram.c:424)
==21747==    by 0x4F8234D: virNetServerProcessMsg (virnetserver.c:170)
==21747==  Address 0xffffffff is not stack'd, malloc'd or (recently) free'd

Comment 4 Alex Jia 2012-09-13 16:30:07 UTC

Patch for upstream and wait for review:

https://www.redhat.com/archives/libvir-list/2012-September/msg00934.html

Comment 6 Alex Jia 2012-09-13 16:45:41 UTC

In POST:

commit 9ed534f081da8ed31a0539c0f039082710e2404a
Author: Alex Jia <ajia>
Date:   Thu Sep 13 23:36:17 2012 +0800

    conf: avoid libvirt crash with empty address guestfwd channel
    
    The 'def->target.addr' hasn't been initialized in virDomainChrDefNew() and
    its value is always '0xffffffff', in addition, the following test scenario
    hasn't also include 'address' element in channel XML block, so the branch
    'if (addrStr == NULL)' is hit in virDomainChrDefParseTargetXML(), the
    programming jumps to 'error' label to release relevant resources, and the
    statement 'if (VIR_ALLOC(def->target.addr) < 0)' hasn't been executed then
    the virDomainChrDefFree() will free 'def->target.addr'(0xffffffff) via
    VIR_FREE(), which results in libvirt crash, to use valgrind can also
    find a 'Invalid free() / delete / delete[]' error. This patch just adjusts
    codes order to initialize 'def->target.addr' firstly.
    
    With this patch, libvirt hasn't crash and can get a expected error message "
    XML error: guestfwd channel does not define a target address".

Comment 8 zhe peng 2012-09-19 08:09:37 UTC

can reproduce this with build :libvirt-0.10.1-1.el6.x86_64

verify with : libvirt-0.10.2-0rc1.el6.x86_64

step:
 1 . prepare a guest with xml
 <channel type='pty'>
      <target type='virtio' name='arbitrary.virtio.serial.port.name'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
 2 . start the guest and edit the xml
  #virsh edit $guest
  change type='virtio' to 'guestfwd' and save
 error: XML error: guestfwd channel does not define a target address
 Failed. Try again? [y,n,f,?]:
 the error msg as expect.
  check libvirtd ,
 #service libvirtd status
 libvirtd (pid  15843) is running...
 no crash, verification passed.

Comment 9 errata-xmlrpc 2013-02-21 07:23:30 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2013-0276.html

Note You need to log in before you can comment on or make changes to this bug.