Bug 984821

Summary: Crash of libvirtd without guest agent configuration
Product: Red Hat Enterprise Linux 6 Reporter: Alex Jia <ajia>
Component: libvirtAssignee: Alex Jia <ajia>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: high Docs Contact:
Priority: high    
Version: 6.5CC: acathrow, dyuan, eblake, herrold, jdenemar, jmiao, pkrempa, weizhan
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: libvirt-0.10.2-21.el6 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-11-21 09:07:04 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 986383, 986386    

Description Alex Jia 2013-07-16 06:28:43 UTC
Description of problem:
If users haven't configured guest agent then qemuAgentCommand() will
dereference a NULL 'mon' pointer, which causes crash of libvirtd.

Version-Release number of selected component (if applicable):
# rpm -q libvirt qemu-kvm kernel
libvirt-0.10.2-20.el6.x86_64
package qemu-kvm is not installed
kernel-2.6.32-288.el6.x86_64

Notes, need to bypass bug 984793 and successfully start a guest firstly.

How reproducible:
always

Steps to Reproduce:
1. virsh start <domain>  (without guest agent configuration)
2. virsh vcpucount <domain> --guest


Actual results:
# virsh vcpucount hello --guest
error: End of file while reading data: Input/output error
error: One or more references were leaked after disconnect from the hypervisor
error: Failed to reconnect to the hypervisor

Expected results:
fix it.

Additional info:

Thread 11 (Thread 0x7f394b091700 (LWP 25677)):
#0  virNetServerFatalSignal (sig=11, siginfo=<value optimized out>, context=<value optimized out>) at rpc/virnetserver.c:326
        sig_action = {__sigaction_handler = {sa_handler = 0x70, sa_sigaction = 0x70}, sa_mask = {__val = {139884217688997, 206158430248, 139884048746048, 206158430256, 139884048746120, 206158430256, 
              139884048746136, 139884048745936, 139884048745520, 1, 139884048745648, 28, 139884265387128, 139884048746208, 139884218222960, 139887012315137}}, sa_flags = 805312336, 
          sa_restorer = 0x7f3930000020}
        origerrno = <value optimized out>
#1  <signal handler called>
No symbol table info available.
#2  qemuAgentCommand (mon=0x0, cmd=0x7f39300017b0, reply=0x7f394b090910, seconds=-2) at qemu/qemu_agent.c:975
        ret = <value optimized out>
        msg = {txBuffer = 0x0, txOffset = 941387424, txLength = 32569, rxBuffer = 0x7f394b090890 "\240\b\tK9\177", rxLength = 1898801172, rxObject = 0x0, finished = 224}
        cmdstr = 0x7f394b0908d0 "\240n\034\070\071\177"
        await_event = <value optimized out>
        __FUNCTION__ = "qemuAgentCommand"
        __func__ = "qemuAgentCommand"
#3  0x00007f39429507f6 in qemuAgentGetVCPUs (mon=0x0, info=0x7f394b0909b8) at qemu/qemu_agent.c:1475
        ret = -1
        i = <value optimized out>
        cmd = 0x7f39300017b0
        reply = 0x0
        data = 0x0
        ndata = <value optimized out>
        __FUNCTION__ = "qemuAgentGetVCPUs"
#4  0x00007f39429d9857 in qemuDomainGetVcpusFlags (dom=<value optimized out>, flags=9) at qemu/qemu_driver.c:4849
        driver = 0x7f3938009840
        priv = 0x7f39381c6ea0
        vm = 0x7f39381c6d40
        def = 0x7f39381d3a00
        ret = -1
        caps = 0x7f39381c4fe0
        cpuinfo = 0x0
        ncpuinfo = -1
        i = <value optimized out>
        __FUNCTION__ = "qemuDomainGetVcpusFlags"
#5  0x00007f3957dffd8d in virDomainGetVcpusFlags (domain=0x7f39300009c0, flags=8) at libvirt.c:9843
        ret = <value optimized out>
        conn = <value optimized out>
        __func__ = "virDomainGetVcpusFlags"
        __FUNCTION__ = "virDomainGetVcpusFlags"
#6  0x00007f395882dceb in remoteDispatchDomainGetVcpusFlags (server=<value optimized out>, client=<value optimized out>, msg=<value optimized out>, rerr=0x7f394b090ba0, args=0x7f3930000a00, 
    ret=0x7f3930001e60) at remote_dispatch.h:4343
        rv = -1
        dom = 0x7f39300009c0
        num = <value optimized out>
        priv = <value optimized out>

Comment 1 Alex Jia 2013-07-16 06:34:01 UTC
Path on upstream:
https://www.redhat.com/archives/libvir-list/2013-July/msg00980.html

Comment 3 Peter Krempa 2013-07-16 12:26:39 UTC
Fix upstream in v1.1.0-199-g96518d4 with:

commit 96518d4316b711c72205117f8d5c967d5127bbb6
Author: Alex Jia <ajia>
Date:   Tue Jul 16 17:30:20 2013 +0800

    qemu: Prevent crash of libvirtd without guest agent configuration
    
    If users haven't configured guest agent then qemuAgentCommand() will
    dereference a NULL 'mon' pointer, which causes crash of libvirtd when
    using agent based cpu (un)plug.
    
    With the patch, when the qemu-ga service isn't running in the guest,
    a expected error "error: Guest agent is not responding: Guest agent
    not available for now" will be raised, and the error "error: argument
    unsupported: QEMU guest agent is not configured" is raised when the
    guest hasn't configured guest agent.
    
    GDB backtrace:
    
     (gdb) bt
     #0  virNetServerFatalSignal (sig=11, siginfo=<value optimized out>, context=<value optimized out>) at rpc/virnetserver.c:326
     #1  <signal handler called>
     #2  qemuAgentCommand (mon=0x0, cmd=0x7f39300017b0, reply=0x7f394b090910, seconds=-2) at qemu/qemu_agent.c:975
     #3  0x00007f39429507f6 in qemuAgentGetVCPUs (mon=0x0, info=0x7f394b0909b8) at qemu/qemu_agent.c:1475
     #4  0x00007f39429d9857 in qemuDomainGetVcpusFlags (dom=<value optimized out>, flags=9) at qemu/qemu_driver.c:4849
     #5  0x00007f3957dffd8d in virDomainGetVcpusFlags (domain=0x7f39300009c0, flags=8) at libvirt.c:9843
    
    How to reproduce?
    
     # To start a guest without guest agent configuration
     # then run the following cmdline
    
     # virsh vcpucount foobar --guest
     error: End of file while reading data: Input/output error
     error: One or more references were leaked after disconnect from the hypervisor
     error: Failed to reconnect to the hypervisor
    
    RHBZ: https://bugzilla.redhat.com/show_bug.cgi?id=984821
    
    Signed-off-by: Alex Jia <ajia>
    Signed-off-by: Peter Krempa <pkrempa>

Comment 5 Peter Krempa 2013-07-16 15:10:54 UTC
While testing the patch above I also found another bug in this section of code that also crashes the daemon. This issue is fixed upstream with:

commit dfc692350a04a70b4ca65667c30869b3bfdaf034
Author: Peter Krempa <pkrempa>
Date:   Tue Jul 16 15:39:06 2013 +0200

    qemu: Fix double free of returned JSON array in qemuAgentGetVCPUs()
    
    A part of the returned monitor response was freed twice and caused
    crashes of the daemon when using guest agent cpu count retrieval.
    
     # virsh vcpucount dom --guest
    
    Introduced in v1.0.6-48-gc6afcb0

v1.1.0-208-gdfc6923

Comment 11 Jincheng Miao 2013-07-23 03:08:18 UTC
1.
# rpm -q libvirt
libvirt-0.10.2-21.el6.x86_64

2. no guest-agnet configuration
# virsh dumpxml r6 | grep agent

3. start 
# virsh start r6
Domain r6 started

4. vcpucount
# virsh vcpucount r6 --guest
error: argument unsupported: QEMU guest agent is not configured

No crash happened, so change the status to VERIFIED.

Comment 13 errata-xmlrpc 2013-11-21 09:07:04 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-1581.html