Bug 1665000

Summary: libvirtd encountered sigsegv due to null pointer in virJSONValueObjectHasKey()
Product: Red Hat Enterprise Linux 8 Reporter: Ján Tomko <jtomko>
Component: libvirtAssignee: Ján Tomko <jtomko>
Status: CLOSED CURRENTRELEASE QA Contact: Luyao Huang <lhuang>
Severity: high Docs Contact:
Priority: unspecified    
Version: 8.0CC: blc, jdenemar, jiyan, lhuang, rbalakri, xuzhang
Target Milestone: rcFlags: rule-engine: mirror+
Target Release: 8.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: libvirt-4.5.0-18.el8 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1663051 Environment:
Last Closed: 2019-06-14 01:20:12 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1663051    
Bug Blocks:    

Description Ján Tomko 2019-01-10 09:08:01 UTC
+++ This bug was initially created as a clone of Bug #1663051 +++

Description of problem:

libvirtd encountered a segmentation fault and core-dumped. This was on a RHEL 7.5 host in a RHV environment.

The backtrace in question was;

(gdb) bt
#0  virJSONValueObjectHasKey (object=object@entry=0x0, key=key@entry=0x7ff4e1ae6826 "error") at util/virjson.c:751
#1  0x00007ff4e19ee19d in qemuAgentCheckError (cmd=cmd@entry=0x7ff4cc06bbe0, reply=0x0) at qemu/qemu_agent.c:1098
#2  0x00007ff4e19f2040 in qemuAgentGetInterfaces (mon=mon@entry=0x7ff4a8001090, ifaces=ifaces@entry=0x7ff4ee95cad8) at qemu/qemu_agent.c:2070
#3  0x00007ff4e1aac785 in qemuDomainInterfaceAddresses (dom=0x7ff4cc04a030, ifaces=0x7ff4ee95cad8, source=<optimized out>, flags=<optimized out>) at qemu/qemu_driver.c:20403
#4  0x00007ff4ff66e12b in virDomainInterfaceAddresses (dom=dom@entry=0x7ff4cc04a030, ifaces=ifaces@entry=0x7ff4ee95cad8, source=1, flags=0) at libvirt-domain.c:11780
#5  0x00005565c557f415 in remoteDispatchDomainInterfaceAddresses (server=0x5565c73921c0, msg=0x5565c73ba6a0, ret=0x7ff4cc0016a0, args=0x7ff4cc007d40, rerr=0x7ff4ee95cc10, client=0x5565c73ba1c0) at remote.c:6837
#6  remoteDispatchDomainInterfaceAddressesHelper (server=0x5565c73921c0, client=0x5565c73ba1c0, msg=0x5565c73ba6a0, rerr=0x7ff4ee95cc10, args=0x7ff4cc007d40, ret=0x7ff4cc0016a0) at remote_dispatch.h:6578
#7  0x00007ff4ff6cf782 in virNetServerProgramDispatchCall (msg=0x5565c73ba6a0, client=0x5565c73ba1c0, server=0x5565c73921c0, prog=0x5565c73a9dc0) at rpc/virnetserverprogram.c:437
#8  virNetServerProgramDispatch (prog=0x5565c73a9dc0, server=server@entry=0x5565c73921c0, client=0x5565c73ba1c0, msg=0x5565c73ba6a0) at rpc/virnetserverprogram.c:307
#9  0x00005565c55b3c7d in virNetServerProcessMsg (msg=<optimized out>, prog=<optimized out>, client=<optimized out>, srv=0x5565c73921c0) at rpc/virnetserver.c:148
#10 virNetServerHandleJob (jobOpaque=<optimized out>, opaque=0x5565c73921c0) at rpc/virnetserver.c:169
#11 0x00007ff4ff5a9811 in virThreadPoolWorker (opaque=opaque@entry=0x5565c739dae0) at util/virthreadpool.c:167
#12 0x00007ff4ff5a8b98 in virThreadHelper (data=<optimized out>) at util/virthread.c:206
#13 0x00007ff4fc9aedd5 in start_thread (arg=0x7ff4ee95d700) at pthread_create.c:308
#14 0x00007ff4fc6d8b3d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113



Version-Release number of selected component (if applicable):

RHV 4.2
RHEL 7.5 host;
  libvirt-client-3.9.0-14.el7_5.6

--- Additional comment from Luyao Huang on 2019-01-07 04:02:02 CET ---

Here is my reproducing steps:

1. In guest (already install qemu-guest-agent):

# cat script.gdb 
set width 0
set height 0
set verbose off

b qmp_guest_network_get_interfaces
commands 1
  kill
end

b qmp_guest_shutdown
commands 2
  return
  c
end

c

# gdb -batch --command=script.gdb qemu-ga `pidof qemu-ga`

2. On host:

# virsh shutdown vm1 --mode agent; virsh domifaddr vm1 --source agent
error: Failed to shutdown domain vm1
error: Guest agent is not responding: Guest agent not available for now

error: Disconnected from qemu:///system due to end of file
error: Failed to query for interfaces addresses
error: End of file while reading data: Input/output error

--- Additional comment from Ján Tomko on 2019-01-10 10:03:56 CET ---

Patch sent upstream:
https://www.redhat.com/archives/libvir-list/2019-January/msg00241.html

Comment 3 Ján Tomko 2019-01-10 11:11:46 UTC
Pushed upstream as:
commit 7cfd1fbb1332ae5df678b9f41a62156cb2e88c73
Author:     Ján Tomko <jtomko>
CommitDate: 2019-01-10 12:09:48 +0100

    qemu: require reply from guest agent in qemuAgentGetInterfaces
    
git describe: v4.10.0-164-g7cfd1fbb13

Comment 7 Luyao Huang 2019-01-15 04:38:57 UTC
Verify this bug with libvirt-4.5.0-18.module+el8+2691+dc742e5d.x86_64:

1. prepare a running guest with guest agent

2. Login guest and run script:

# cat script.gdb 
set width 0
set height 0
set verbose off

b qmp_guest_network_get_interfaces
commands 1
  kill
end

b qmp_guest_shutdown
commands 2
  return
  c
end

c

# gdb -batch --command=script.gdb qemu-ga `pidof qemu-ga`

3. Run shutdown and domifaddr command on host:

# virsh shutdown vm1 --mode agent; virsh domifaddr vm1 --source agent
error: Failed to shutdown domain vm1
error: Guest agent is not responding: Guest agent not available for now

error: Failed to query for interfaces addresses
error: Guest agent is not responding: Guest agent disappeared while executing command