Bug 1510781

Summary: libvirtd crashes in qemuDomainHasBlockjob after qemu-attach
Product: Red Hat Enterprise Linux 7 Reporter: Dan Zheng <dzheng>
Component: libvirtAssignee: Peter Krempa <pkrempa>
Status: CLOSED ERRATA QA Contact: Fangge Jin <fjin>
Severity: high Docs Contact:
Priority: high    
Version: 7.5CC: dyuan, mtessun, pkrempa, rbalakri, xuzhang, yalzhang
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: libvirt-3.9.0-2.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-04-10 10:59:09 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
libvirtd_crash_qemu_attach.log none

Description Dan Zheng 2017-11-08 08:46:37 UTC
Description of problem:

The command execution fails and the libvirtd service will crash after qemu-attach <qemu_PID>.

Version-Release number of selected component (if applicable):
qemu-kvm-rhev-2.10.0-4.el7.ppc64le
libvirt-3.9.0-1.el7.ppc64le
3.10.0-768.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1. 
# systemctl status libvirtd
● libvirtd.service - Virtualization daemon
   Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled; vendor preset: enabled)
   Active: active (running) since Wed 2017-11-08 16:43:10 CST; 7s ago
     Docs: man:libvirtd(8)
           https://libvirt.org
 Main PID: 13453 (libvirtd)
    Tasks: 19 (limit: 32768)
   CGroup: /system.slice/libvirtd.service
           ├─ 5219 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/libexec/libvirt_...
           ├─ 5220 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/libexec/libvirt_...
           └─13453 /usr/sbin/libvirtd

# 
# /usr/libexec/qemu-kvm -m 2048 -drive file=/var/lib/avocado/data/avocado-vt/images/jeos-25-64.qcow2,if=none,index=0 -monitor unix:/tmp/ss,server,nowait -name test -uuid 1fdf7c78-866a-4dcf-b017-5a9299682e1f &
[1] 13992
# VNC server running on ::1:5900


# ps -ef|grep qemu
root     13992  6435 45 16:43 pts/3    00:00:12 /usr/libexec/qemu-kvm -m 2048 -drive file=/var/lib/avocado/data/avocado-vt/images/jeos-25-64.qcow2,if=none,index=0 -monitor unix:/tmp/ss,server,nowait -name test -uuid 1fdf7c78-866a-4dcf-b017-5a9299682e1f


# virsh qemu-attach 13992
error: Disconnected from qemu:///system due to end of file
error: Failed to attach to pid 13992
error: Cannot recv data: Connection reset by peer

# systemctl status libvirtd
● libvirtd.service - Virtualization daemon
   Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled; vendor preset: enabled)
   Active: active (running) since Wed 2017-11-08 16:45:30 CST; 13s ago
     Docs: man:libvirtd(8)
           https://libvirt.org
 Main PID: 14938 (libvirtd)



Actual results:
libvirtd did crash.

Expected results:
libvirtd does not crash and qemu process can be attached.

Additional info:
See attachment for gdb log

Comment 2 Dan Zheng 2017-11-08 08:48:44 UTC
Created attachment 1349321 [details]
libvirtd_crash_qemu_attach.log

Comment 3 Peter Krempa 2017-11-08 08:52:41 UTC
Thread 14 (Thread 0x7f06fdc14700 (LWP 13457)):
#0  0x00007f06f14e63fd in qemuDomainHasBlockjob (vm=vm@entry=0x7f06d4003ce0, copy_only=copy_only@entry=false) at qemu/qemu_domain.c:6342
#1  0x00007f06f14e68d4 in qemuDomainObjPrivateXMLFormatBlockjobs (vm=0x7f06d4003ce0, buf=0x7f06fdc13820) at qemu/qemu_domain.c:1913
#2  qemuDomainObjPrivateXMLFormat (buf=0x7f06fdc13820, vm=0x7f06d4003ce0) at qemu/qemu_domain.c:2056
#3  0x00007f070ec50248 in virDomainObjFormat (xmlopt=0x7f06e82f81f0, obj=obj@entry=0x7f06d4003ce0, caps=0x7f06e821b6c0, 
    flags=flags@entry=313) at conf/domain_conf.c:26518
#4  0x00007f070ec5031c in virDomainSaveStatus (xmlopt=<optimized out>, statusDir=0x7f06e810c5f0 "/var/run/libvirt/qemu", 
    obj=obj@entry=0x7f06d4003ce0, caps=<optimized out>) at conf/domain_conf.c:26684
#5  0x00007f06f150de65 in qemuProcessAttach (conn=conn@entry=0x7f06d80009a0, driver=driver@entry=0x7f06e80f2bf0, vm=0x7f06d4003ce0,

Comment 4 Peter Krempa 2017-11-13 12:35:16 UTC
Fixed upstream:

commit 994c49ac34181cb664b18856df38c2f5c326ff03
Author: Peter Krempa <pkrempa>
Date:   Wed Nov 8 20:44:35 2017 +0100

    qemu: parse: Allocate disk definition with private data
    
    Use virDomainDiskDefNew instead of VIR_ALLOC in
    qemuParseCommandLineDisk.

Comment 7 Fangge Jin 2017-12-04 07:26:36 UTC
Reproduce this bug with libvirt-3.9.0-1.el7.x86_64

Verified with 3.9.0-4.virtcov.el7

Comment 11 errata-xmlrpc 2018-04-10 10:59:09 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:0704