Bug 1030736

Summary: libvirtd crash when create/destroy VM repeatedly
Product: Red Hat Enterprise Linux 6 Reporter: yanbing du <ydu>
Component: libvirtAssignee: Michal Privoznik <mprivozn>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 6.5CC: dyuan, eblake, mzhan, rbalakri, tlavigne, zhwang, zsong
Target Milestone: rcKeywords: ZStream
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: libvirt-0.10.2-30.el6 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-10-14 04:18:21 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1038816    

Comment 5 Michal Privoznik 2013-11-19 16:53:37 UTC
Just for the record, this bug isn't reproducible on upstream nor RHEL-7.0. So I've run git bisect to find the commit which fixed the crash and found it was this one:

commit a9e97e0c3057dfaefc7217b7db7fbba001cf8f0b
Author:     Daniel P. Berrange <berrange>
AuthorDate: Wed Feb 6 18:17:20 2013 +0000
Commit:     Daniel P. Berrange <berrange>
CommitDate: Wed Feb 13 11:10:30 2013 +0000

    Remove qemuDriverLock from almost everywhere
    
    With the majority of fields in the virQEMUDriverPtr struct
    now immutable or self-locking, there is no need for practically
    any methods to be using the QEMU driver lock. Only a handful
    of helper APIs in qemu_conf.c now need it

v1.0.2-153-ga9e97e0

which has huge diff stat:

10 files changed, 388 insertions(+), 984 deletions(-)

so it is not desired to backport it into z-stream (among with all the preparation and follow up patches). So I'll work on a RHEL-only patch that will do for 6.5.z.

Comment 6 yanbing du 2013-11-20 06:27:38 UTC
(In reply to Michal Privoznik from comment #5)
> Just for the record, this bug isn't reproducible on upstream nor RHEL-7.0.

Actually, there's a different behavor on RHEL7, see bug 1022924 comment 30.

> So I've run git bisect to find the commit which fixed the crash and found it
> was this one:
> 
> commit a9e97e0c3057dfaefc7217b7db7fbba001cf8f0b
> Author:     Daniel P. Berrange <berrange>
> AuthorDate: Wed Feb 6 18:17:20 2013 +0000
> Commit:     Daniel P. Berrange <berrange>
> CommitDate: Wed Feb 13 11:10:30 2013 +0000
> 
>     Remove qemuDriverLock from almost everywhere
>     
>     With the majority of fields in the virQEMUDriverPtr struct
>     now immutable or self-locking, there is no need for practically
>     any methods to be using the QEMU driver lock. Only a handful
>     of helper APIs in qemu_conf.c now need it
> 
> v1.0.2-153-ga9e97e0
> 
> which has huge diff stat:
> 
> 10 files changed, 388 insertions(+), 984 deletions(-)
> 
> so it is not desired to backport it into z-stream (among with all the
> preparation and follow up patches). So I'll work on a RHEL-only patch that
> will do for 6.5.z.

Comment 7 Michal Privoznik 2013-11-20 13:49:38 UTC
I believe I got a patch that fixes the problem:

http://post-office.corp.redhat.com/archives/rhvirt-patches/2013-November/msg00958.html

A scratch build to test:

http://brewweb.devel.redhat.com/brew/taskinfo?taskID=6617467

Comment 8 Michal Privoznik 2013-11-20 16:44:18 UTC
I've just realized my fix wasn't quite good. So round two of the patch among with a scratch build to test:

http://post-office.corp.redhat.com/archives/rhvirt-patches/2013-November/msg00964.html


http://brewweb.devel.redhat.com/brew/taskinfo?taskID=6618787

Comment 11 yanbing du 2014-04-14 07:48:55 UTC
1. Reproduce this bug with libvirt-0.10.2-29.el6.x86_64
Steps:
Just run the following command in 3 terminals:
#while /bin/true ; do virsh create vm.xml ; virsh destroy rhel6 ; done

After a while(about 3 mins), libvirtd crashed.

2. Verify bug with libvirt-0.10.2-32.el6.x86_64. Re-run the command and libvirtd still running after 5 mins.

Comment 13 errata-xmlrpc 2014-10-14 04:18:21 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2014-1374.html