Bug 1326652

Summary: libvirtd SIGSEGV when snapshot-create parallelly
Product: Red Hat Enterprise Linux 6 Reporter: Han Han <hhan>
Component: libvirtAssignee: Peter Krempa <pkrempa>
Status: CLOSED ERRATA QA Contact: Han Han <hhan>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 6.8CC: dyuan, jdenemar, rbalakri, xuzhang, yanyang
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: libvirt-0.10.2-61.el6 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-03-21 10:38:54 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On:    
Bug Blocks: 1359965    
Attachments:
Description Flags
gdb backtrace none

Description Han Han 2016-04-13 09:50:10 UTC
Created attachment 1146779 [details]
gdb backtrace

Description of problem:
As subject

Version-Release number of selected component (if applicable):
libvirt-0.10.2-60.el6.x86_64
qemu-kvm-rhev-0.12.1.2-2.491.el6.x86_64

How reproducible:
100%

Steps to Reproduce:
1. prepart a running guest without snapshot:
#  virsh list
 Id    Name                           State
----------------------------------------------------
 199   repeat                         running

# virsh snapshot-list repeat            
 Name                 Creation Time             State
------------------------------------------------------------

2.Run the following script on two terminials, which do snapshot operations parallelly:
#!/bin/bash -x
DOM=repeat
VM_IP=192.168.122.233
for i in {1..1000};do
    SNAP=`virsh snapshot-create $DOM|grep -o '[0-9]*'`
    virsh snapshot-dumpxml $DOM $SNAP > /tmp/$SNAP.xml
    virsh snapshot-create $DOM --redefine /tmp/$SNAP.xml --current
    ssh root@$VM_IP lsblk
    if [ $? -ne 0 ];then
        ssh root@$VM_IP lsblk
        ping $VM_IP -c 4
        wall BUGS
        echo $i BUGS
        break
    fi
    virsh snapshot-delete $DOM $SNAP
done

When libvirtd SIGSEGV, term1 running at :
+ for i in '{1..1000}'
++ virsh snapshot-create repeat
++ grep -o '[0-9]*'
error: End of file while reading data: Input/output error
error: One or more references were leaked after disconnect from the hypervisor
error: Failed to reconnect to the hypervisor
term2 running at:
+ for i in '{1..1000}'
++ virsh snapshot-create repeat
++ grep -o '[0-9]*'
error: End of file while reading data: Input/output error
error: One or more references were leaked after disconnect from the hypervisor
error: Failed to reconnect to the hypervisor

Actual results:
As step2

Expected results:
Snapshot operation should add lock to avoid parallel operations.

Additional info:
backtrace info is in attachment.

Comment 5 Han Han 2016-11-15 07:02:55 UTC
Verify it on libvirt-0.10.2-62.el6.x86_64:
On terminal one:
# virsh snapshot-create b2

On terminal two:
# virsh snapshot-create b2&;sleep 1; virsh snapshot-create b2&; sleep 1; virsh snapshot-create b2&;                                                                                        
The result:
Terminal one:
Domain snapshot 1479193131 created

Terminal two:                     
[1] 15880
[2] 15883
[3] 15888
error: Timed out during operation: cannot acquire state change lock

[1]    15880 exit 1     virsh snapshot-create b2
➜error: Timed out during operation: cannot acquire state change lock

[2]  - 15883 exit 1     virsh snapshot-create b2
error: Timed out during operation: cannot acquire state change lock

[3]  + 15888 exit 1     virsh snapshot-create b2

No SIGSEGV, bug fixed.

Comment 7 errata-xmlrpc 2017-03-21 10:38:54 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2017-0682.html