Bug 770919

Summary: Sometimes virsh command screenshot may hang
Product: Red Hat Enterprise Linux 6 Reporter: weizhang <weizhan>
Component: libvirtAssignee: Michal Privoznik <mprivozn>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: medium Docs Contact:
Priority: medium    
Version: 6.3CC: acathrow, dallan, dyuan, honzhang, mzhan, rwu, yupzhang
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: libvirt-0.9.10-1.el6 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-06-20 06:40:30 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description weizhang 2011-12-30 08:30:32 UTC
Description of problem:
Sometimes virsh command screenshot may hang

Version-Release number of selected component (if applicable):
kernel-2.6.32-223.el6.x86_64
libvirt-0.9.9-0rc1.el6.x86_64
qemu-kvm-0.12.1.2-2.213.el6.x86_64

How reproducible:
sometimes

Steps to Reproduce:
1. Prepare a running guest
2. # for i in {1..100}; do virsh screenshot guest; sleep 1; done
3.
  
Actual results:
It may hang after several loops

Expected results:
Command executed without hang

Additional info:

Comment 2 weizhang 2011-12-30 08:46:54 UTC
When I set log_level=1 on libvirtd.conf and then restart libvirtd, it will not hang any more on loop execution

Comment 3 Michal Privoznik 2012-01-10 16:31:32 UTC
Patch sent upstream:

https://www.redhat.com/archives/libvir-list/2012-January/msg00343.html

NB, this is not restricted to screenshot only, but any API using streams could have hit this issue. But yeah, all other stream using APIs do take longer time to finish, so screenshot is perfect for verification and testing.

Comment 6 Michal Privoznik 2012-01-12 11:11:09 UTC
Moving to POST:

commit 833b901cb711a9db148b4fe39e658411f82f3467
Author:     Michal Privoznik <mprivozn>
AuthorDate: Tue Jan 10 16:57:30 2012 +0100
Commit:     Michal Privoznik <mprivozn>
CommitDate: Thu Jan 12 12:02:40 2012 +0100

    stream: Check for stream EOF
    
    If client stream does not have any data to sink and neither received
    EOF, a dummy packet is sent to the daemon signalising client is ready to
    sink some data. However, after we added event loop to client a race may
    occur:
    
    Thread 1 calls virNetClientStreamRecvPacket and since no data are cached
    nor stream has EOF, it decides to send dummy packet to server which will
    sent some data in turn. However, during this decision and actual message
    exchange with server -
    
    Thread 2 receives last stream data from server. Therefore an EOF is set
    on stream and if there is a call waiting (which is not yet) it is woken
    up. However, Thread 1 haven't sent anything so far, so there is no call
    to be woken up. So this thread sent dummy packet to daemon, which
    ignores that as no stream is associated with such packet and therefore
    no reply will ever come.
    
    This race causes client to hang indefinitely.


v0.9.9-30-g833b901

Comment 9 hongming 2012-02-15 08:14:30 UTC
Reproduce the bug using libvirt-0.9.9-1.el6.x86_64



Verify it using libvirt-0.9.10-1.el6.x86_64

Steps
1. Prepare a running guest
2. # for i in {1..100}; do virsh screenshot guest; sleep 1; done

Results
Command executed without hang

Comment 11 errata-xmlrpc 2012-06-20 06:40:30 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2012-0748.html