Bug 705435

Summary: Unable to destroy VM when NFS share holding iso mounted as virtual disk disappears
Product: [Fedora] Fedora Reporter: Tim Flink <tflink>
Component: libvirtAssignee: Eric Blake <eblake>
Status: CLOSED WONTFIX QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 15CC: aquini, berrange, clalance, crobinso, eblake, itamar, jforbes, laine, veillard, virt-maint
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-06-07 00:45:55 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Tim Flink 2011-05-17 16:46:43 UTC
I have a qemu/KVM virtual machine with a .iso file (on a remote NFS share) mounted as a virtual CDROM drive. After the VM was booted, the machine hosting the NFS share was turned off. When I tried to shut down the VM, I got errors from both virsh and VMM. If I turn the NFS share back on, I am able to turn off the VM.

error from virsh:
[root@testlap ~]# virsh destroy f14upgrade
error: Failed to destroy domain f14upgrade
error: Timed out during operation: cannot acquire state change lock

error from VMM:
Dialog box with:
Error shutting down domain: Timed out during operation: cannot acquire state change lock

dialog box details:
Traceback (most recent call last):
  File "/usr/share/virt-manager/virtManager/engine.py", line 911, in _do_destroy_domain
    vm.destroy()
  File "/usr/share/virt-manager/virtManager/domain.py", line 1168, in destroy
    self._backend.destroy()
  File "/usr/lib64/python2.7/site-packages/libvirt.py", line 346, in destroy
    if ret == -1: raise libvirtError ('virDomainDestroy() failed', dom=self)
libvirtError: Timed out during operation: cannot acquire state change lock


from /var/log/messages:
libvirtd: 10:42:01.000: 1321: error : qemuDomainObjBeginJobWithDriver:453 : Timed out during operation: cannot acquire state change lock


Version-Release number of selected component (if applicable):
libvirt-0.8.8-4.fc15
qemu-2:0.14.0-7.fc15

How reproducible:
Every time.

Steps to Reproduce:
1. start VM with .iso over NFS as virtual CDROM
2. turn off NFS share without unmounting anything
3. attempt to shutdown VM
  
Actual results:
Unable to shutdown the virtual machine through virsh or VMM 'force off'.

Expected results:
Shutdown the virtual machine

Additional info:
HW info for VM host:
http://smolts.org/show?uuid=pub_75513d90-c6e6-4765-9c59-ec9c09387bd4

Comment 1 Fedora Admin XMLRPC Client 2011-09-22 17:52:15 UTC
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

Comment 2 Fedora Admin XMLRPC Client 2011-09-22 17:55:28 UTC
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

Comment 3 Fedora Admin XMLRPC Client 2011-11-30 20:01:41 UTC
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

Comment 4 Fedora Admin XMLRPC Client 2011-11-30 20:01:59 UTC
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

Comment 5 Fedora Admin XMLRPC Client 2011-11-30 20:06:38 UTC
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

Comment 6 Fedora Admin XMLRPC Client 2011-11-30 20:06:52 UTC
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

Comment 7 Eric Blake 2011-12-06 13:39:13 UTC
This sounds very similar to bug 746666.  At least your case gave a sensible error message instead of deadlocking.  I'm not sure if libvirt can ever fully recover from a missing NFS server, but I am trying to work on making a hung NFS have less impact on the rest of libvirt.

Comment 8 Cole Robinson 2012-06-07 00:45:55 UTC
Given the limited scope of this problem, and the potential invasiveness of the changes, particularly some of those pointed to in 746666, this will probably never be a fedora backport candidate.

Tim, if you are still seeing issues on F17, please file bugs against libvirt's upstream tracker (Product: Virtualization Tools in this bugzilla).