Bug 872277 - 3.2 vdsm: cannot migrate VM after live storage migration
3.2 vdsm: cannot migrate VM after live storage migration
Status: CLOSED CURRENTRELEASE
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: vdsm (Show other bugs)
3.2.0
x86_64 Linux
high Severity high
: ---
: 3.2.0
Assigned To: Amador Pahim
Dafna Ron
virt
: Patch, Regression, ZStream
Depends On: 913253
Blocks: 885777 891351 915537
  Show dependency treegraph
 
Reported: 2012-11-01 13:29 EDT by Dafna Ron
Modified: 2015-09-22 09 EDT (History)
12 users (show)

See Also:
Fixed In Version: vdsm-4.10.2-3
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 891351 (view as bug list)
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
logs (988.66 KB, application/x-gzip)
2012-11-01 13:29 EDT, Dafna Ron
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 285373 None None None Never
oVirt gerrit 9139 None None None Never

  None (edit)
Description Dafna Ron 2012-11-01 13:29:48 EDT
Created attachment 636752 [details]
logs

Description of problem:

I could not migrate a vm after live storage migration with error: 

libvirtError: Requested operation is not valid: cannot migrate domain with 1 snapshots

Version-Release number of selected component (if applicable):

si23
vdsm-4.9.6-40.0.el6_3.x86_64

How reproducible:

100%

Steps to Reproduce:
1. create and run a vm (I had iscsi storage). 
2. move disk
3. after the move finishes migrate vm
  
Actual results:

we fail to migrate the vm:

libvirtError: Requested operation is not valid: cannot migrate domain with 1 snapshots

Expected results:

we should be able to migrate the vm 

Additional info: logs
Comment 1 Itamar Heim 2012-11-01 13:35:57 EDT
is the vm useable after shutdown/restart and can then be live migrated?
(i.e., is the issue transient and goes away on restart of the vm)
Comment 2 Dafna Ron 2012-11-01 13:46:56 EDT
after I shut down the vm and re-run it I can migrate to the second host.
Comment 3 Amador Pahim 2012-11-04 20:16:03 EST
Some additional information:


- Before live storage migration:

virsh # list --all
 Id    Name                           State
----------------------------------------------------
 3     vm01                           running

virsh # snapshot-list vm01 
 Name                 Creation Time             State         
------------------------------------------------------------


- Live storage migration successful. After live storage migration:

virsh # snapshot-list vm01
 Name                 Creation Time             State         
------------------------------------------------------------
 1352073732           2012-11-05 00:02:12 +0000 disk-snapshot


- Trying to migrate VM:

Thread-5793::ERROR::2012-11-04 23:04:44,000::vm::262::vm.Vm::(run) vmId=`36e3dbd3-4495-475a-901f-054feeb066a8`::Failed to migrate
Traceback (most recent call last):
  File "/usr/share/vdsm/vm.py", line 245, in run
  File "/usr/share/vdsm/libvirtvm.py", line 478, in _startUnderlyingMigration
  File "/usr/share/vdsm/libvirtvm.py", line 518, in f
  File "/usr/lib64/python2.6/site-packages/vdsm/libvirtconnection.py", line 83, in wrapper
  File "/usr/lib64/python2.6/site-packages/libvirt.py", line 1104, in migrateToURI2
libvirtError: Requested operation is not valid: cannot migrate domain with 1 snapshots


- After migration error, can't connect to VM console:

Thread-6039::ERROR::2012-11-04 23:11:00,000::BindingXMLRPC::903::vds::(wrapper) libvirt error
Traceback (most recent call last):
  File "/usr/share/vdsm/BindingXMLRPC.py", line 899, in wrapper
  File "/usr/share/vdsm/BindingXMLRPC.py", line 232, in vmSetTicket
  File "/usr/share/vdsm/API.py", line 536, in setTicket
  File "/usr/share/vdsm/libvirtvm.py", line 2196, in setTicket
  File "/usr/share/vdsm/libvirtvm.py", line 526, in f
TimeoutError: Timed out during operation: cannot acquire state change lock


- Because of migration failure, we are affected by:
https://bugzilla.redhat.com/show_bug.cgi?id=676205


- Trying to manually remove snapshot :

virsh # snapshot-delete vm01 --snapshotname 1352073732
error: Failed to delete snapshot 1352073732
error: unsupported configuration: deletion of 1 external disk snapshots not supported yet


- Shutting down and starting VM again, no snapshot is showed in virsh:

virsh # snapshot-list vm01 
 Name                 Creation Time             State         
------------------------------------------------------------


- And migration works well.
Comment 4 Amador Pahim 2012-11-09 13:22:09 EST
Proposed patch:
http://gerrit.ovirt.org/9139
Comment 9 Dafna Ron 2013-02-20 12:54:49 EST
blocked: https://bugzilla.redhat.com/show_bug.cgi?id=913253
Comment 12 Dafna Ron 2013-04-08 02:11:04 EDT
verified on sf13 vdsm-4.10.2-14.0.el6ev.x86_64
Comment 13 Itamar Heim 2013-06-11 05:47:00 EDT
3.2 has been released
Comment 14 Itamar Heim 2013-06-11 05:47:10 EDT
3.2 has been released
Comment 15 Itamar Heim 2013-06-11 05:56:38 EDT
3.2 has been released

Note You need to log in before you can comment on or make changes to this bug.