Bug 1147971 - Snapshot locked after successful live storage migration
Summary: Snapshot locked after successful live storage migration
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: oVirt
Classification: Retired
Component: ovirt-engine-core
Version: 3.5
Hardware: Unspecified
OS: Unspecified
high
urgent
Target Milestone: ---
: 3.5.0
Assignee: Ravi Nori
QA Contact: Jiri Belka
URL:
Whiteboard: infra
: 1148320 (view as bug list)
Depends On: 1149460 1150428
Blocks: 1073943
TreeView+ depends on / blocked
 
Reported: 2014-09-30 12:35 UTC by Nir Soffer
Modified: 2016-02-10 19:33 UTC (History)
12 users (show)

Fixed In Version: ovirt-3.5.0_rc4
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2014-10-17 12:30:01 UTC
oVirt Team: Infra
Embargoed:


Attachments (Terms of Use)
engine log (213.82 KB, text/plain)
2014-09-30 12:35 UTC, Nir Soffer
no flags Details
vdsm log (193.61 KB, application/x-xz)
2014-09-30 12:36 UTC, Nir Soffer
no flags Details


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 33552 0 master MERGED core : Fix NPE in CommandBase buildCommandEntity Never
oVirt gerrit 33602 0 master MERGED core: LiveSnapshotTaskHandler - remove setExecutionContext(null) Never
oVirt gerrit 33615 0 ovirt-engine-3.5 MERGED core : Fix NPE in CommandBase buildCommandEntity Never
oVirt gerrit 33617 0 ovirt-engine-3.5 MERGED core: LiveSnapshotTaskHandler - remove setExecutionContext(null) Never

Description Nir Soffer 2014-09-30 12:35:29 UTC
Created attachment 942690 [details]
engine log

Description of problem:

After live storage migration completes successfuly, the 
"Auto-generated for Live Storage Migration" snapshot is left 
in "Locked" state.

This prevents various actions with the vm or the disk.

Version-Release number of selected component (if applicable):
oVirt Engine Version: 3.5.0-0.0.master.20140911091402.gite1c5ffd.fc20

How reproducible:
Always

Steps to Reproduce:
1. Setup a dc with 2 iscsi storage domains
2. Create and start a vim with one disk on one of the domains
3. Move disk to the other domain

Actual results:
Migration succeeds, but the auto-generated snapshot is left in "Locked" state.

Expected results:
Migration succeeds, but the auto-generated snapshot is left in "OK" state.

Additional info:

While the migration is running, there are two tasks:

# vdsClient -s 0 getAllTasksInfo
0dc68bba-1b9b-4f5f-8811-ba9db222b633 :
	verb = syncImageData
	id = 0dc68bba-1b9b-4f5f-8811-ba9db222b633
186121fa-8604-462b-a8d3-c09adf830775 :
	verb = createVolume
	id = 186121fa-8604-462b-a8d3-c09adf830775

When the migration finish, we have only one task:

# vdsClient -s 0 getAllTasksInfo
186121fa-8604-462b-a8d3-c09adf830775 :
	verb = createVolume
	id = 186121fa-8604-462b-a8d3-c09adf830775

This task is finished, but not cleaned up by engine:

# vdsClient -s 0 getAllTasks
186121fa-8604-462b-a8d3-c09adf830775 :
	verb = createVolume
	code = 0
	state = finished
	tag = spm
	result = {'uuid': 'f6d11460-fcd1-4b40-8e50-56a2ad6347a5'}
	message = 1 jobs completed successfully
	id = 186121fa-8604-462b-a8d3-c09adf830775


Workaround:

Unlock the snapshot using the unlock_entity.sh tool:

$ sudo /usr/share/ovirt-engine/dbscripts/unlock_entity.sh -t snapshot 98e6058d-cf96-4929-b118-4fc5542a30ba d3496bdc-76c4-4090-934e-e297fcba2cd8

Comment 1 Nir Soffer 2014-09-30 12:36:31 UTC
Created attachment 942691 [details]
vdsm log

Comment 2 Nir Soffer 2014-09-30 12:45:17 UTC
Forgot to mention that using unlock_entity tool, we can see that snapshot is locked after migration finished:

$ sudo /usr/share/ovirt-engine/dbscripts/unlock_entity.sh -t snapshot -q
98e6058d-cf96-4929-b118-4fc5542a30ba | d3496bdc-76c4-4090-934e-e297fcba2cd8

Comment 3 Allon Mureinik 2014-09-30 12:46:36 UTC
Daniel, can you take a look please?

Comment 4 Daniel Erez 2014-09-30 13:02:18 UTC
Yair/Ravi,

It looks like the first task invoked by SEAT mechanism isn't getting cleaned (while the consequence tasks are being cleaned properly).
(it seems that a similar issue has been found on disk snapshots removal: bug 1134866).

Could it be related to recent CoCo infrastructure changes?

Comment 5 Allon Mureinik 2014-09-30 19:41:16 UTC
The source of this issue is a behavior change in CommandBase (as seen in the attached patch by Ravi).
Moving to infra to track this to completion. Of course, if there is a storage specific issue here, please return the whiteboard to storage.

Comment 6 Daniel Erez 2014-10-02 08:20:41 UTC
*** Bug 1148320 has been marked as a duplicate of this bug. ***

Comment 7 Jiri Belka 2014-10-08 09:48:57 UTC
iscsi disk migration doesn't work for me.

Comment 8 Sandro Bonazzola 2014-10-17 12:30:01 UTC
oVirt 3.5 has been released and should include the fix for this issue.


Note You need to log in before you can comment on or make changes to this bug.