Bug 1284250

Summary:	When performing rollback and failing to delete the disk it'll remain LOCKED (Context is always passed on rollback)
Product:	[oVirt] ovirt-engine	Reporter:	Liron Aravot <laravot>
Component:	BLL.Storage	Assignee:	Liron Aravot <laravot>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Elad <ebenahar>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	3.6.0	CC:	amureini, bugs, ebenahar, ecohen, gklein, kgoldbla, laravot, lsurette, rbalakri, Rhev-m-bugs, tnisan, yeylon, ylavi
Target Milestone:	ovirt-3.6.1	Keywords:	Regression
Target Release:	3.6.1	Flags:	rule-engine: ovirt-3.6.z+ rule-engine: blocker+ ylavi: planning_ack+ tnisan: devel_ack+ acanan: testing_ack+
Hardware:	x86_64
OS:	Unspecified
Whiteboard:	storage
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:	1232396	Environment:
Last Closed:	2015-12-16 12:18:25 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	Storage	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1232396
Bug Blocks:	1282693, 1282694

Description Liron Aravot 2015-11-22 12:02:39 UTC

+++ This bug was initially created as a clone of Bug #1232396 +++

Version-Release number of selected component (if applicable):
v3.6
ovirt-engine-3.6.0-0.0.master.20150519172219.git9a2e2b3.el6.noarch
vdsm-4.17.0-822.git9b11a18.el7.noarch

How reproducible:
100%

Steps to Reproduce:
1. Have a DC with 2 data domains and export domain.
1. Create a VM with 2 disks
2. Export the vm
3. Import the vm, use different domain for each disk
4. while the import is ongoing, block the access to the non master domain.
5. the import will fail, the engine will attempt to revert the operation
6. the engine will attempt to revert the operation by deleting the disks

Actual results:
the disk on the blocked domain remains in LOCKED status

Expected results:
the disk on the blocked domain should be marked as ILLEGAL

Comment 1 Red Hat Bugzilla Rules Engine 2015-11-22 12:30:25 UTC

This bug is not marked for z-stream, yet the milestone is for a z-stream version, therefore the milestone has been reset.
Please set the correct milestone or add the z-stream flag.

Comment 2 Red Hat Bugzilla Rules Engine 2015-11-27 04:38:08 UTC

Bug tickets that are moved to testing must have target release set to make sure tester knows what to test. Please set the correct target release before moving to ON_QA.

Comment 3 Sandro Bonazzola 2015-12-01 15:07:14 UTC

This bug is referenced in git log for ovirt-engine-3.6.1.1.
Please set target release to 3.6.1.1 accordingly unless additional patches are needed.

Comment 4 Elad 2015-12-08 09:29:07 UTC

Following the steps in the description:
Blocked connectivity to the non master domain while import of a VM with 2 disk, one to the master and one to the non-master domain, is ongoing:

CopyImage for 2 disks, to different domains:

2015-12-08 09:13:45,237 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (org.ovirt.thread.pool-7-thread-39) [10082b6] START, CopyImageVDSCommand( CopyImageVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000158', ignoreFailoverLimit='false', storageDomainId='e89bff1b-b661-419a-85cc-1b083219879b', imageGroupId='2990d342-85fc-4245-a7ec-9d8d43978edc', imageId='07fd5d8f-272a-4aeb-a3cb-3a1b6a28f274', dstImageGroupId='c6945e78-ee4f-41ca-add5-62700c92c3db', vmId='7ca79620-703e-46a7-a97f-a16c1b922c67', dstImageId='d15e563c-e912-4153-8dbc-48749f05cb1c', imageDescription='', dstStorageDomainId='6605112f-7f26-48b9-b64e-f661bfe18799', copyVolumeType='LeafVol', volumeFormat='RAW', preallocate='Preallocated', postZero='false', force='true'}), log id: 363d371b


2015-12-08 09:13:46,709 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (org.ovirt.thread.pool-7-thread-39) [654825fa] START, CopyImageVDSCommand( CopyImageVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000158', ignoreFailoverLimit='false', storageDomainId='e89bff1b-b661-419a-85cc-1b083219879b', imageGroupId='cc4e7750-dc78-476b-b652-f41ee7eec01b', imageId='fbcc45a9-b387-413b-af77-7c93a27cf65f', dstImageGroupId='cdfb3998-268b-436c-a443-ecb24cd521d4', vmId='7ca79620-703e-46a7-a97f-a16c1b922c67', dstImageId='ef6aaa0b-ecd2-46ff-9f3e-7a95c7b2d895', imageDescription='', dstStorageDomainId='504c7595-7ca5-459c-be6f-44a3d7f1b5d7', copyVolumeType='LeafVol', volumeFormat='RAW', preallocate='Preallocated', postZero='false', force='true'}), log id: 5a3bebf7




DeleteImage and CopyImage failed due to connectivity failure with the non-master domain:


2015-12-08 09:14:44,652 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-7-thread-45) [54b19f70] START, DeleteImageGroupVDSCommand( Del
eteImageGroupVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000158', ignoreFailoverLimit='false', storageDomainId='504c7595-7ca5-459c-be6f-44a3d7f1b5
d7', imageGroupId='cdfb3998-268b-436c-a443-ecb24cd521d4', postZeros='false', forceDelete='false'}), log id: 4b199bb8
2015-12-08 09:14:44,878 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-7-thread-45) [54b19f70] Failed in 'DeleteImageGroupVDS' method
2015-12-08 09:14:44,958 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-45) [54b19f70] Correlation ID: null, Call Stack: null
, Custom Event ID: -1, Message: VDSM command failed: Storage domain does not exist: (u'504c7595-7ca5-459c-be6f-44a3d7f1b5d7',)
2015-12-08 09:14:44,958 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (org.ovirt.thread.pool-7-thread-45) [54b19f70] IrsBroker::Failed::DeleteImageGroupVDS: IRSGener
icException: IRSErrorException: Failed to DeleteImageGroupVDS, error = Storage domain does not exist: (u'504c7595-7ca5-459c-be6f-44a3d7f1b5d7',), code = 358
2015-12-08 09:14:44,961 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-7-thread-45) [54b19f70] FINISH, DeleteImageGroupVDSCommand, lo
g id: 4b199bb8
2015-12-08 09:14:44,962 ERROR [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-7-thread-45) [54b19f70] Command 'org.ovirt.engine.core.bll.RemoveImageCommand' failed:
 EngineException: org.ovirt.engine.core.vdsbroker.irsbroker.IRSErrorException: IRSGenericException: IRSErrorException: Failed to DeleteImageGroupVDS, error = Storage domain does not exis
t: (u'504c7595-7ca5-459c-be6f-44a3d7f1b5d7',), code = 358 (Failed with error StorageDomainDoesNotExist and code 358)
2015-12-08 09:14:44,994 INFO  [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-7-thread-45) [54b19f70] Command [id=5f481258-6a96-4522-9be0-c31a9f510621]: Compensatin
g CHANGED_STATUS_ONLY of org.ovirt.engine.core.common.businessentities.storage.Image; snapshot: EntityStatusSnapshot:{id='ef6aaa0b-ecd2-46ff-9f3e-7a95c7b2d895', status='ILLEGAL'}.
2015-12-08 09:14:45,036 INFO  [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-7-thread-45) [54b19f70] Removed task 'aecd1b74-0035-4765-89f3-4e6969af9934' from D
ataBase
2015-12-08 09:14:45,281 ERROR [org.ovirt.engine.core.bll.CopyImageGroupCommand] (org.ovirt.thread.pool-7-thread-45) [10082b6] Ending command 'org.ovirt.engine.core.bll.CopyImageGroupComm
and' with failure.



RemoveImage fails:
2015-12-08 09:14:55,725 ERROR [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-7-thread-5) [37b12ea4] Ending command 'org.ovirt.engine.core.bll.RemoveImageCommand' with failure.


Disk state changes to 'Illegal':


             image_group_id            | imagestatus 
--------------------------------------+-------------
 cdfb3998-268b-436c-a443-ecb24cd521d4 |           4



Verified using:
rhevm-3.6.1.1-0.1.el6.noarch
vdsm-4.17.12-0.el7ev.noarch

Comment 5 Sandro Bonazzola 2015-12-16 12:18:25 UTC

According to verification status and target milestone this issue should be fixed in oVirt 3.6.1. Closing current release.