Bug 1284250 - When performing rollback and failing to delete the disk it'll remain LOCKED (Context is always passed on rollback)
Summary: When performing rollback and failing to delete the disk it'll remain LOCKED (...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: ovirt-engine
Classification: oVirt
Component: BLL.Storage
Version: 3.6.0
Hardware: x86_64
OS: Unspecified
unspecified
high
Target Milestone: ovirt-3.6.1
: 3.6.1
Assignee: Liron Aravot
QA Contact: Elad
URL:
Whiteboard: storage
Depends On: 1232396
Blocks: 1282693 1282694
TreeView+ depends on / blocked
 
Reported: 2015-11-22 12:02 UTC by Liron Aravot
Modified: 2016-05-08 08:20 UTC (History)
13 users (show)

Fixed In Version:
Clone Of: 1232396
Environment:
Last Closed: 2015-12-16 12:18:25 UTC
oVirt Team: Storage
Embargoed:
rule-engine: ovirt-3.6.z+
rule-engine: blocker+
ylavi: planning_ack+
tnisan: devel_ack+
acanan: testing_ack+

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 48779 0 master MERGED core: do not pass the context automatically on rollback Never
oVirt gerrit 48914 0 master MERGED core: RemoveImage - fix parameters ctor Never
oVirt gerrit 48930 0 ovirt-engine-3.6 MERGED core: RemoveImage - fix parameters ctor Never
oVirt gerrit 48931 0 ovirt-engine-3.6 MERGED core: do not pass the context automatically on rollback Never
oVirt gerrit 49093 0 ovirt-engine-3.6.1 MERGED core: RemoveImage - fix parameters ctor Never
oVirt gerrit 49094 0 ovirt-engine-3.6.1 MERGED core: do not pass the context automatically on rollback Never

Description Liron Aravot 2015-11-22 12:02:39 UTC 
+++ This bug was initially created as a clone of Bug #1232396 +++

Version-Release number of selected component (if applicable):
v3.6
ovirt-engine-3.6.0-0.0.master.20150519172219.git9a2e2b3.el6.noarch
vdsm-4.17.0-822.git9b11a18.el7.noarch

How reproducible:
100%

Steps to Reproduce:
1. Have a DC with 2 data domains and export domain.
1. Create a VM with 2 disks
2. Export the vm
3. Import the vm, use different domain for each disk
4. while the import is ongoing, block the access to the non master domain.
5. the import will fail, the engine will attempt to revert the operation
6. the engine will attempt to revert the operation by deleting the disks

Actual results:
the disk on the blocked domain remains in LOCKED status

Expected results:
the disk on the blocked domain should be marked as ILLEGAL
Comment 1 Red Hat Bugzilla Rules Engine 2015-11-22 12:30:25 UTC 
This bug is not marked for z-stream, yet the milestone is for a z-stream version, therefore the milestone has been reset.
Please set the correct milestone or add the z-stream flag.
Comment 2 Red Hat Bugzilla Rules Engine 2015-11-27 04:38:08 UTC 
Bug tickets that are moved to testing must have target release set to make sure tester knows what to test. Please set the correct target release before moving to ON_QA.
Comment 3 Sandro Bonazzola 2015-12-01 15:07:14 UTC 
This bug is referenced in git log for ovirt-engine-3.6.1.1.
Please set target release to 3.6.1.1 accordingly unless additional patches are needed.
Comment 4 Elad 2015-12-08 09:29:07 UTC 
Following the steps in the description:
Blocked connectivity to the non master domain while import of a VM with 2 disk, one to the master and one to the non-master domain, is ongoing:

CopyImage for 2 disks, to different domains:

2015-12-08 09:13:45,237 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (org.ovirt.thread.pool-7-thread-39) [10082b6] START, CopyImageVDSCommand( CopyImageVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000158', ignoreFailoverLimit='false', storageDomainId='e89bff1b-b661-419a-85cc-1b083219879b', imageGroupId='2990d342-85fc-4245-a7ec-9d8d43978edc', imageId='07fd5d8f-272a-4aeb-a3cb-3a1b6a28f274', dstImageGroupId='c6945e78-ee4f-41ca-add5-62700c92c3db', vmId='7ca79620-703e-46a7-a97f-a16c1b922c67', dstImageId='d15e563c-e912-4153-8dbc-48749f05cb1c', imageDescription='', dstStorageDomainId='6605112f-7f26-48b9-b64e-f661bfe18799', copyVolumeType='LeafVol', volumeFormat='RAW', preallocate='Preallocated', postZero='false', force='true'}), log id: 363d371b


2015-12-08 09:13:46,709 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (org.ovirt.thread.pool-7-thread-39) [654825fa] START, CopyImageVDSCommand( CopyImageVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000158', ignoreFailoverLimit='false', storageDomainId='e89bff1b-b661-419a-85cc-1b083219879b', imageGroupId='cc4e7750-dc78-476b-b652-f41ee7eec01b', imageId='fbcc45a9-b387-413b-af77-7c93a27cf65f', dstImageGroupId='cdfb3998-268b-436c-a443-ecb24cd521d4', vmId='7ca79620-703e-46a7-a97f-a16c1b922c67', dstImageId='ef6aaa0b-ecd2-46ff-9f3e-7a95c7b2d895', imageDescription='', dstStorageDomainId='504c7595-7ca5-459c-be6f-44a3d7f1b5d7', copyVolumeType='LeafVol', volumeFormat='RAW', preallocate='Preallocated', postZero='false', force='true'}), log id: 5a3bebf7




DeleteImage and CopyImage failed due to connectivity failure with the non-master domain:


2015-12-08 09:14:44,652 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-7-thread-45) [54b19f70] START, DeleteImageGroupVDSCommand( Del
eteImageGroupVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000158', ignoreFailoverLimit='false', storageDomainId='504c7595-7ca5-459c-be6f-44a3d7f1b5
d7', imageGroupId='cdfb3998-268b-436c-a443-ecb24cd521d4', postZeros='false', forceDelete='false'}), log id: 4b199bb8
2015-12-08 09:14:44,878 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-7-thread-45) [54b19f70] Failed in 'DeleteImageGroupVDS' method
2015-12-08 09:14:44,958 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-45) [54b19f70] Correlation ID: null, Call Stack: null
, Custom Event ID: -1, Message: VDSM command failed: Storage domain does not exist: (u'504c7595-7ca5-459c-be6f-44a3d7f1b5d7',)
2015-12-08 09:14:44,958 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (org.ovirt.thread.pool-7-thread-45) [54b19f70] IrsBroker::Failed::DeleteImageGroupVDS: IRSGener
icException: IRSErrorException: Failed to DeleteImageGroupVDS, error = Storage domain does not exist: (u'504c7595-7ca5-459c-be6f-44a3d7f1b5d7',), code = 358
2015-12-08 09:14:44,961 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (org.ovirt.thread.pool-7-thread-45) [54b19f70] FINISH, DeleteImageGroupVDSCommand, lo
g id: 4b199bb8
2015-12-08 09:14:44,962 ERROR [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-7-thread-45) [54b19f70] Command 'org.ovirt.engine.core.bll.RemoveImageCommand' failed:
 EngineException: org.ovirt.engine.core.vdsbroker.irsbroker.IRSErrorException: IRSGenericException: IRSErrorException: Failed to DeleteImageGroupVDS, error = Storage domain does not exis
t: (u'504c7595-7ca5-459c-be6f-44a3d7f1b5d7',), code = 358 (Failed with error StorageDomainDoesNotExist and code 358)
2015-12-08 09:14:44,994 INFO  [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-7-thread-45) [54b19f70] Command [id=5f481258-6a96-4522-9be0-c31a9f510621]: Compensatin
g CHANGED_STATUS_ONLY of org.ovirt.engine.core.common.businessentities.storage.Image; snapshot: EntityStatusSnapshot:{id='ef6aaa0b-ecd2-46ff-9f3e-7a95c7b2d895', status='ILLEGAL'}.
2015-12-08 09:14:45,036 INFO  [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-7-thread-45) [54b19f70] Removed task 'aecd1b74-0035-4765-89f3-4e6969af9934' from D
ataBase
2015-12-08 09:14:45,281 ERROR [org.ovirt.engine.core.bll.CopyImageGroupCommand] (org.ovirt.thread.pool-7-thread-45) [10082b6] Ending command 'org.ovirt.engine.core.bll.CopyImageGroupComm
and' with failure.



RemoveImage fails:
2015-12-08 09:14:55,725 ERROR [org.ovirt.engine.core.bll.RemoveImageCommand] (org.ovirt.thread.pool-7-thread-5) [37b12ea4] Ending command 'org.ovirt.engine.core.bll.RemoveImageCommand' with failure.


Disk state changes to 'Illegal':


             image_group_id            | imagestatus 
--------------------------------------+-------------
 cdfb3998-268b-436c-a443-ecb24cd521d4 |           4



Verified using:
rhevm-3.6.1.1-0.1.el6.noarch
vdsm-4.17.12-0.el7ev.noarch
Comment 5 Sandro Bonazzola 2015-12-16 12:18:25 UTC 
According to verification status and target milestone this issue should be fixed in oVirt 3.6.1. Closing current release.
Note You need to log in before you can comment on or make changes to this bug.