Bug 1151841
| Summary: | Failed to delete snapshot disk | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Virtualization Manager | Reporter: | Kevin Alon Goldblatt <kgoldbla> | ||||
| Component: | ovirt-engine | Assignee: | Daniel Erez <derez> | ||||
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Aharon Canan <acanan> | ||||
| Severity: | urgent | Docs Contact: | |||||
| Priority: | unspecified | ||||||
| Version: | 3.5.0 | CC: | acanan, amureini, derez, ecohen, gklein, iheim, kgoldbla, lpeer, lsurette, rbalakri, Rhev-m-bugs, scohen, tnisan, yeylon | ||||
| Target Milestone: | --- | ||||||
| Target Release: | 3.5.0 | ||||||
| Hardware: | x86_64 | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | storage | ||||||
| Fixed In Version: | org.ovirt.engine-root-3.5.0-18 | Doc Type: | Bug Fix | ||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | Type: | Bug | |||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | Storage | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Attachments: |
|
||||||
Hi Kevin, * In order to isolate the underline issue, can you please check if it is reproduced when deleting snapshots from different disks only (as opposed to deleting multiple snapshots of the same disk). * Can you please try to verify in block domain and xml-rpc (in order to understand whether it's related to domain type or json-rpc). * Seems the vdsm log been rotated out, can you attach the previous log where the error is mentioned. @Kevin - Another thing, is the snapshot disk being removed successfully on second try? The issue seems to be isolated to a deletion of multiple disk snapshots from the same disk and only when done in consecutive unsorted manner (not from parent to leaf). A workaround would be to simply trying to delete the failed disk snapshots again, which ends successfully. Merging inconsecutive set of disk snapshots still fails:
ae15057a-54b2-4392-bf59-971882dfcace::ERROR::2014-10-22 16:19:35,184::task::866::Storage.TaskManager.Task::(_setError) Task=`ae15057a-54b2-4392-bf59-971882dfcace`::Unexpected error
Traceback (most recent call last):
File "/usr/share/vdsm/storage/task.py", line 873, in _run
return fn(*args, **kargs)
File "/usr/share/vdsm/storage/task.py", line 334, in run
return self.cmd(*self.argslist, **self.argsdict)
File "/usr/share/vdsm/storage/securable.py", line 77, in wrapper
return method(self, *args, **kwargs)
File "/usr/share/vdsm/storage/sp.py", line 1756, in mergeSnapshots
sdUUID, vmUUID, imgUUID, ancestor, successor, postZero)
File "/usr/share/vdsm/storage/image.py", line 1156, in merge
dstVol = vols[ancestor]
KeyError: u'5a97fd91-b15e-4bf2-bb2b-3748af22bb81'
Used vt7
Moving back to ASSIGNED
The issue resolved by [1] is removing non-consecutive snapshots disks (that belong to the same disk) in any order. E.g. 'disk3 from snapshot1' and 'disk1 from snapshot1'. [1] http://gerrit.ovirt.org/34103 verified using vt8 all 3 disks removed. RHEV-M 3.5.0 has been released, closing this bug. RHEV-M 3.5.0 has been released, closing this bug. |
Created attachment 946097 [details] screenshots, server and vdsm logs Checked with: rhevm-3.5.0-0.13.beta.el6ev.noarch vdsm-4.16.5-2.el6ev.x86_64 1.Created a VM with 4 disks (2 preallocated and 2 thin) 2.Created snapshot s1. 3.Added 2 additional disks (1 preallocated and 1 thin) 4.Created snapshot s2. 5.From Storage domain (block storage) select 3 snapshot disks for deletion (2 6.from snapshot s1 and 1 from snapshot s2) 7.The 2 snapshot disks from snapshot s1 are successfully deleted. 8.The 1 snapshot disk from snapshot s2 failed to delete Actual results: 2 out of 3 disks were deleted. 1 disk faile to delete Expected results: All snapshot disks should have been deleted Additional info: From the engine log: ----------------------------------------- 2014-10-12 14:40:15,327 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-7-thread-39) [22ed4eee] Start running CanDoAction for command number 1/2 (Command type: RemoveDiskSnapshots) 2014-10-12 14:40:15,338 INFO [org.ovirt.engine.core.bll.RemoveDiskSnapshotsCommand] (org.ovirt.thread.pool-7-thread-39) [22ed4eee] Lock Acquired to object EngineLock [exclusiveLocks= key: b61ec151-5788-4e98-ba5d-6a92ca382bc9 value: DISK 2014-10-12 14:40:15,382 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-7-thread-39) [22ed4eee] Finish handling CanDoAction for command number 1/2 (Command type: RemoveDiskSnapshots) 2014-10-12 14:40:15,409 INFO [org.ovirt.engine.core.bll.RemoveDiskSnapshotsCommand] (org.ovirt.thread.pool-7-thread-30) [22ed4eee] Running command: RemoveDiskSnapshotsCommand Task handler: RemoveDiskSnapshotTaskHandler internal: false. Entities affected : ID: 2f374279-8637-425e-aa5f-03faa71e5c18 Type: VMAction group MANIPULATE_VM_SNAPSHOTS with role type USER 2014-10-12 14:40:15,578 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-30) [1cd16e2e] Correlation ID: 22ed4eee, Job ID: 45917d32-d872-47f2-8658-2088ac2c3053, Call Stack: null, Custom Event ID: -1, Message: Disk 'ddd_Disk1' from Snapshot(s) 's2, s1' of VM 'ddd' deletion was initiated by admin. 2014-10-12 14:43:06,310 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-10) Correlation ID: 22ed4eee, Call Stack: null, Custom Event ID: -1, Message: Failed to complete deletion of Disk 'ddd_Disk1' from snapshot(s) 's1' of VM 'ddd' (User: admin).