Created attachment 1515082 [details] logs Description of problem: An attempt to clone a VM from a snapshot, for a VM that has a VM lease on the storage (iSCSI), fails on vdsm. Version-Release number of selected component (if applicable): ovirt-engine-4.3.0-0.0.master.20181116185756.gite19db6e.el7.noarch vdsm-4.30.2-11.git6c62ff1.el7.x86_64 lvm2-2.02.180-10.el7_6.1.x86_64 How reproducible: Tested once Steps to Reproduce: 1. Create a VM with disk and lease, both on iSCSI 2. Create a snapshot for the VM 3. Clone a new VM from the snapshot Actual results: 2018-12-17 15:48:40,632+0200 INFO (tasks/1) [storage.Volume] Request to extend LV 173ace7f-cf63-4009-a5c9-e6a32920e5ff of image e1ca2e5b-5abb-499d-8060-832c7641ddf8 in VG fb15be96-6ff4-4a7b-8c2d-786ac1685784 wi th size = 39852160 (blockVolume:676) 2018-12-17 15:48:40,632+0200 INFO (tasks/1) [storage.LVM] Extending LV fb15be96-6ff4-4a7b-8c2d-786ac1685784/173ace7f-cf63-4009-a5c9-e6a32920e5ff to 19460 megabytes (lvm:1213) 2018-12-17 15:48:40,633+0200 DEBUG (tasks/1) [storage.Misc.excCmd] /usr/bin/taskset --cpu-list 0-0 /usr/bin/sudo -n /usr/sbin/lvm lvextend --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ '\''a|/dev/mapper/20024f40058540000|/dev/mapper/20024f400585400a1|/dev/mapper/20024f400585400a2|/dev/mapper/20024f400585400a3|/dev/mapper/20024f400585400a4|/dev/mapper/20024f400585400a5|/dev/mapper/20024f400585400a6|/dev/mapper/360002ac0000000000000002b00021f6b|/dev/mapper/360002ac0000000000000002c00021f6b|/dev/mapper/360002ac0000000000000002d00021f6b|/dev/mapper/360002ac0000000000000002e00021f6b|/dev/mapper/360002ac0000000000000002f00021f6b|/dev/mapper/360002ac0000000000000003000021f6b|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --autobackup n --size 19460m fb15be96-6ff4-4a7b-8c2d-786ac1685784/173ace7f-cf63-4009-a5c9-e6a32920e5ff (cwd None) (commands:198) 2018-12-17 15:48:41,527+0200 DEBUG (tasks/1) [storage.Misc.excCmd] SUCCESS: <err> = ''; <rc> = 0 (commands:219) 2018-12-17 15:48:41,529+0200 ERROR (tasks/1) [storage.Image] Unexpected error (image:801) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/image.py", line 796, in copyCollapsed dstVol.extend(dstVolAllocBlk) File "/usr/lib/python2.7/site-packages/vdsm/storage/blockVolume.py", line 680, in extend lvm.extendLV(self.sdUUID, self.volUUID, sizemb) File "/usr/lib/python2.7/site-packages/vdsm/storage/lvm.py", line 1240, in extendLV % (vgName, lvName, free_extents, needed_extents)) VolumeGroupSizeError: Volume Group not big enough: (u'Not enough free extents for extending LV fb15be96-6ff4-4a7b-8c2d-786ac1685784/173ace7f-cf63-4009-a5c9-e6a32920e5ff (free=143, needed=152)',) Engine: 2018-12-17 15:48:33,042+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-237) [vms_create_95a91c6b-4866-42b7] START, CopyImageVDSCommand( CopyImageVDSCommandParameters:{stor agePoolId='33130e3a-ab53-42dd-a135-41a7ef325379', ignoreFailoverLimit='false', storageDomainId='fb15be96-6ff4-4a7b-8c2d-786ac1685784', imageGroupId='10f008a3-7793-4b3b-9058-fda1af473f98', imageId='ea2c1f93-ce4f- 4cc7-9284-246a08161d2f', dstImageGroupId='e1ca2e5b-5abb-499d-8060-832c7641ddf8', vmId='1b92a0f6-3bcd-4361-87a5-f921acd48262', dstImageId='173ace7f-cf63-4009-a5c9-e6a32920e5ff', imageDescription='', dstStorageDom ainId='fb15be96-6ff4-4a7b-8c2d-786ac1685784', copyVolumeType='LeafVol', volumeFormat='COW', preallocate='Preallocated', postZero='false', discard='false', force='false'}), log id: 74c8c858 2018-12-17 15:48:53,441+02 ERROR [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (EE-ManagedThreadFactory-engine-Thread-634620) [] [within thread]: endAction for action type AddVmFromSnapshot threw an excepti on.: java.lang.ClassCastException: org.ovirt.engine.core.common.action.VmLeaseParameters cannot be cast to org.ovirt.engine.core.common.action.MoveOrCopyImageGroupParameters at org.ovirt.engine.core.bll.AddVmAndCloneImageCommand.removeVmImages(AddVmAndCloneImageCommand.java:123) [bll.jar:] at org.ovirt.engine.core.bll.AddVmAndCloneImageCommand.removeVmRelatedEntitiesFromDb(AddVmAndCloneImageCommand.java:115) [bll.jar:] at org.ovirt.engine.core.bll.AddVmCommand.endWithFailure(AddVmCommand.java:1416) [bll.jar:] at org.ovirt.engine.core.bll.AddVmFromSnapshotCommand.endWithFailure(AddVmFromSnapshotCommand.java:304) [bll.jar:] at org.ovirt.engine.core.bll.CommandBase.internalEndWithFailure(CommandBase.java:723) [bll.jar:] at org.ovirt.engine.core.bll.CommandBase.endActionInTransactionScope(CommandBase.java:645) [bll.jar:] at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1950) [bll.jar:] Expected results: Should succeed Additional info:
Created attachment 1515084 [details] VM events There were few operations done for the VM before VM creation from its snapshot (snapshot restore, VM disk resize, etc.). Attached the event log of this VM.
Most likely the VM lease is unrelated here, Eyal please check
I cannot reproduce this flow due to bug 1661989. But according to the log it seems like the LV extend failed because the VG was full - VolumeGroupSizeError: Volume Group not big enough: (u'Not enough free extents for extending LV fb15be96-6ff4-4a7b-8c2d-786ac1685784/173ace7f-cf63-4009-a5c9-e6a32920e5ff (free=143, needed=152)',) This is not related to the lease.
This bug has not been marked as blocker for oVirt 4.3.0. Since we are releasing it tomorrow, January 29th, this bug has been re-targeted to 4.3.1.
Elad, we've made substantial changes to the lease creation flow, can you please verify that this still reproduces? My guess is that it's not
Elad, can you please check it out ?
Couldn't reproduce, closing for now