Bug 1660115

Summary: Clone VM from snapshot ends with VolumeGroupSizeError
Product: [oVirt] vdsm Reporter: Elad <ebenahar>
Component: CoreAssignee: Eyal Shenitzky <eshenitz>
Status: CLOSED WORKSFORME QA Contact: Elad <ebenahar>
Severity: high Docs Contact:
Priority: unspecified    
Version: 4.30.3CC: aefrat, bugs, lleistne, tnisan
Target Milestone: ovirt-4.3.4Flags: rule-engine: ovirt-4.3+
Target Release: ---   
Hardware: x86_64   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-03-21 10:52:07 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Storage RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1661989    
Bug Blocks:    
Attachments:
Description Flags
logs
none
VM events none

Description Elad 2018-12-17 14:26:16 UTC
Created attachment 1515082 [details]
logs

Description of problem:
An attempt to clone a VM from a snapshot, for a VM that has a VM lease on the storage (iSCSI), fails on vdsm.


Version-Release number of selected component (if applicable):
ovirt-engine-4.3.0-0.0.master.20181116185756.gite19db6e.el7.noarch
vdsm-4.30.2-11.git6c62ff1.el7.x86_64
lvm2-2.02.180-10.el7_6.1.x86_64

How reproducible:
Tested once

Steps to Reproduce:
1. Create a VM with disk and lease, both on iSCSI
2. Create a snapshot for the VM
3. Clone a new VM from the snapshot


Actual results:


2018-12-17 15:48:40,632+0200 INFO  (tasks/1) [storage.Volume] Request to extend LV 173ace7f-cf63-4009-a5c9-e6a32920e5ff of image e1ca2e5b-5abb-499d-8060-832c7641ddf8 in VG fb15be96-6ff4-4a7b-8c2d-786ac1685784 wi
th size = 39852160 (blockVolume:676)
2018-12-17 15:48:40,632+0200 INFO  (tasks/1) [storage.LVM] Extending LV fb15be96-6ff4-4a7b-8c2d-786ac1685784/173ace7f-cf63-4009-a5c9-e6a32920e5ff to 19460 megabytes (lvm:1213)
2018-12-17 15:48:40,633+0200 DEBUG (tasks/1) [storage.Misc.excCmd] /usr/bin/taskset --cpu-list 0-0 /usr/bin/sudo -n /usr/sbin/lvm lvextend --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ '\''a|/dev/mapper/20024f40058540000|/dev/mapper/20024f400585400a1|/dev/mapper/20024f400585400a2|/dev/mapper/20024f400585400a3|/dev/mapper/20024f400585400a4|/dev/mapper/20024f400585400a5|/dev/mapper/20024f400585400a6|/dev/mapper/360002ac0000000000000002b00021f6b|/dev/mapper/360002ac0000000000000002c00021f6b|/dev/mapper/360002ac0000000000000002d00021f6b|/dev/mapper/360002ac0000000000000002e00021f6b|/dev/mapper/360002ac0000000000000002f00021f6b|/dev/mapper/360002ac0000000000000003000021f6b|'\'', '\''r|.*|'\'' ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1  use_lvmetad=0 }  backup {  retain_min = 50  retain_days = 0 } ' --autobackup n --size 19460m fb15be96-6ff4-4a7b-8c2d-786ac1685784/173ace7f-cf63-4009-a5c9-e6a32920e5ff (cwd None) (commands:198)



2018-12-17 15:48:41,527+0200 DEBUG (tasks/1) [storage.Misc.excCmd] SUCCESS: <err> = ''; <rc> = 0 (commands:219)
2018-12-17 15:48:41,529+0200 ERROR (tasks/1) [storage.Image] Unexpected error (image:801)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/vdsm/storage/image.py", line 796, in copyCollapsed
    dstVol.extend(dstVolAllocBlk)
  File "/usr/lib/python2.7/site-packages/vdsm/storage/blockVolume.py", line 680, in extend
    lvm.extendLV(self.sdUUID, self.volUUID, sizemb)
  File "/usr/lib/python2.7/site-packages/vdsm/storage/lvm.py", line 1240, in extendLV
    % (vgName, lvName, free_extents, needed_extents))
VolumeGroupSizeError: Volume Group not big enough: (u'Not enough free extents for extending LV fb15be96-6ff4-4a7b-8c2d-786ac1685784/173ace7f-cf63-4009-a5c9-e6a32920e5ff (free=143, needed=152)',)





Engine:

2018-12-17 15:48:33,042+02 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-237) [vms_create_95a91c6b-4866-42b7] START, CopyImageVDSCommand( CopyImageVDSCommandParameters:{stor
agePoolId='33130e3a-ab53-42dd-a135-41a7ef325379', ignoreFailoverLimit='false', storageDomainId='fb15be96-6ff4-4a7b-8c2d-786ac1685784', imageGroupId='10f008a3-7793-4b3b-9058-fda1af473f98', imageId='ea2c1f93-ce4f-
4cc7-9284-246a08161d2f', dstImageGroupId='e1ca2e5b-5abb-499d-8060-832c7641ddf8', vmId='1b92a0f6-3bcd-4361-87a5-f921acd48262', dstImageId='173ace7f-cf63-4009-a5c9-e6a32920e5ff', imageDescription='', dstStorageDom
ainId='fb15be96-6ff4-4a7b-8c2d-786ac1685784', copyVolumeType='LeafVol', volumeFormat='COW', preallocate='Preallocated', postZero='false', discard='false', force='false'}), log id: 74c8c858




2018-12-17 15:48:53,441+02 ERROR [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (EE-ManagedThreadFactory-engine-Thread-634620) [] [within thread]: endAction for action type AddVmFromSnapshot threw an excepti
on.: java.lang.ClassCastException: org.ovirt.engine.core.common.action.VmLeaseParameters cannot be cast to org.ovirt.engine.core.common.action.MoveOrCopyImageGroupParameters
        at org.ovirt.engine.core.bll.AddVmAndCloneImageCommand.removeVmImages(AddVmAndCloneImageCommand.java:123) [bll.jar:]
        at org.ovirt.engine.core.bll.AddVmAndCloneImageCommand.removeVmRelatedEntitiesFromDb(AddVmAndCloneImageCommand.java:115) [bll.jar:]
        at org.ovirt.engine.core.bll.AddVmCommand.endWithFailure(AddVmCommand.java:1416) [bll.jar:]
        at org.ovirt.engine.core.bll.AddVmFromSnapshotCommand.endWithFailure(AddVmFromSnapshotCommand.java:304) [bll.jar:]
        at org.ovirt.engine.core.bll.CommandBase.internalEndWithFailure(CommandBase.java:723) [bll.jar:]
        at org.ovirt.engine.core.bll.CommandBase.endActionInTransactionScope(CommandBase.java:645) [bll.jar:]
        at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1950) [bll.jar:]



Expected results:
Should succeed

Additional info:

Comment 1 Elad 2018-12-17 14:44:41 UTC
Created attachment 1515084 [details]
VM events

There were few operations done for the VM before VM creation from its snapshot (snapshot restore, VM disk resize, etc.).
Attached the event log of this VM.

Comment 2 Tal Nisan 2018-12-24 09:25:51 UTC
Most likely the VM lease is unrelated here, Eyal please check

Comment 3 Eyal Shenitzky 2018-12-25 07:29:08 UTC
I cannot reproduce this flow due to bug 1661989.

But according to the log it seems like the LV extend failed because the VG was full - 

VolumeGroupSizeError: Volume Group not big enough: 
(u'Not enough free extents for extending LV fb15be96-6ff4-4a7b-8c2d-786ac1685784/173ace7f-cf63-4009-a5c9-e6a32920e5ff (free=143, needed=152)',)

This is not related to the lease.

Comment 4 Sandro Bonazzola 2019-01-28 09:36:48 UTC
This bug has not been marked as blocker for oVirt 4.3.0.
Since we are releasing it tomorrow, January 29th, this bug has been re-targeted to 4.3.1.

Comment 5 Tal Nisan 2019-03-12 10:34:43 UTC
Elad, we've made substantial changes to the lease creation flow, can you please verify that this still reproduces? My guess is that it's not

Comment 6 Avihai 2019-03-14 10:51:04 UTC
Elad, can you please check it out ?

Comment 7 Elad 2019-03-21 10:52:07 UTC
Couldn't reproduce, closing for now