Bug 1660115 - Clone VM from snapshot ends with VolumeGroupSizeError
Summary: Clone VM from snapshot ends with VolumeGroupSizeError
Keywords:
Status: CLOSED WORKSFORME
Alias: None
Product: vdsm
Classification: oVirt
Component: Core
Version: 4.30.3
Hardware: x86_64
OS: Unspecified
unspecified
high vote
Target Milestone: ovirt-4.3.4
: ---
Assignee: Eyal Shenitzky
QA Contact: Elad
URL:
Whiteboard:
Depends On: 1661989
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-12-17 14:26 UTC by Elad
Modified: 2019-03-21 10:52 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-03-21 10:52:07 UTC
oVirt Team: Storage
rule-engine: ovirt-4.3+


Attachments (Terms of Use)
logs (1.15 MB, application/gzip)
2018-12-17 14:26 UTC, Elad
no flags Details
VM events (359.40 KB, image/png)
2018-12-17 14:44 UTC, Elad
no flags Details

Description Elad 2018-12-17 14:26:16 UTC
Created attachment 1515082 [details]
logs

Description of problem:
An attempt to clone a VM from a snapshot, for a VM that has a VM lease on the storage (iSCSI), fails on vdsm.


Version-Release number of selected component (if applicable):
ovirt-engine-4.3.0-0.0.master.20181116185756.gite19db6e.el7.noarch
vdsm-4.30.2-11.git6c62ff1.el7.x86_64
lvm2-2.02.180-10.el7_6.1.x86_64

How reproducible:
Tested once

Steps to Reproduce:
1. Create a VM with disk and lease, both on iSCSI
2. Create a snapshot for the VM
3. Clone a new VM from the snapshot


Actual results:


2018-12-17 15:48:40,632+0200 INFO  (tasks/1) [storage.Volume] Request to extend LV 173ace7f-cf63-4009-a5c9-e6a32920e5ff of image e1ca2e5b-5abb-499d-8060-832c7641ddf8 in VG fb15be96-6ff4-4a7b-8c2d-786ac1685784 wi
th size = 39852160 (blockVolume:676)
2018-12-17 15:48:40,632+0200 INFO  (tasks/1) [storage.LVM] Extending LV fb15be96-6ff4-4a7b-8c2d-786ac1685784/173ace7f-cf63-4009-a5c9-e6a32920e5ff to 19460 megabytes (lvm:1213)
2018-12-17 15:48:40,633+0200 DEBUG (tasks/1) [storage.Misc.excCmd] /usr/bin/taskset --cpu-list 0-0 /usr/bin/sudo -n /usr/sbin/lvm lvextend --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ '\''a|/dev/mapper/20024f40058540000|/dev/mapper/20024f400585400a1|/dev/mapper/20024f400585400a2|/dev/mapper/20024f400585400a3|/dev/mapper/20024f400585400a4|/dev/mapper/20024f400585400a5|/dev/mapper/20024f400585400a6|/dev/mapper/360002ac0000000000000002b00021f6b|/dev/mapper/360002ac0000000000000002c00021f6b|/dev/mapper/360002ac0000000000000002d00021f6b|/dev/mapper/360002ac0000000000000002e00021f6b|/dev/mapper/360002ac0000000000000002f00021f6b|/dev/mapper/360002ac0000000000000003000021f6b|'\'', '\''r|.*|'\'' ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1  use_lvmetad=0 }  backup {  retain_min = 50  retain_days = 0 } ' --autobackup n --size 19460m fb15be96-6ff4-4a7b-8c2d-786ac1685784/173ace7f-cf63-4009-a5c9-e6a32920e5ff (cwd None) (commands:198)



2018-12-17 15:48:41,527+0200 DEBUG (tasks/1) [storage.Misc.excCmd] SUCCESS: <err> = ''; <rc> = 0 (commands:219)
2018-12-17 15:48:41,529+0200 ERROR (tasks/1) [storage.Image] Unexpected error (image:801)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/vdsm/storage/image.py", line 796, in copyCollapsed
    dstVol.extend(dstVolAllocBlk)
  File "/usr/lib/python2.7/site-packages/vdsm/storage/blockVolume.py", line 680, in extend
    lvm.extendLV(self.sdUUID, self.volUUID, sizemb)
  File "/usr/lib/python2.7/site-packages/vdsm/storage/lvm.py", line 1240, in extendLV
    % (vgName, lvName, free_extents, needed_extents))
VolumeGroupSizeError: Volume Group not big enough: (u'Not enough free extents for extending LV fb15be96-6ff4-4a7b-8c2d-786ac1685784/173ace7f-cf63-4009-a5c9-e6a32920e5ff (free=143, needed=152)',)





Engine:

2018-12-17 15:48:33,042+02 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (default task-237) [vms_create_95a91c6b-4866-42b7] START, CopyImageVDSCommand( CopyImageVDSCommandParameters:{stor
agePoolId='33130e3a-ab53-42dd-a135-41a7ef325379', ignoreFailoverLimit='false', storageDomainId='fb15be96-6ff4-4a7b-8c2d-786ac1685784', imageGroupId='10f008a3-7793-4b3b-9058-fda1af473f98', imageId='ea2c1f93-ce4f-
4cc7-9284-246a08161d2f', dstImageGroupId='e1ca2e5b-5abb-499d-8060-832c7641ddf8', vmId='1b92a0f6-3bcd-4361-87a5-f921acd48262', dstImageId='173ace7f-cf63-4009-a5c9-e6a32920e5ff', imageDescription='', dstStorageDom
ainId='fb15be96-6ff4-4a7b-8c2d-786ac1685784', copyVolumeType='LeafVol', volumeFormat='COW', preallocate='Preallocated', postZero='false', discard='false', force='false'}), log id: 74c8c858




2018-12-17 15:48:53,441+02 ERROR [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (EE-ManagedThreadFactory-engine-Thread-634620) [] [within thread]: endAction for action type AddVmFromSnapshot threw an excepti
on.: java.lang.ClassCastException: org.ovirt.engine.core.common.action.VmLeaseParameters cannot be cast to org.ovirt.engine.core.common.action.MoveOrCopyImageGroupParameters
        at org.ovirt.engine.core.bll.AddVmAndCloneImageCommand.removeVmImages(AddVmAndCloneImageCommand.java:123) [bll.jar:]
        at org.ovirt.engine.core.bll.AddVmAndCloneImageCommand.removeVmRelatedEntitiesFromDb(AddVmAndCloneImageCommand.java:115) [bll.jar:]
        at org.ovirt.engine.core.bll.AddVmCommand.endWithFailure(AddVmCommand.java:1416) [bll.jar:]
        at org.ovirt.engine.core.bll.AddVmFromSnapshotCommand.endWithFailure(AddVmFromSnapshotCommand.java:304) [bll.jar:]
        at org.ovirt.engine.core.bll.CommandBase.internalEndWithFailure(CommandBase.java:723) [bll.jar:]
        at org.ovirt.engine.core.bll.CommandBase.endActionInTransactionScope(CommandBase.java:645) [bll.jar:]
        at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1950) [bll.jar:]



Expected results:
Should succeed

Additional info:

Comment 1 Elad 2018-12-17 14:44:41 UTC
Created attachment 1515084 [details]
VM events

There were few operations done for the VM before VM creation from its snapshot (snapshot restore, VM disk resize, etc.).
Attached the event log of this VM.

Comment 2 Tal Nisan 2018-12-24 09:25:51 UTC
Most likely the VM lease is unrelated here, Eyal please check

Comment 3 Eyal Shenitzky 2018-12-25 07:29:08 UTC
I cannot reproduce this flow due to bug 1661989.

But according to the log it seems like the LV extend failed because the VG was full - 

VolumeGroupSizeError: Volume Group not big enough: 
(u'Not enough free extents for extending LV fb15be96-6ff4-4a7b-8c2d-786ac1685784/173ace7f-cf63-4009-a5c9-e6a32920e5ff (free=143, needed=152)',)

This is not related to the lease.

Comment 4 Sandro Bonazzola 2019-01-28 09:36:48 UTC
This bug has not been marked as blocker for oVirt 4.3.0.
Since we are releasing it tomorrow, January 29th, this bug has been re-targeted to 4.3.1.

Comment 5 Tal Nisan 2019-03-12 10:34:43 UTC
Elad, we've made substantial changes to the lease creation flow, can you please verify that this still reproduces? My guess is that it's not

Comment 6 Avihai 2019-03-14 10:51:04 UTC
Elad, can you please check it out ?

Comment 7 Elad 2019-03-21 10:52:07 UTC
Couldn't reproduce, closing for now


Note You need to log in before you can comment on or make changes to this bug.