Bug 970932 - engine: disk space issues in LSM - LSM steps are sent to vdsm although there is not enough disk space to complete them
Summary: engine: disk space issues in LSM - LSM steps are sent to vdsm although there ...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine
Version: 3.2.0
Hardware: x86_64
OS: Linux
medium
high
Target Milestone: ---
: 3.4.0
Assignee: Allon Mureinik
QA Contact: Elad
URL:
Whiteboard: storage
Depends On:
Blocks: 960934 rhev3.4beta 1142926
TreeView+ depends on / blocked
 
Reported: 2013-06-05 09:10 UTC by Dafna Ron
Modified: 2016-02-10 19:36 UTC (History)
9 users (show)

Fixed In Version: ovirt-3.4.0-alpha1
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed:
oVirt Team: Storage
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
logs (1.94 MB, application/x-gzip)
2013-06-05 09:10 UTC, Dafna Ron
no flags Details


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 22447 0 None None None Never

Description Dafna Ron 2013-06-05 09:10:36 UTC
Created attachment 757083 [details]
logs

Description of problem:

we have two issues here.

1. when the target domain does not have enough disk space to create the initial volume we still sent the create volume to vdsm even after alerts that there is not enough disk space. 
2. we do not calculate the amount of disk space in the target domain which causes the domain to reach to <1G 

Version-Release number of selected component (if applicable):

sf17.4

How reproducible:

100%

Steps to Reproduce:
1. create several vms and LSM their disks from one domain to the other several times. 
2. 
3.

Actual results:

even though we do not have enough disk space in both the src and dst domains we start the LSM 

Expected results:

we should not start a LSM operation if we do not have enough disk space to move the disks. 

Additional info: logs


8691f03c-80a2-452d-9231-56e7f5acb81f::ERROR::2013-06-05 11:41:18,033::threadPool::209::Misc.ThreadPool.WorkerThread::(run) Task <function runTask at 0x7f5aab894230> failed
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/threadPool.py", line 201, in run
    cmd(args)
  File "/usr/share/vdsm/storage/storage_mailbox.py", line 75, in runTask
    ctask.prepare(cmd, *args)
  File "/usr/share/vdsm/storage/task.py", line 1159, in prepare
    raise self.error
RuntimeError: Volume extension failed for hda (domainID: 7414f930-bbdb-4ec6-8132-4640cbb3c722, volumeID: 2cdffe84-7cbf-41d8-a0ad-34721f9e5956)


18d57932-f978-49eb-ac32-0a5c8a1be413::ERROR::2013-06-05 11:48:17,640::storage_mailbox::153::Storage.SPM.Messages.Extend::(processRequest) processRequest: Exception caught while
 trying to extend volume: 269fa277-1db7-426f-891c-51b777311a79 in domain: 81ef11d0-4c0c-47b4-8953-d61a6af442d8
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/storage_mailbox.py", line 149, in processRequest
    pool.extendVolume(volume['domainID'], volume['volumeID'], size)
  File "/usr/share/vdsm/storage/securable.py", line 68, in wrapper
    return f(self, *args, **kwargs)
  File "/usr/share/vdsm/storage/sp.py", line 1324, in extendVolume
    sdCache.produce(sdUUID).extendVolume(volumeUUID, size, isShuttingDown)
  File "/usr/share/vdsm/storage/blockSD.py", line 1228, in extendVolume
    lvm.extendLV(self.sdUUID, volumeUUID, size) #, isShuttingDown) # FIXME
  File "/usr/share/vdsm/storage/lvm.py", line 1093, in extendLV
    free_size / constants.MEGAB))
VolumeGroupSizeError: Volume Group not big enough: ('81ef11d0-4c0c-47b4-8953-d61a6af442d8/269fa277-1db7-426f-891c-51b777311a79 3072 > 768 (MiB)',)
18d57932-f978-49eb-ac32-0a5c8a1be413::DEBUG::2013-06-05 11:48:17,641::storage_mailbox::109::Storage.SPM.Messages.Extend::(__init__) new extend msg created: domain: 81ef11d0-4c0
c-47b4-8953-d61a6af442d8, volume: 269fa277-1db7-426f-891c-51b777311a79

Comment 1 Allon Mureinik 2014-01-01 16:23:17 UTC
This was solved by the patch at http://gerrit.ovirt.org/#/c/22447/, which was created as part of the fix for bug 960934.
Moving to MODIFIED.

Comment 2 Elad 2014-01-16 12:29:22 UTC
Engine won't allow LSM when there is not enough free space in the destination domain:

Error on UI:

Error while executing action: Cannot move Virtual Machine Disk. Low disk space on target Storage Domain iscsi1-2.

Engine.log:

2014-01-16 14:26:32,935 WARN  [org.ovirt.engine.core.bll.lsm.LiveMigrateVmDisksCommand] (ajp--127.0.0.1-8702-5) [1efa621d] CanDoAction of action LiveMigrateVmDisks failed. Reasons:VAR__ACTION__MOVE,VAR__TYPE__VM_D
ISK,ACTION_TYPE_FAILED_DISK_SPACE_LOW_ON_TARGET_STORAGE_DOMAIN,$storageName iscsi1-2


Verified on ovirt-3.4.0-alpha1
ovirt-engine-3.4.0-0.2.master.20140106180914.el6.noarch

Comment 4 Itamar Heim 2014-06-12 14:08:06 UTC
Closing as part of 3.4.0


Note You need to log in before you can comment on or make changes to this bug.