Bug 970932 - engine: disk space issues in LSM - LSM steps are sent to vdsm although there is not enough disk space to complete them
engine: disk space issues in LSM - LSM steps are sent to vdsm although there ...
Status: CLOSED CURRENTRELEASE
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine (Show other bugs)
3.2.0
x86_64 Linux
medium Severity high
: ---
: 3.4.0
Assigned To: Allon Mureinik
Elad
storage
:
Depends On:
Blocks: 960934 rhev3.4beta 1142926
  Show dependency treegraph
 
Reported: 2013-06-05 05:10 EDT by Dafna Ron
Modified: 2016-02-10 14:36 EST (History)
9 users (show)

See Also:
Fixed In Version: ovirt-3.4.0-alpha1
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: Storage
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
logs (1.94 MB, application/x-gzip)
2013-06-05 05:10 EDT, Dafna Ron
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
oVirt gerrit 22447 None None None Never

  None (edit)
Description Dafna Ron 2013-06-05 05:10:36 EDT
Created attachment 757083 [details]
logs

Description of problem:

we have two issues here.

1. when the target domain does not have enough disk space to create the initial volume we still sent the create volume to vdsm even after alerts that there is not enough disk space. 
2. we do not calculate the amount of disk space in the target domain which causes the domain to reach to <1G 

Version-Release number of selected component (if applicable):

sf17.4

How reproducible:

100%

Steps to Reproduce:
1. create several vms and LSM their disks from one domain to the other several times. 
2. 
3.

Actual results:

even though we do not have enough disk space in both the src and dst domains we start the LSM 

Expected results:

we should not start a LSM operation if we do not have enough disk space to move the disks. 

Additional info: logs


8691f03c-80a2-452d-9231-56e7f5acb81f::ERROR::2013-06-05 11:41:18,033::threadPool::209::Misc.ThreadPool.WorkerThread::(run) Task <function runTask at 0x7f5aab894230> failed
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/threadPool.py", line 201, in run
    cmd(args)
  File "/usr/share/vdsm/storage/storage_mailbox.py", line 75, in runTask
    ctask.prepare(cmd, *args)
  File "/usr/share/vdsm/storage/task.py", line 1159, in prepare
    raise self.error
RuntimeError: Volume extension failed for hda (domainID: 7414f930-bbdb-4ec6-8132-4640cbb3c722, volumeID: 2cdffe84-7cbf-41d8-a0ad-34721f9e5956)


18d57932-f978-49eb-ac32-0a5c8a1be413::ERROR::2013-06-05 11:48:17,640::storage_mailbox::153::Storage.SPM.Messages.Extend::(processRequest) processRequest: Exception caught while
 trying to extend volume: 269fa277-1db7-426f-891c-51b777311a79 in domain: 81ef11d0-4c0c-47b4-8953-d61a6af442d8
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/storage_mailbox.py", line 149, in processRequest
    pool.extendVolume(volume['domainID'], volume['volumeID'], size)
  File "/usr/share/vdsm/storage/securable.py", line 68, in wrapper
    return f(self, *args, **kwargs)
  File "/usr/share/vdsm/storage/sp.py", line 1324, in extendVolume
    sdCache.produce(sdUUID).extendVolume(volumeUUID, size, isShuttingDown)
  File "/usr/share/vdsm/storage/blockSD.py", line 1228, in extendVolume
    lvm.extendLV(self.sdUUID, volumeUUID, size) #, isShuttingDown) # FIXME
  File "/usr/share/vdsm/storage/lvm.py", line 1093, in extendLV
    free_size / constants.MEGAB))
VolumeGroupSizeError: Volume Group not big enough: ('81ef11d0-4c0c-47b4-8953-d61a6af442d8/269fa277-1db7-426f-891c-51b777311a79 3072 > 768 (MiB)',)
18d57932-f978-49eb-ac32-0a5c8a1be413::DEBUG::2013-06-05 11:48:17,641::storage_mailbox::109::Storage.SPM.Messages.Extend::(__init__) new extend msg created: domain: 81ef11d0-4c0
c-47b4-8953-d61a6af442d8, volume: 269fa277-1db7-426f-891c-51b777311a79
Comment 1 Allon Mureinik 2014-01-01 11:23:17 EST
This was solved by the patch at http://gerrit.ovirt.org/#/c/22447/, which was created as part of the fix for bug 960934.
Moving to MODIFIED.
Comment 2 Elad 2014-01-16 07:29:22 EST
Engine won't allow LSM when there is not enough free space in the destination domain:

Error on UI:

Error while executing action: Cannot move Virtual Machine Disk. Low disk space on target Storage Domain iscsi1-2.

Engine.log:

2014-01-16 14:26:32,935 WARN  [org.ovirt.engine.core.bll.lsm.LiveMigrateVmDisksCommand] (ajp--127.0.0.1-8702-5) [1efa621d] CanDoAction of action LiveMigrateVmDisks failed. Reasons:VAR__ACTION__MOVE,VAR__TYPE__VM_D
ISK,ACTION_TYPE_FAILED_DISK_SPACE_LOW_ON_TARGET_STORAGE_DOMAIN,$storageName iscsi1-2


Verified on ovirt-3.4.0-alpha1
ovirt-engine-3.4.0-0.2.master.20140106180914.el6.noarch
Comment 4 Itamar Heim 2014-06-12 10:08:06 EDT
Closing as part of 3.4.0

Note You need to log in before you can comment on or make changes to this bug.