Created attachment 757083[details]
logs
Description of problem:
we have two issues here.
1. when the target domain does not have enough disk space to create the initial volume we still sent the create volume to vdsm even after alerts that there is not enough disk space.
2. we do not calculate the amount of disk space in the target domain which causes the domain to reach to <1G
Version-Release number of selected component (if applicable):
sf17.4
How reproducible:
100%
Steps to Reproduce:
1. create several vms and LSM their disks from one domain to the other several times.
2.
3.
Actual results:
even though we do not have enough disk space in both the src and dst domains we start the LSM
Expected results:
we should not start a LSM operation if we do not have enough disk space to move the disks.
Additional info: logs
8691f03c-80a2-452d-9231-56e7f5acb81f::ERROR::2013-06-05 11:41:18,033::threadPool::209::Misc.ThreadPool.WorkerThread::(run) Task <function runTask at 0x7f5aab894230> failed
Traceback (most recent call last):
File "/usr/share/vdsm/storage/threadPool.py", line 201, in run
cmd(args)
File "/usr/share/vdsm/storage/storage_mailbox.py", line 75, in runTask
ctask.prepare(cmd, *args)
File "/usr/share/vdsm/storage/task.py", line 1159, in prepare
raise self.error
RuntimeError: Volume extension failed for hda (domainID: 7414f930-bbdb-4ec6-8132-4640cbb3c722, volumeID: 2cdffe84-7cbf-41d8-a0ad-34721f9e5956)
18d57932-f978-49eb-ac32-0a5c8a1be413::ERROR::2013-06-05 11:48:17,640::storage_mailbox::153::Storage.SPM.Messages.Extend::(processRequest) processRequest: Exception caught while
trying to extend volume: 269fa277-1db7-426f-891c-51b777311a79 in domain: 81ef11d0-4c0c-47b4-8953-d61a6af442d8
Traceback (most recent call last):
File "/usr/share/vdsm/storage/storage_mailbox.py", line 149, in processRequest
pool.extendVolume(volume['domainID'], volume['volumeID'], size)
File "/usr/share/vdsm/storage/securable.py", line 68, in wrapper
return f(self, *args, **kwargs)
File "/usr/share/vdsm/storage/sp.py", line 1324, in extendVolume
sdCache.produce(sdUUID).extendVolume(volumeUUID, size, isShuttingDown)
File "/usr/share/vdsm/storage/blockSD.py", line 1228, in extendVolume
lvm.extendLV(self.sdUUID, volumeUUID, size) #, isShuttingDown) # FIXME
File "/usr/share/vdsm/storage/lvm.py", line 1093, in extendLV
free_size / constants.MEGAB))
VolumeGroupSizeError: Volume Group not big enough: ('81ef11d0-4c0c-47b4-8953-d61a6af442d8/269fa277-1db7-426f-891c-51b777311a79 3072 > 768 (MiB)',)
18d57932-f978-49eb-ac32-0a5c8a1be413::DEBUG::2013-06-05 11:48:17,641::storage_mailbox::109::Storage.SPM.Messages.Extend::(__init__) new extend msg created: domain: 81ef11d0-4c0
c-47b4-8953-d61a6af442d8, volume: 269fa277-1db7-426f-891c-51b777311a79
Engine won't allow LSM when there is not enough free space in the destination domain:
Error on UI:
Error while executing action: Cannot move Virtual Machine Disk. Low disk space on target Storage Domain iscsi1-2.
Engine.log:
2014-01-16 14:26:32,935 WARN [org.ovirt.engine.core.bll.lsm.LiveMigrateVmDisksCommand] (ajp--127.0.0.1-8702-5) [1efa621d] CanDoAction of action LiveMigrateVmDisks failed. Reasons:VAR__ACTION__MOVE,VAR__TYPE__VM_D
ISK,ACTION_TYPE_FAILED_DISK_SPACE_LOW_ON_TARGET_STORAGE_DOMAIN,$storageName iscsi1-2
Verified on ovirt-3.4.0-alpha1
ovirt-engine-3.4.0-0.2.master.20140106180914.el6.noarch
Created attachment 757083 [details] logs Description of problem: we have two issues here. 1. when the target domain does not have enough disk space to create the initial volume we still sent the create volume to vdsm even after alerts that there is not enough disk space. 2. we do not calculate the amount of disk space in the target domain which causes the domain to reach to <1G Version-Release number of selected component (if applicable): sf17.4 How reproducible: 100% Steps to Reproduce: 1. create several vms and LSM their disks from one domain to the other several times. 2. 3. Actual results: even though we do not have enough disk space in both the src and dst domains we start the LSM Expected results: we should not start a LSM operation if we do not have enough disk space to move the disks. Additional info: logs 8691f03c-80a2-452d-9231-56e7f5acb81f::ERROR::2013-06-05 11:41:18,033::threadPool::209::Misc.ThreadPool.WorkerThread::(run) Task <function runTask at 0x7f5aab894230> failed Traceback (most recent call last): File "/usr/share/vdsm/storage/threadPool.py", line 201, in run cmd(args) File "/usr/share/vdsm/storage/storage_mailbox.py", line 75, in runTask ctask.prepare(cmd, *args) File "/usr/share/vdsm/storage/task.py", line 1159, in prepare raise self.error RuntimeError: Volume extension failed for hda (domainID: 7414f930-bbdb-4ec6-8132-4640cbb3c722, volumeID: 2cdffe84-7cbf-41d8-a0ad-34721f9e5956) 18d57932-f978-49eb-ac32-0a5c8a1be413::ERROR::2013-06-05 11:48:17,640::storage_mailbox::153::Storage.SPM.Messages.Extend::(processRequest) processRequest: Exception caught while trying to extend volume: 269fa277-1db7-426f-891c-51b777311a79 in domain: 81ef11d0-4c0c-47b4-8953-d61a6af442d8 Traceback (most recent call last): File "/usr/share/vdsm/storage/storage_mailbox.py", line 149, in processRequest pool.extendVolume(volume['domainID'], volume['volumeID'], size) File "/usr/share/vdsm/storage/securable.py", line 68, in wrapper return f(self, *args, **kwargs) File "/usr/share/vdsm/storage/sp.py", line 1324, in extendVolume sdCache.produce(sdUUID).extendVolume(volumeUUID, size, isShuttingDown) File "/usr/share/vdsm/storage/blockSD.py", line 1228, in extendVolume lvm.extendLV(self.sdUUID, volumeUUID, size) #, isShuttingDown) # FIXME File "/usr/share/vdsm/storage/lvm.py", line 1093, in extendLV free_size / constants.MEGAB)) VolumeGroupSizeError: Volume Group not big enough: ('81ef11d0-4c0c-47b4-8953-d61a6af442d8/269fa277-1db7-426f-891c-51b777311a79 3072 > 768 (MiB)',) 18d57932-f978-49eb-ac32-0a5c8a1be413::DEBUG::2013-06-05 11:48:17,641::storage_mailbox::109::Storage.SPM.Messages.Extend::(__init__) new extend msg created: domain: 81ef11d0-4c0 c-47b4-8953-d61a6af442d8, volume: 269fa277-1db7-426f-891c-51b777311a79