Bug 1008146 - When run multiple vms then filter don't check correctly free resources.
When run multiple vms then filter don't check correctly free resources.
Status: CLOSED CURRENTRELEASE
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine (Show other bugs)
3.3.0
Unspecified Unspecified
high Severity high
: ---
: 3.3.0
Assigned To: Martin Sivák
Lukas Svaty
sla
:
Depends On:
Blocks: 3.3snap4
  Show dependency treegraph
 
Reported: 2013-09-15 04:32 EDT by Ondra Machacek
Modified: 2016-02-10 15:14 EST (History)
11 users (show)

See Also:
Fixed In Version: is26
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: SLA
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
engine.log (13.30 KB, text/plain)
2013-09-15 04:32 EDT, Ondra Machacek
no flags Details
engine.log (44.05 KB, text/plain)
2013-09-16 08:17 EDT, Ondra Machacek
no flags Details
engine log (27.90 KB, text/x-log)
2013-12-05 06:50 EST, Lukas Svaty
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
oVirt gerrit 20172 None None None Never
oVirt gerrit 21514 None None None Never

  None (edit)
Description Ondra Machacek 2013-09-15 04:32:03 EDT
Created attachment 797822 [details]
engine.log

Description of problem:
When multiple vms are run at one time, then free resources of hosts are not
checked correctly. Ie. I have two hosts both have ~7GB free memory.
I have two vms both have 7GB guaranted memory. When I run those two vms at one
time then both vms start at one host, which has 7GB free memory.

Version-Release number of selected component (if applicable):
is13

How reproducible:
always

Steps to Reproduce:
1. Have two hosts both with ~7GB RAM.
2. Create cluster policy only with RAM filter.
3. Create two vms, both with ~7GB guaranteed memory.
4. Select both vms in webadmin and run them.

Actual results:
Both vms run on one host.

Expected results:
Both vms run on different hosts.

Additional info:
Comment 1 Doron Fediuck 2013-09-16 05:00:15 EDT
Hi Ondra,
I'm missing the log parts from the scheduling process. Can you please add them?

Also, what is your cluster policy optimization (servers? desktops?) 
There are several options that allow overcommitment, such as ksm, ballooning etc
so we really need you to specify all these settings to evaluate the memory calculations.
Comment 2 Ondra Machacek 2013-09-16 08:17:26 EDT
Created attachment 798263 [details]
engine.log

Hi Doron,

attaching log which reflect all stepts to reproduce
creating new cluster policy with filter RAM filter, selecting it as cluster policy
for cluster with those two hosts. creating two 7GB vms, and running them in
parallel.

I use Memory optimization - None - Disable memory page sharing.
I don't use balooning neither ksm.
Comment 3 Ondra Machacek 2013-09-16 08:20:50 EDT
Also note that when I run those vm one by one, then first start on the first
host and second vm on the second host and then when I try to migrate one
of those vm, it fails with error:

Error while executing action:
rhel_7gb:

    Cannot migrate VM. There are no available running Hosts with sufficient memory in VM's Cluster .
Comment 4 Martin Sivák 2013-09-23 04:49:07 EDT
I think I know the reason now:

The engine check the hosts for sufficient memory in SlaValidator.hasMemoryToRunVM. The fields that we use there are:

curVds.getMemCommited()
curVds.getPendingVmemSize()
curVds.getGuestOverhead()
curVds.getReservedMem()

All of them seem to be updated only when VDSM report comes in.

The field that contains the memory reserved for VMs is getMemCommited and the amount is recomputed in VdsUpdateRunTimeInfo.refreshCommitedMemory that is called only by refreshVmStats.

Since we poll VDSM every couple of seconds, the engine does not see the memory allocation when it is trying to start a second VM.

The fact that the second VM is properly scheduled to the second host when started with a small delay since the first VM seems to confirm this behaviour.
Comment 5 Lukas Svaty 2013-12-05 06:49:57 EST
both VMs failed to run on 1st host then started on 2nd host attaching engine.log
Comment 6 Lukas Svaty 2013-12-05 06:50:26 EST
Created attachment 833111 [details]
engine log
Comment 7 Martin Sivák 2013-12-09 09:00:40 EST
Please enable DEBUG mode when attaching logs. And describe the actual verification setup (especially the Cluster policy assigned and memory sizes) as the fix is working for me locally.
Comment 8 Martin Sivák 2013-12-12 04:16:05 EST
We just tested it with movciari and it seems to be working correctly. The environment probably had Memory filter disabled.
Comment 9 Lukas Svaty 2013-12-17 03:03:04 EST
retested, seems to be some kind of environment issue, which is fixed nowm moving to VERIFIED
Comment 10 Itamar Heim 2014-01-21 17:32:03 EST
Closing - RHEV 3.3 Released
Comment 11 Itamar Heim 2014-01-21 17:32:03 EST
Closing - RHEV 3.3 Released
Comment 12 Itamar Heim 2014-01-21 17:34:36 EST
Closing - RHEV 3.3 Released

Note You need to log in before you can comment on or make changes to this bug.