Bug 1304346

Summary:	[RFE] Prepare a method to compute (guess) the required memory for starting a VM
Product:	[oVirt] ovirt-engine	Reporter:	Martin Sivák <msivak>
Component:	BLL.Virt	Assignee:	Milan Zamazal <mzamazal>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Israel Pinto <ipinto>
Severity:	medium	Docs Contact:
Priority:	medium
Version:	4.0.0	CC:	bugs, hannsj_uhl, mavital, mgoldboi, michal.skrivanek, msivak, rgolan, sbonazzo, s.kieske
Target Milestone:	ovirt-4.0.0-beta	Keywords:	CodeChange, FutureFeature
Target Release:	4.0.0	Flags:	rule-engine: ovirt-4.0.0+ mavital: testing_plan_complete? mgoldboi: planning_ack+ michal.skrivanek: devel_ack+ mavital: testing_ack+
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Enhancement
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2016-08-03 11:16:35 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	Virt	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1304300, 1308609, 1362557

Description Martin Sivák 2016-02-03 11:42:52 UTC

Description of problem:

oVirt uses a fixed 65 MiB overhead per VM when computing the required memory. Unfortunately that is way too low and not enough at all especially because of two or three areas:

- Graphical adapters require some memory for framebuffers (and that can be couple of hundred MiBs when there are more displays configured)
- Memory hotplug currently increases the size of some kind of memory hash map that is needed to track the memory for the VM (see https://bugzilla.redhat.com/show_bug.cgi?id=1282833#c16)
- Disk buffers and connected LUNs might eat some memory too

It is even more complicated when platforms (i686, ppc) and machine types are added to the "equation".

Actual results:

We can get to a point where the engine will try to start a VM on a host with not enough available memory just because it things that the VM will fit.

This is not really good and might confuse users.


Possible solution:

We could create a method (on the VM object?) that would take all configuration from VM static and compute a "guess" that would be a bit more realistic than the current 65 MiB constant.

This value then has to be included in the REST API too to make other projects like optimizer aware of the overhead.

Comment 1 Michal Skrivanek 2016-02-03 14:33:14 UTC

please coordinate with SLA
please pay special attention to ppc64le where things are a bit different

we should be able to come up with something better than 65MB hardcoded in 4.0 for sure, maybe won't be perfect, but anything is better than that:)

Comment 2 Milan Zamazal 2016-04-25 16:23:44 UTC

Libvirt documentation says that the overhead of QEMU itself "is hard to determine so one needs guess and try". We follow that advice here. Based on various experiments and measurements I suggest the following formula:

  fixed + video + mem_coeff * memory + 8 * cpu + 8 * iothreads

where

- fixed is 64 MB on x86_64, and 100 MB on Power  
- video is amount of the video RAM allocated to the VM; it equals to `ram + vram' where ram and vram are values computed and set by Engine
- mem_coeff is 1/512 on x86_64 and 1/64 on Power
- memory is current VM memory on x86_64, and maximum possible VM memory (i.e. including amount for hotplug) rounded to the nearest higher or equal power of 2 on Power
- cpu is number of active CPU units (including cores and threads)
- iothreads is number of allocated IOThreads (0 if IOThreads are not enabled)

Please note that it is really based on the "guess and try" approach and the values are generally only approximate. We may find out that the values are not optimum ones (in either direction) or that something is missing in the formula. So we may need to adjust the formula in future based on newly acquired knowledge. For now, the computed value should be better than the current fixed 65 MB value and it should be probably on the conservative side (i.e. higher rather than lower).

Comment 3 Milan Zamazal 2016-04-26 11:14:54 UTC

Minor correction of the formula -- missing units:

fixed + video + mem_coeff * memory + 8 MB * cpu + 8 MB * iothreads

Comment 4 Sandro Bonazzola 2016-05-02 09:56:26 UTC

Moving from 4.0 alpha to 4.0 beta since 4.0 alpha has been already released and bug is not ON_QA.

Comment 7 Milan Zamazal 2016-05-12 10:02:36 UTC

There is a new method VmUtils.getTotalRequiredMemoryInMb available now that returns the whole memory requirements guess for the given VM (i.e. VM memory + all the overheads).

Comment 9 Israel Pinto 2016-08-03 11:12:04 UTC

Check with engine version 4.0.2.3-0.1.el7ev
Steps:
Run automation virt sanity the check nothing is broken with the code
Resutls:
PASS

Comment 10 Michal Skrivanek 2016-08-03 11:16:35 UTC

method is ready in 4.0