Bug 1461476 - Extend sla architecture to consider hugepages-enabled VMs
Extend sla architecture to consider hugepages-enabled VMs
Status: VERIFIED
Product: ovirt-engine
Classification: oVirt
Component: Backend.Core (Show other bugs)
4.2.0
Unspecified Unspecified
high Severity high (vote)
: ovirt-4.2.0
: 4.2.0
Assigned To: Martin Sivák
Artyom
: FutureFeature, Triaged
Depends On: 1481246
Blocks: 1457239
  Show dependency treegraph
 
Reported: 2017-06-14 10:31 EDT by Martin Polednik
Modified: 2017-10-17 08:31 EDT (History)
13 users (show)

See Also:
Fixed In Version:
Doc Type: No Doc Update
Doc Text:
undefined
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: SLA
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
rule-engine: ovirt‑4.2+
rule-engine: blocker+
alukiano: testing_plan_complete+
mgoldboi: planning_ack+
rule-engine: devel_ack+
mavital: testing_ack+


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
oVirt gerrit 79504 master MERGED common: Fix HugePagesUtils units (KiB vs MiB) and make it static 2017-07-19 04:21 EDT
oVirt gerrit 79505 master MERGED scheduling: Add support for HugePages to scheduler 2017-07-19 11:24 EDT

  None (edit)
Description Martin Polednik 2017-06-14 10:31:47 EDT
The scheduler should be able to schedule VMs that have memory backed by memory - such VMs do not consume regular memory pages we normally consider, but rather hugepages from previously allocated pool.

Specification of hugepages feature:
- hugepage cannot be split: VM with 800 MB ram and hugepage size of 1048576 will consume exactly one hugepage
- only the memory backing of the guest is stored on hugepages; QEMU overhead resides in regular memory
- host reports available hugepages for VM scheduling in it's stats
```
    ...
    "hugepages": {
        "1048576": {
            "resv_hugepages": 0,
            "free_hugepages": 16,
            "nr_overcommit_hugepages": 0,
            "surplus_hugepages": 0,
            "vm.free_hugepages": 16, # <--- the amount of hugepages that we may use for VMs (with hugepage size of 1048576)
            "nr_hugepages": 16,
            "nr_hugepages_mempolicy": 16
        },
        "2048": {
            "resv_hugepages": 0,
            "free_hugepages": 0,
            "nr_overcommit_hugepages": 0,
            "surplus_hugepages": 0,
            "vm.free_hugepages": 0, # <--- the amount of hugepages that we may use for VMs (with hugepage size of 2048)
            "nr_hugepages": 0,
            "nr_hugepages_mempolicy": 0
        }
    },
    "dateTime": "2017-06-14T13:02:53 GMT",
    ...
```
- - scheduler *must* use the highlighted field as certain number of system free hugepages may be reserved for something else (e.g. DPDK)
- host reports available hugepages sizes in capabilities
```
    ...
    "hostedEngineDeployed": false,
    "hugepages": [
        1048576,
        2048
    ],
    "kvmEnabled": "true",
    ...
```
- allocating hugepages reduces available system memory by number of hugepages times hugepages size (even if the hugepages are unused)
- - in other words, each hugepage size can be treated as separate memory pool
- - example: if the host has 64 GiB RAM and 14 1048576 hugepages, it's free memory follows 0 <= free memory <= (64 - 14) GiB
- - the hugepages will be allocated at the boot time
- as far as scheduler is concerned, the hugepages cannot be dynamically (de)allocated
- migration for VMs with hugepages is disabled
- memory hot(unplug) is disabled
- no overcommit
- quota support (maximum number of hugepages consumed by user)
- no special permissions required
- not numa aware at the moment
Comment 1 Yaniv Kaul 2017-06-14 11:45:46 EDT
Do we wish to have different clusters for hosts with static huge pages?
Comment 2 Michal Skrivanek 2017-06-15 07:08:30 EDT
it wouldn't really help much with most of the work needed. once scheduler is aware there is not real need to separate. I mean, sure, it may make sense to do that, but I do not see a reason to enforce it
Comment 5 Michal Skrivanek 2017-06-21 06:21:10 EDT
sorry, it was supposed to be an oVirt bug
Comment 6 Artyom 2017-09-26 06:22:57 EDT
Verified according to polarion plan.
ovirt-engine-4.2.0-0.0.master.20170921184504.gitfcfc9a7.el7.centos.noarch

Small notes:
1) Quota does not recognize hugepages - Martin Sivak said it expected behavior.
2) Memory hot-plug does not work for 1 GB pages - https://bugzilla.redhat.com/show_bug.cgi?id=1495535
3) Memory hot-unplug works for 2 MB hugepages, I need to get answer if it expected behavior or bug(see under description "memory hot(unplug) is disabled")

Note You need to log in before you can comment on or make changes to this bug.