Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1114226

Summary: [RFE] optimize ksm scanning pressure
Product: [Retired] oVirt Reporter: Markus Stockhausen <mst>
Component: vdsmAssignee: Dan Kenigsberg <danken>
Status: CLOSED DEFERRED QA Contact: Gil Klein <gklein>
Severity: medium Docs Contact:
Priority: high    
Version: 3.4CC: asegundo, bazulay, bugs, dfediuck, gklein, iheim, mgoldboi, rbalakri, s.kieske, yeylon
Target Milestone: ---Keywords: FutureFeature
Target Release: 3.6.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: sla
Fixed In Version: Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-04-19 14:12:06 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: SLA RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
ksm graphs none

Description Markus Stockhausen 2014-06-28 20:59:47 UTC 
Setup: 

Ovirt hypervisor nodes with ksm. Several running VMs with high memory usage but light load. That means ksm shared pages are not split very often. VDSM version is 4.14.9-0.fc20

Observation:

vdsm calculates the ksm parameter "pages_to_scan" mainly from the used memory. As memory usage rises ksm pressure is increased with higher values for "pages_to_scan". This value can be far too high in the above described setup.

Desired behaviour:

vdsm should calculate pages_to_scan not only from the memory load but also from the memory changing speed. If this is too complex at least the cpu consumption should be taken into consideration.

Additional info. 

attached you will find a screenshot of one of our servers. The graphs consist of

1) a CPU consumption graph. The main aspects are 
- green: the load generated by VMs (aggregated values from /proc/qemu-pid/stat)
- orange: the load generated by ksm (system time from /proc/ksm-pid/stat)

2) a memory consumption graph with colors:
- orange: ksm pages_shared
- blue: normal used memory
- black line: memory that would be used if ksm sharing would be disabled

3) ksm pressure
- value of /sys/kernel/mm/ksm/pages_to_scan

As one can see with default settings pages_to_scan is set to 1200 by vdsm. Nevertheless the same effect can be reached by setting the parameter to 150. Without sacrifying memory we can reduce the CPU load by ksm.
Comment 1 Markus Stockhausen 2014-06-28 21:00:35 UTC 
Created attachment 913093 [details]
ksm graphs
Comment 2 Markus Stockhausen 2014-06-29 11:33:53 UTC 
From reading /etc/vdsm/mom.d/03-ksm.policy current logic boils down to the following single comparison:

       (if (< (Host.StatAvg "mem_free") ksm_pressure_threshold)
            (change_npages ksm_pages_boost)
           (change_npages ksm_pages_decay))

- define hard limit at 20% of total memory
- if free memory is above limit -> decrease pages_to_scan by 50
- if free memory is below limit -> increase pages_to_scan by 300

E.g. in case of a 128 GB we always will have a "full speed" running ksmd when memory usage is above 102GB. Or in other words: We assume that we can compress the memory down to 80% if we only raise the ksm pressure to the max.
Comment 3 Markus Stockhausen 2014-06-29 12:00:15 UTC 
Just noticed another interesting side effect. The calculation does not consider swap usage. So

- Mem usage below 80% & swap used (might even be full) -> ksm is on light load
- Mem usage above 80% & swap empty -> ksm is on full load.
Comment 4 Doron Fediuck 2015-04-19 14:12:06 UTC 
As a part of 3.6.0 we're planning to handle Bug 1093038, which may resolve
this issue as well. Closing issue for now. If still relevant post Bug 1093038 resolution please re-open and explain what (more) should be done.