Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.
Bug 609016 - incorrect committed memory on idle host
incorrect committed memory on idle host
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: qemu-kvm (Show other bugs)
6.0
All Linux
low Severity medium
: rc
: ---
Assigned To: Dan Kenigsberg
Virtualization Bugs
: ZStream
Depends On:
Blocks: 580954 634249
  Show dependency treegraph
 
Reported: 2010-06-29 04:15 EDT by lihuang
Modified: 2013-01-10 22:05 EST (History)
14 users (show)

See Also:
Fixed In Version: qemu-kvm-0.12.1.2-2.116.el6
Doc Type: Bug Fix
Doc Text:
Due to error in the committed_memory() function, the ksmtuned service was unable to determine the correct amount of memory used by qemu-kvm processes when no such process existed. This has been fixed, the relevant part of the source code has been corrected to return 0 when no qemu-kvm process is found, and ksmtuned now works as expected.
Story Points: ---
Clone Of:
Environment:
Last Closed: 2011-05-19 07:32:06 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
calculate committed_memory with no qemus (853 bytes, patch)
2010-08-26 04:22 EDT, Dan Kenigsberg
no flags Details | Diff


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2011:0534 normal SHIPPED_LIVE Important: qemu-kvm security, bug fix, and enhancement update 2011-05-19 07:20:36 EDT

  None (edit)
Description lihuang 2010-06-29 04:15:46 EDT
Description of problem:
there is a idle host,no qemu-kvm, no memery overcommit. 
but ksm scan is not stop .


[root@t78 ksm]# uptime 
 02:30:19 up  1:49,  4 users,  load average: 0.00, 0.00, 0.00
[root@t78 ksm]# free -m
             total       used       free     shared    buffers     cached
Mem:          7856        756       7099          0         50        212
-/+ buffers/cache:        493       7362
Swap:        10047          0      10047


[root@t78 ksm]# cat /sys/kernel/mm/ksm/run 
1
[root@t78 ksm]# cat /sys/kernel/mm/ksm/sleep_millisecs 
20
[root@t78 ksm]# cat /sys/kernel/mm/ksm/pages_to_scan 
64

[root@t78 ksm]# ps -aef | grep qemu
root     12118  2976  0 02:36 pts/0    00:00:00 grep qemu


from log /var/log/ksmtuned,the committed memory size is up to 9G.

Tue Jun 29 02:34:26 EDT 2010: committed 9100088 free 7537548
Tue Jun 29 02:34:26 EDT 2010: 10709059 > 8044856, start ksm
Tue Jun 29 02:34:26 EDT 2010: 7537548 > 1608971, decay
Tue Jun 29 02:34:26 EDT 2010: KSMCTL start 64 20
[root@t78 ksm]# date 
Tue Jun 29 02:34:59 EDT 2010

from /usr/sbin/ksmtuned. the committed memory is from "ps -o vsz"

committed_memory () {
    # calculate how much memory is committed to running qemu processes
    local progname
    progname=${1:-qemu}
    ps -o vsz `pgrep $progname` | awk '{ sum += $1 }; END { print sum }'
}


Version-Release number of selected component (if applicable):
qemu-kvm-0.12.1.2-2.82.el6.x86_64

How reproducible:
for the script it is 100%.but most of time ksm scan is stop because the committed memory size is not big enough

Steps to Reproduce:
1.start rhel6 host
2.
3.
  
Actual results:


Expected results:


Additional info:
Comment 2 RHEL Product and Program Management 2010-06-29 04:42:54 EDT
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux major release.  Product Management has requested further
review of this request by Red Hat Engineering, for potential inclusion in a Red
Hat Enterprise Linux Major release.  This request is not yet committed for
inclusion.
Comment 3 Dan Kenigsberg 2010-06-29 05:20:03 EDT
I see that ksmtuned tuned ksm down to its minimum number of scanned pages per iteration (64). This is the intended feature; we do not want to turn ksm off completely, as the guest may start dirtying and clearing pages any minute.

Is there a real problem? How much cpu does ksm take from your machine?
top -p `pgrep ksmd`
Comment 4 lihuang 2010-06-29 05:50:32 EDT
(In reply to comment #3)
> I see that ksmtuned tuned ksm down to its minimum number of scanned pages per
> iteration (64). This is the intended feature; we do not want to turn ksm off
> completely, as the guest may start dirtying and clearing pages any minute.
> 
from the script,if there is a qemu-kvm process running with low mem size. then ksm will stop.
So I got such a scenario :
boot  host -> ksm start
start vm   -> ksm top
quit  vm   -> ksm start
this is why created this BZ.

> Is there a real problem? How much cpu does ksm take from your machine?
> top -p `pgrep ksmd`    
yes, it is not a problem in cpu usage.
Comment 5 Dan Kenigsberg 2010-06-29 10:17:56 EDT
(In reply to comment #4)
> from the script,if there is a qemu-kvm process running with low mem size. then
> ksm will stop.

No. When started, ksmtuned will never stop ksmd. It will only tune it very low. This was the intended behavior.

> So I got such a scenario :
> boot  host -> ksm start
> start vm   -> ksm top
> quit  vm   -> ksm start

Now I've lost you. After you stared a VM, ksm was stopped? How did this happen? Did you run `service ksmtuned retune`?
Comment 6 lihuang 2010-06-29 13:06:34 EDT
(In reply to comment #5)
> (In reply to comment #4)
> > from the script,if there is a qemu-kvm process running with low mem size. then
> > ksm will stop.
> 
> No. When started, ksmtuned will never stop ksmd. It will only tune it very low.
> This was the intended behavior.
> 
seems ksmturned behavior differently ...please check following log.

> > So I got such a scenario :
> > boot  host -> ksm start
> > start vm   -> ksm top
> > quit  vm   -> ksm start
> 
> Now I've lost you. After you stared a VM, ksm was stopped? How did this happen?
> Did you run `service ksmtuned retune`?    


the scenario i got is as following,

Tue Jun 29 11:21:52 EDT 2010: committed 7975364 free 7525744
                                        ^^^^^^^
                                        boot host,no qemu-kvm process. so the committed mem size is from " ps -o vsz ",no idea why it is so high.
Tue Jun 29 11:21:52 EDT 2010: 9584335 > 8044856, start ksm
Tue Jun 29 11:21:52 EDT 2010: 7525744 > 1608971, decay
Tue Jun 29 11:21:52 EDT 2010: KSMCTL start 64 20

Tue Jun 29 11:22:52 EDT 2010: committed 2454768 free 6378492
                                        ^^^^^^^^
                                        start vm with 2G ram. so the committed mem size if from "ps -o vsz $pid_of_qemu-kvm"
Tue Jun 29 11:22:52 EDT 2010: 4063739 < 8044856 and free > 1608971, stop ksm
...
Tue Jun 29 11:35:53 EDT 2010: committed 2454692 free 6378100
Tue Jun 29 11:35:53 EDT 2010: 4063663 < 8044856 and free > 1608971, stop ksm

Tue Jun 29 11:36:53 EDT 2010: committed 9264764 free 7474880
                                        ^^^^^^^
                                        after stop guest.
Tue Jun 29 11:36:53 EDT 2010: 10873735 > 8044856, start ksm
Tue Jun 29 11:36:53 EDT 2010: 7474880 > 1608971, decay
Tue Jun 29 11:36:53 EDT 2010: KSMCTL start 64 20(In reply to comment #5)
Comment 7 RHEL Product and Program Management 2010-07-15 10:06:42 EDT
This issue has been proposed when we are only considering blocker
issues in the current Red Hat Enterprise Linux release. It has
been denied for the current Red Hat Enterprise Linux release.

** If you would still like this issue considered for the current
release, ask your support representative to file as a blocker on
your behalf. Otherwise ask that it be considered for the next
Red Hat Enterprise Linux release. **
Comment 8 Dan Kenigsberg 2010-08-17 04:12:05 EDT
Did you remember to run
  service ksmtuned retune
after qemu is started/stopped? I have no other guess why the logged committed memory is wrong.

(sorry for the late response, I probably lost the notification email)
Comment 12 Dan Kenigsberg 2010-08-26 04:22:42 EDT
Created attachment 441139 [details]
calculate committed_memory with no qemus

Only now do I get your point. Sorry for being a bit thick. Would the attached patch solve the issue?
Comment 15 lihuang 2010-08-29 22:38:32 EDT
(In reply to comment #12)
> Created attachment 441139 [details]
> calculate committed_memory with no qemus
> 
> Only now do I get your point. Sorry for being a bit thick. Would the attached
> patch solve the issue?

yes . works for me 
Sun Aug 29 22:37:53 EDT 2010: committed 0 free 7509288
Sun Aug 29 22:37:53 EDT 2010: 1612176 < 8060880 and free > 1612176, stop ksm
Comment 21 Keqin Hong 2010-10-15 06:35:05 EDT
Verified on qemu-kvm-0.12.1.2-2.116.el6. PASS

(no committed memory on idle host)
Steps:
1. edit /etc/ksmtuned.conf
# uncomment the following if you want ksmtuned debug info
 LOGFILE=/var/log/ksmtuned
 DEBUG=1
2. reboot host (or #service ksmtuned restart)
3. check "committed memory" at /var/log/ksmtuned
Comment 22 Oliver Henshaw 2010-10-16 13:00:47 EDT
In this patch you replace vsz by rsz. Was this intended?

I mean, it certainly seem to make more sense to use resident size than virtual size but I'm not sure that this change was deliberate - this bug isn't directly about use of vsz and the commit message http://gitorious.org/ksm-control-scripts/ksm-control-scripts/commit/1305fcb6bb2cfc0d86b3d5faed2056cca4c31164 doesn't mention it.
Comment 23 Jaromir Hradilek 2010-10-27 09:53:11 EDT
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
Due to error in the committed_memory() function, the ksmtuned service was unable to determine the correct amount of memory used by qemu-kvm processes when no such process existed. This has been fixed, the relevant part of the source code has been corrected to return 0 when no qemu-kvm process is found, and ksmtuned now works as expected.
Comment 25 Keqin Hong 2011-01-12 04:02:47 EST
Re-verified on qemu-kvm-0.12.1.2-2.129.el6.x86_64, using steps mentioned in Comment 21. PASS
Comment 26 errata-xmlrpc 2011-05-19 07:32:06 EDT
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2011-0534.html
Comment 27 errata-xmlrpc 2011-05-19 08:46:13 EDT
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2011-0534.html

Note You need to log in before you can comment on or make changes to this bug.