Bug 1101915

Summary: [RFE] qemu-ga should report memory statistics
Product: Red Hat Enterprise Linux 6 Reporter: Jiri Belka <jbelka>
Component: qemu-kvmAssignee: Marc-Andre Lureau <marcandre.lureau>
Status: CLOSED WONTFIX QA Contact: Virtualization Bugs <virt-bugs>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 6.5CC: chayang, jbelka, juzhang, michen, mkenneth, msivak, qzhang, rbalakri, rpacheco, virt-maint
Target Milestone: rcKeywords: FutureFeature
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-12-01 16:01:47 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1102046    

Description Jiri Belka 2014-05-28 07:34:00 UTC
Description of problem:

qemu-ga should report memory statistics. IIUC a memory overcommitment manager (MOM) runs on the host and based on data from hosts and guest and its policy decides to give/take memory from guest via virtio memory ballooning.

right now the data from guest are taken (IIUC) from RHEVM's vdsm which gets populated by RHEVM ovirt-guest-agent.

too many agents :) qemu-ga is dependency of ovirt-guest-agent anyway. qemu-ga should report memory statistics so these data are available out of RHEVM environment. qemu-ga is not RHEVM specific, so it would be nice to have such data in basic qemu-ga.

Comment 2 Jiri Belka 2014-05-28 11:39:25 UTC
*** Bug 1101916 has been marked as a duplicate of this bug. ***

Comment 3 Marc-Andre Lureau 2015-07-29 17:19:42 UTC
No specific requirements? ovirt-guest-agent messages seem to be:

free-ram: total available memory, including cache (MemFree+Buffers+Cached from /proc/meminfo)

memory-stats: {
"mem_total": MemTotal from /proc/meminfo
"mem_free": MemFree+Buffers+Cached from /proc/meminfo
"mem_unused": MemFree from /proc/meminfo
"mem_cached": Cached from /proc/meminfo
"mem_buffers": Buffers from /proc/meminfo
"swap_usage": SwapTotal - SwapFree from /proc/meminfo
"swap_total": SwapTotal from /proc/meminfo
"swap_in": pswpin from /proc/vmstat
"swap_out": pswpout from /proc/vmstat
"pageflt": pgfault from /proc/vmstat
"pgmajfault": majflt from /proc/vmstat

}

Apparently, most fields are optionnal since win32 backend doesn't always implement them or have a 0 default value.

It shouldn't be too difficult or controversial to add to qemu, since the requirement is to have common interfaces for various guest OS (not having to rely on various guest agent operations, such as read file there or execute something for that other guest etc). Only the fields need to be better specified, but I suppose we can just follow Linux man definition.

Comment 5 Marc-Andre Lureau 2015-07-30 16:10:53 UTC
Would this cover your needs (both linux & win32)?

# Information about guest memory, in kiB.
#
# @mem-total: Total usable RAM.
#
# @mem-free:  Total of RAM that can be used without 
#             having to swap contents to disk.
#
# @mem-cached: In-RAM cache.
#
# @swap-total: Total amount of swap space available.
#
# @swap-free: Amount of swap space that is currently unused.
#
# @swap-in: Number of pages swapped-in per second.
#
# @swap-out: Number of pages swapped-out per second.
#
# @pf-major: Number of major page fault per second.
#
# @pf-minor: Number of minor page fault per second.
#

Comment 6 Jiri Belka 2015-07-31 06:41:30 UTC
If it reports same memory info as (oVirt|RHEVM)-guest-agent I'm fine, thx.

Comment 7 Marc-Andre Lureau 2015-07-31 08:41:59 UTC
(In reply to Jiri Belka from comment #6)
> If it reports same memory info as (oVirt|RHEVM)-guest-agent I'm fine, thx.

It's not clear what ovirt-guest-agent is supposed to report. I removed mem_buffers, because this information is unclear (similar to mem_cache). The rest of the fields should match more or less.

I'll use MemAvailable instead of MemFree from meminfo on Linux, see https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=34e431b0ae398fc54ea69ff85ec700722c9da773 for reasons.

So please confirm that the above proposal fits your needs before I submit to qemu.

thanks

Comment 8 Marc-Andre Lureau 2015-07-31 08:45:53 UTC
It might be difficult for me to convince qemu maintainer to have the last 4 fields, because they are not so easy to give, and it's not clear what you can do with that for me. I might need your help to justify all this.

Comment 9 Marc-Andre Lureau 2015-07-31 19:30:21 UTC
sent RFC series:
https://lists.nongnu.org/archive/html/qemu-devel/2015-07/msg06294.html

Comment 10 Jiri Belka 2015-08-03 09:25:05 UTC
Martin, if anybody would like to use qemu GA instead of ovirt-guest-agent with MoM would changes done by marcandre.lureau@ be enough? I don't say MoM would support memory info from qemu GA but idea was to have such info without ovirt/RHEVM specific app...

"qemu-ga should report memory statistics so these data are available out of RHEVM environment. qemu-ga is not RHEVM specific, so it would be nice to have such data in basic qemu-ga."

(If I'm completely mistaken how MoM gets memory info from guests then I apoligize.)

Comment 11 Martin Sivák 2015-08-05 08:26:56 UTC
Hi,

MOM reads the data from VDSM when started in oVirt environment. The list in comment #5 looks like vdsm api description and so the information is all we need. We are currently not using swap-in/out or pf-* fields for anything.

The actual stats collection in MOM is done either through VDSM (which requires our guest agent) or using libvirt's domain getMemoryStats _(https://gerrit.ovirt.org/gitweb?p=mom.git;a=blob;f=mom/HypervisorInterfaces/libvirtInterface.py;h=641dd2480f138fa8eb6c7c55a6275519b778f367;hb=de2e934a9e32d7e4d99a6a9aa9bcc3dc21d93ec4#l147) when running without oVirt.

So supporting the same fields in both guest agents is a good start, but it will take a change to VDSM to be able to use qemu quest agent directly I think.

Comment 13 Marc-Andre Lureau 2015-12-01 14:40:36 UTC
The patches for qemu-ga have been posted:
https://lists.nongnu.org/archive/html/qemu-devel/2015-07/msg06294.html

However Daniel pointed out redundant function with the balloon driver. The discussion continued, but it needs better rationale and thus I'll move the bug to needinfo.

Comment 14 Marc-Andre Lureau 2015-12-01 14:41:18 UTC
defering

Comment 15 Jiri Belka 2015-12-01 15:50:30 UTC
(In reply to Marc-Andre Lureau from comment #13)
> The patches for qemu-ga have been posted:
> https://lists.nongnu.org/archive/html/qemu-devel/2015-07/msg06294.html
> 
> However Daniel pointed out redundant function with the balloon driver. The
> discussion continued, but it needs better rationale and thus I'll move the
> bug to needinfo.

If similar data can be obtain directly from qemu via balloon driver it's even better as it is transparent and there's no need for additional agent.

Comment 16 Marc-Andre Lureau 2015-12-01 16:01:47 UTC
(In reply to Jiri Belka from comment #15)
> If similar data can be obtain directly from qemu via balloon driver it's
> even better as it is transparent and there's no need for additional agent.

closing then, thanks