Bug 822092

Summary: RFE: QMP notification of balloon memory changes
Product: Red Hat Enterprise Linux 7 Reporter: Daniel Berrangé <berrange>
Component: qemu-kvmAssignee: Daniel Berrangé <berrange>
Status: CLOSED CURRENTRELEASE QA Contact: Virtualization Bugs <virt-bugs>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.0CC: amit.shah, areis, berrange, flang, juzhang, knoel, michen, mrezanin, qzhang, virt-maint, ypu
Target Milestone: rcKeywords: FutureFeature
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: qemu-kvm-1.4.0-1.el7 Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-06-13 12:19:28 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 822094    

Description Daniel Berrangé 2012-05-16 10:18:18 UTC
Description of problem:
Currently whenever libvirt runs its virDomainGetXMLDesc or virDomainGetInfo APIs, it must issue a 'query-balloon' monitor call, to determine what level the guest has currently ballooned to. This is problematic because it slows down this APIs, which would otherwise be very fast and makes libvirt & the mgmt app susceptible to a "hung" QEMU monitor.  This has caused real world pain for RHEV-M, virt-manager and GNOME Boxes and thus it is well overdue time for us to fix this problem.

QEMU knows exactly when the guest changes its actual balloon level and can trivially emit a monitor event to notify mgmt application when this happens. This would avoid the need to poll 'query-balloon' at all, and thus avoids any potential for slow operation / hangs of the afore mentioned libvirt APIs.

Comment 1 Daniel Berrangé 2012-05-16 10:51:50 UTC
Here is an suitable impl

http://lists.nongnu.org/archive/html/qemu-devel/2012-05/msg02215.html

Comment 2 Daniel Berrangé 2012-05-16 13:16:01 UTC
It would also need the 'query-events' command to let libvirt detect whether the balloon event is present

https://lists.gnu.org/archive/html/qemu-devel/2012-05/msg02255.html

Comment 3 Luiz Capitulino 2012-05-18 13:35:32 UTC
Daniel, are you going to keep pursuing this upstream? If you will, then I think this bz has to be re-assigned to you, because a backport won't be needed (ie. this will be automatically in RHEL7.0 if merged before QEMU 1.2).

Comment 4 Daniel Berrangé 2012-05-18 13:44:41 UTC
Yes, I aim to continue posting updates upstream.  Depending on the final result, I may ultimately request a backport for RHEL-6, but I'll create a separate BZ for that when required.

Comment 5 Luiz Capitulino 2012-05-18 13:54:33 UTC
Ok, re-assigning to you then. If this misses the 1.2 deadline for whatever reason, then you can re-assign it back to me for the RHEL7.0 backport work.

Comment 6 Daniel Berrangé 2012-05-21 19:33:13 UTC
V2 of the patches, with rate limiting added

http://lists.nongnu.org/archive/html/qemu-devel/2012-05/msg02833.html

Comment 7 Daniel Berrangé 2012-06-15 16:44:02 UTC
All 3 patches now accepted upstream in Luiz' QMP tree

commit 4860853d60ecea44b65e9cdefce980de3a641dce
Author: Daniel P. Berrange <berrange>
Date:   Mon May 21 17:59:51 2012 +0100

    Add 'query-events' command to QMP to query async events


commit 1bcf107bbae883540f45e11e9fdc912a496c3094
Author: Daniel P. Berrange <berrange>
Date:   Wed May 16 11:01:58 2012 +0100

    Add event notification for guest balloon changes


commit d218a99b8d590224229969594e3411e940a77a7b
Author: Daniel P. Berrange <berrange>
Date:   Mon May 21 17:27:20 2012 +0100

    Add rate limiting of RTC_CHANGE, BALLOON_CHANGE & WATCHDOG events

Comment 8 Miroslav Rezanina 2013-04-23 12:48:40 UTC
Fixed in qemu-kvm-1.4.0-1.el7

Comment 10 langfang 2014-01-26 06:29:08 UTC
Test this bug on latest version:
Host:
# uname -r
3.10.0-79.el7.x86_64
# rpm -q qemu-kvm
qemu-kvm-1.5.3-41.el7.x86_64

steps:
1.boot guest with 
 ...-device virtio-balloon-pci,bus=pci.0,id=balloon0

2.QMP
# telnet 10.66.83.192 4444
{"execute":"query-balloon"}
{"return": {"actual": 2147483648}}
{"execute":"balloon","arguments":{"value":900000}}
{"return": {}}
{"timestamp": {"seconds": 1390717029, "microseconds": 460666}, "event": "BALLOON_CHANGE", "data": {"actual": 2146435072}}
{"timestamp": {"seconds": 1390717030, "microseconds": 460216}, "event": "BALLOON_CHANGE", "data": {"actual": 455081984}}
{"timestamp": {"seconds": 1390717030, "microseconds": 752577}, "event": "BALLOON_CHANGE", "data": {"actual": 300941312}}

3.{"execute":"query-events"}

Results:
After step 2, when change balloon size ,added the balloon events
After step 3,will see the balloon event:
...{"name": "BALLOON_CHANGE"},...{"name": "RTC_CHANGE"}..{"name": "WATCHDOG"}...

Test on old version:
Host:
# uname -r
3.7.0-0.36.el7.x86_64
# rpm -q qemu-kvm
qemu-kvm-1.3.0-3.el7.x86_64

i can get the same results.

Addtional test:

Also test change balloon size too small and big on latest version, not hit any problem.

Hi, Daniel

   According to above test , could we can verify pass this bug ?

tanks

Comment 11 Daniel Berrangé 2014-01-27 10:03:22 UTC
When the balloon target is set, the guest will slowly adjust to meat the target. This will result in a series of BALLOON_CHANGE events. You should wait for the last event to arrive, and then compare the value with the guest RAM shown in /proc/meminfo inside the guest. They should be pretty much the same, if not identical.

Comment 12 langfang 2014-01-29 11:56:32 UTC
(In reply to Daniel Berrange from comment #11)
> When the balloon target is set, the guest will slowly adjust to meat the
> target. This will result in a series of BALLOON_CHANGE events. You should
> wait for the last event to arrive, and then compare the value with the guest
> RAM shown in /proc/meminfo inside the guest. They should be pretty much the
> same, if not identical.


yes ,test on latest version

steps:
1.Boot guest with 
/usr/libexec/qemu-kvm -m 4G ....-device virtio-balloon-pci,bus=pci.0,id=balloon0

2.Change balloon size 
# telnet 10.66.83.192 4445
Trying 10.66.83.192...
Connected to 10.66.83.192.
Escape character is '^]'.
{"QMP": {"version": {"qemu": {"micro": 3, "minor": 5, "major": 1}, "package": " (qemu-kvm-1.5.3-41.el7)"}, "capabilities": []}}
{"execute":"qmp_capabilities"}
{"return": {}}
{"execute":"query-balloon"}                      
{"return": {"actual": 4294967296}}
{"execute":"balloon","arguments":{"value":1073741824}}
{"return": {}}
{"timestamp": {"seconds": 1390995440, "microseconds": 402484}, "event": "BALLOON_CHANGE", "data": {"actual": 4293918720}}
{"timestamp": {"seconds": 1390995441, "microseconds": 402074}, "event": "BALLOON_CHANGE", "data": {"actual": 2600468480}}
{"timestamp": {"seconds": 1390995442, "microseconds": 279347}, "event": "BALLOON_CHANGE", "data": {"actual": 1073741824}}
{"execute":"query-balloon"}
{"return": {"actual": 1073741824}}

3.In guest
# cat /proc/meminfo
MemTotal:         770748 kB
MemFree:          503988 kB
Buffers:            1468 kB
...

4.extend the balloon size
{"execute":"balloon","arguments":{"value":  2147483648}}
{"timestamp": {"seconds": 1390995917, "microseconds": 311927}, "event": "BALLOON_CHANGE", "data": {"actual": 806354944}}
{"timestamp": {"seconds": 1390995917, "microseconds": 837996}, "event": "BALLOON_CHANGE", "data": {"actual": 2147483648}}

4.In guest
# cat /proc/meminfo
MemTotal:        1819324 kB
MemFree:         1552348 kB
Buffers:            1468 kB
Cached:            98656 kB
SwapCached:            0 kB
Active:            97880 kB
....


According to above test ,when balloon size change,will show the balloon events ,and the mem size in guest (cat /proc/meminfo) will change too.


Hi,Daniel
    Could we can verify this bug now?

Comment 13 Daniel Berrangé 2014-01-29 12:04:07 UTC
Looks reasonable to me.

Comment 15 Ludek Smid 2014-06-13 12:19:28 UTC
This request was resolved in Red Hat Enterprise Linux 7.0.

Contact your manager or support representative in case you have further questions about the request.