Bug 624189 - [virtio] virtio-balloon doesn't work after s3/s4 in kvm guest.
[virtio] virtio-balloon doesn't work after s3/s4 in kvm guest.
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: kernel (Show other bugs)
All Linux
low Severity medium
: rc
: ---
Assigned To: Amit Shah
Virtualization Bugs
: Reopened
Depends On:
Blocks: 580953 720669 748534
  Show dependency treegraph
Reported: 2010-08-14 01:56 EDT by lihuang
Modified: 2013-01-09 18:00 EST (History)
7 users (show)

See Also:
Fixed In Version: kernel-2.6.32-231.el6
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2012-06-20 03:35:21 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description lihuang 2010-08-14 01:56:48 EDT
Description of problem:
after implement s3/s4 inside kvm rhel6 i686 guest. 
query-balloon return incomplete statistics, only the "actual" size returned
balloon stop working.
to make it work again. need reload the virtio-balloon module inside guest.

following is the log

before s3 :
23:26:20.368: debug : qemuMonitorJSONCommandWithFd:217 : Send command '{"execute":"query-balloon"}' for write with FD -1
23:26:20.369: debug : qemuMonitorJSONIOProcessLine:115 : Line [{"return": {"actual": 2147483648, "mem_swapped_in": 0, "minor_page_faults": 482616, "mem_swapped_out": 0, "free_mem": 1981464576, "major_page_faults": 387, "total_mem": 2121478144}}]

after s3:
23:27:15.417: debug : qemuMonitorJSONCommandWithFd:217 : Send command '{"execute":"query-balloon"}' for write with FD -1
23:27:15.417: debug : qemuMonitorJSONIOProcessLine:115 : Line [{"return": {"actual": 2147483648}}]

23:27:15.734: debug : qemuMonitorJSONCommandWithFd:217 : Send command '{"execute":"balloon","arguments":{"value":1073741824}}' for write with FD -1
23:27:15.735: debug : qemuMonitorJSONIOProcessLine:115 : Line [{"return": {}}]

23:27:16.417: debug : qemuMonitorJSONCommandWithFd:217 : Send command '{"execute":"query-balloon"}' for write with FD -1
23:27:16.417: debug : qemuMonitorJSONIOProcessLine:115 : Line [{"return": {"actual": 2147483648}}]

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1. start a rhel6 guest
2. send qmp commond  {"execute":"query-balloon"}
3. (guest) echo mem >  /sys/power/state
4. send qmp commond  {"execute":"query-balloon"}
5. send qmp commond  {"execute":"balloon","arguments":{"value":1073741824}}
Actual results:

Expected results:
virtio-balloon  work after s3/s4 in kvm guest

Additional info:

qemu      4916     1  6 Aug13 ?        00:09:54 /usr/libexec/qemu-kvm -S -M rhel6.0.0 -enable-kvm -m 2048 -smp 2,sockets=2,cores=1,threads=1 -name s0807-clone -uuid 48429687-7da5-6a0d-9fa0-60bde3b2f71f -nodefconfig -nodefaults -chardev socket,id=monitor,path=/var/lib/libvirt/qemu/s0807-clone.monitor,server,nowait -mon chardev=monitor,mode=control -rtc base=utc -boot c -drive file=/home/images/s0807-clone.img,if=none,id=drive-ide0-0-0,boot=on,format=qcow2,cache=none -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -netdev tap,fd=21,id=hostnet0 -device e1000,netdev=hostnet0,id=net0,mac=52:54:00:a2:1c:62,bus=pci.0,addr=0x3 -chardev pty,id=serial0 -device isa-serial,chardev=serial0 -usb -device usb-tablet,id=input0 -vnc -vga cirrus -device AC97,id=sound0,bus=pci.0,addr=0x4 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6
Comment 2 Amit Shah 2011-05-30 06:50:20 EDT

*** This bug has been marked as a duplicate of bug 609286 ***
Comment 3 Amit Shah 2011-12-09 05:30:02 EST
Each virtio driver needs to be updated to handle PM, so re-opening this.
Comment 4 RHEL Product and Program Management 2011-12-12 23:39:55 EST
This request was evaluated by Red Hat Product Management for inclusion
in a Red Hat Enterprise Linux maintenance release. Product Management has 
requested further review of this request by Red Hat Engineering, for potential
inclusion in a Red Hat Enterprise Linux Update release for currently deployed 
products. This request is not yet committed for inclusion in an Update release.
Comment 5 Aristeu Rozanski 2012-02-15 14:55:59 EST
Patch(es) available on kernel-2.6.32-231.el6
Comment 8 Amit Shah 2012-02-27 06:06:29 EST
Note that after resuming from S4, the virtio balloon driver will completely deflate the balloon.  This is as designed, and is explained in the commit log:

Linux kernel commit e562966dbaf49e7804097cd991e5d3a8934fc148

    virtio: balloon: Add freeze, restore handlers to support S4
    Handling balloon hibernate / restore is tricky.  If the balloon was
    inflated before going into the hibernation state, upon resume, the host
    will not have any memory of that.  Any pages that were passed on to the
    host earlier would most likely be invalid, and the host will have to
    re-balloon to the previous value to get in the pre-hibernate state.
    So the only sane thing for the guest to do here is to discard all the
    pages that were put in the balloon.  When to discard the pages is the
    next question.
    One solution is to deflate the balloon just before writing the image to
    the disk (in the freeze() PM callback).  However, asking for pages from
    the host just to discard them immediately after seems wasteful of
    resources.  Hence, it makes sense to do this by just fudging our
    counters soon after wakeup.  This means we don't deflate the balloon
    before sleep, and also don't put unnecessary pressure on the host.
    This also helps in the thaw case: if the freeze fails for whatever
    reason, the balloon should continue to remain in the inflated state.
    This was tested by issuing 'swapoff -a' and trying to go into the S4
    state.  That fails, and the balloon stays inflated, as expected.  Both
    the host and the guest are happy.
    Finally, in the restore() callback, we empty the list of pages that were
    previously given off to the host, add the appropriate number of pages to
    the totalram_pages counter, reset the num_pages counter to 0, and
    all is fine.
    As a last step, delete the vqs on the freeze callback to prepare for
    hibernation, and re-create them in the restore and thaw callbacks to
    resume normal operation.
    The kthread doesn't race with any operations here, since it's frozen
    before the freeze() call and is thawed after the thaw() and restore()
    callbacks, so we're safe with that.
Comment 9 Amit Shah 2012-03-01 05:49:27 EST
(In reply to comment #8)
> Note that after resuming from S4, the virtio balloon driver will completely
> deflate the balloon.  This is as designed, and is explained in the commit log:

Ignore this; Bug 798583 tracks a fix to the optimisation that prevented the re-ballooning after restore.

Once the fix for that bug lands in, the previous value of the balloon will be applied after restore.
Comment 10 Qunfang Zhang 2012-04-19 06:03:10 EDT
Verified this bug on kernel-2.6.32-262.el6.x86_64, this issue does not exist.


/usr/libexec/qemu-kvm -M rhel6.3.0 -cpu Conroe -enable-kvm -m 2G -smp 2,sockets=1,cores=2,threads=1 -name rhel6.3 -uuid 4c84db67-faf8-4498-9829-19a3d6431d9d -rtc base=localtime,driftfix=slew -netdev tap,id=hostnet0,vhost=on,script=/etc/qemu-ifup -device virtio-net-pci,netdev=hostnet0,id=net0,mac=00:1a:2a:42:10:62,bus=pci.0,addr=0x3 -boot c -monitor stdio -spice port=5930,disable-ticketing -vga qxl -global qxl-vga.vram_size=33554432 -qmp tcp:0:4444,server,nowait -bios /usr/share/seabios/bios-pm.bin  -chardev socket,path=/tmp/qzhang-test-lucky,server,nowait,id=isa1 -device isa-serial,chardev=isa1,id=isa-serial1 -drive file=/dev/vgtest-qzhang/lv-6.3,if=none,format=qcow2,id=scsi0 -device virtio-scsi-pci,id=scsi0 -device scsi-disk,bus=scsi0.0,drive=scsi0,scsi-id=0,lun=0 -drive if=none,id=cd -device scsi-cd,drive=cd -usb -device usb-tablet,id=input0 -device virtio-balloon-pci,id=balloon0,bus=pci.0

(1) S3 -> wakeup -> balloon memory to other value. => Pass.
(2) Balloon mem to less value -> S3 -> wakeup -> balloon mem. => Pass
(3) S4 -> resume -> balloon memory => Pass
(4) Balloon memory -> S4 -> resume -> balloon mem. => Pass
(5) Do S3 and S4 alternately -> balloon mem => Pass

Based above, this issue is fixed.
Comment 12 errata-xmlrpc 2012-06-20 03:35:21 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.