Bug 563588

Created attachment 390585 [details]
Port virtio-balloon statistics to RHEL-5.5 kernel


------- Comment on attachment From agl.com 2010-02-12 15:52 EDT-------


This patch adds support for memory statistics reporting to the virtio balloon driver.  This feature is upstream.  See the following links:

* Virtio Specification (defines the interface in the virtio-balloon section)
http://ozlabs.org/~rusty/virtio-spec/

* Upstream patch acceptance
http://marc.info/?l=linux-virtualization&m=125959810620784&w=2

This description is from the accepted patch:

Changes since V3:
 - Do not do endian conversions as they will be done in the host
 - Report stats that reference a quantity of memory in bytes
 - Minor coding style updates

Changes since V2:
 - Increase stat field size to 64 bits
 - Report all sizes in kb (not pages)
 - Drop anon_pages stat and fix endianness conversion

Changes since V1:
 - Use a virtqueue instead of the device config space

When using ballooning to manage overcommitted memory on a host, a system for
guests to communicate their memory usage to the host can provide information
that will minimize the impact of ballooning on the guests.  The current method
employs a daemon running in each guest that communicates memory statistics to a
host daemon at a specified time interval.  The host daemon aggregates this
information and inflates and/or deflates balloons according to the level of
host memory pressure.  This approach is effective but overly complex since a
daemon must be installed inside each guest and coordinated to communicate with
the host.  A simpler approach is to collect memory statistics in the virtio
balloon driver and communicate them directly to the hypervisor.

This patch enables the guest-side support by adding stats collection and
reporting to the virtio balloon driver.

Created attachment 402147 [details]
[PATCH 1/3] virtio: Add memory statistics reporting to the balloon driver (V4)


------- Comment on attachment From agl.com 2010-03-23 17:32 EDT-------


Author: Adam Litke <agl.com>
Date:   Mon Nov 30 10:14:15 2009 -0600

    virtio: Add memory statistics reporting to the balloon driver (V4)
    
    Changes since V3:
     - Do not do endian conversions as they will be done in the host
     - Report stats that reference a quantity of memory in bytes
     - Minor coding style updates
    
    Changes since V2:
     - Increase stat field size to 64 bits
     - Report all sizes in kb (not pages)
     - Drop anon_pages stat and fix endianness conversion
    
    Changes since V1:
     - Use a virtqueue instead of the device config space
    
    When using ballooning to manage overcommitted memory on a host, a system for
    guests to communicate their memory usage to the host can provide information
    that will minimize the impact of ballooning on the guests.  The current method
    employs a daemon running in each guest that communicates memory statistics to a
    host daemon at a specified time interval.  The host daemon aggregates this
    information and inflates and/or deflates balloons according to the level of
    host memory pressure.  This approach is effective but overly complex since a
    daemon must be installed inside each guest and coordinated to communicate with
    the host.  A simpler approach is to collect memory statistics in the virtio
    balloon driver and communicate them directly to the hypervisor.
    
    This patch enables the guest-side support by adding stats collection and
    reporting to the virtio balloon driver.
    
    Signed-off-by: Adam Litke <agl.com>
    Cc: Anthony Liguori <anthony>
    Cc: virtualization.org
    Signed-off-by: Rusty Russell <rusty.au> (minor fixes)

Created attachment 402148 [details]
[PATCH 2/3] virtio: Fix scheduling while atomic in virtio_balloon stats


------- Comment on attachment From agl.com 2010-03-23 17:33 EDT-------


Author: Adam Litke <agl.com>
Date:   Thu Dec 10 16:35:15 2009 -0600

    virtio: Fix scheduling while atomic in virtio_balloon stats
    
    This is a fix for my earlier patch: "virtio: Add memory statistics reporting to
    the balloon driver (V4)".
    
    I discovered that all_vm_events() can sleep and therefore stats collection
    cannot be done in interrupt context.  One solution is to handle the interrupt
    by noting that stats need to be collected and waking the existing vballoon
    kthread which will complete the work via stats_handle_request().  Rusty, is
    this a saner way of doing business?
    
    There is one issue that I would like a broader opinion on.  In stats_request, I
    update vb->need_stats_update and then wake up the kthread.  The kthread uses
    vb->need_stats_update as a condition variable.  Do I need a memory barrier
    between the update and wake_up to ensure that my kthread sees the correct
    value?  My testing suggests that it is not needed but I would like some
    confirmation from the experts.
    
    Signed-off-by: Adam Litke <agl.com>
    To: Rusty Russell <rusty.au>
    Cc: Anthony Liguori <aliguori.ibm.com>
    Cc: linux-kernel.org
    Signed-off-by: Rusty Russell <rusty.au>

Created attachment 402149 [details]
[patch 3/3] virtio: fix balloon without VIRTIO_BALLOON_F_STATS_VQ


------- Comment on attachment From agl.com 2010-03-23 17:34 EDT-------


Author: Rusty Russell <rusty.au>
Date:   Wed Feb 24 14:22:14 2010 -0600

    virtio: fix balloon without VIRTIO_BALLOON_F_STATS_VQ
    
    When running under qemu-kvm-0.11.0:
    
    	BUG: unable to handle kernel paging request at 56e58955
    	...
    	Process vballoon (pid: 1297, ti=c7976000 task=c70a6ca0 task.ti=c7
    	...
    	Call Trace:
    	 [<c88253a3>] ? balloon+0x1b3/0x440 [virtio_balloon]
    	 [<c041c2d7>] ? schedule+0x327/0x9d0
    	 [<c88251f0>] ? balloon+0x0/0x440 [virtio_balloon]
    	 [<c014a2d4>] ? kthread+0x74/0x80
    	 [<c014a260>] ? kthread+0x0/0x80
    	 [<c0103b36>] ? kernel_thread_helper+0x6/0x30
    
    need_stats_update should be zero-initialized.
    
    Signed-off-by: Rusty Russell <rusty.au>
    Acked-by: Adam Litke <agl.com>

IBM is signed up to test and provide feedback.

This enhancement request was evaluated by the full Red Hat Enterprise Linux 
team for inclusion in a Red Hat Enterprise Linux minor release.   As a 
result of this evaluation, Red Hat has tentatively approved inclusion of 
this feature in the next Red Hat Enterprise Linux Update minor release.   
While it is a goal to include this enhancement in the next minor release 
of Red Hat Enterprise Linux, the enhancement is not yet committed for 
inclusion in the next minor release pending the next phase of actual 
code integration and successful Red Hat and partner testing.

These patches have entered the 5.6 tree by way of bug 601692.

Closing as dupe of BZ 601692 which is approved and in POST state.

*** This bug has been marked as a duplicate of bug 601692 ***