Bug 465532

Summary: RFE: libvirt should support KVM huge page backed memory
Product: [Fedora] Fedora Reporter: James Ralston <ralston>
Component: libvirtAssignee: Daniel Berrange <berrange>
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: high    
Version: rawhideCC: berrange, crobinso, hbrock, john.cooper, markmc, nelhawar, veillard, virt-maint
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 515285 (view as bug list) Environment:
Last Closed: 2009-09-15 06:43:27 EDT Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Bug Depends On:    
Bug Blocks: 498969    

Description James Ralston 2008-10-03 14:40:53 EDT
According to reports from KVM developers, backing a KVM virtual guest with hugetlbfs can improve the performance of guests up to 10%:

http://www.linux-kvm.com/content/get-performance-boost-backing-your-kvm-guest-hugetlbfs

The steps described in the above link aren't too difficult, but at the minimum, virt-manager should support passing –mem-path to qemu-kvm (e.g., allowing you to specify the path to the hugetlbfs to use).

(It would be super cool and spiffy if virt-manager were to allow the user to setup the hugetlbfs automagically, but that's a lot more complex, and would probably require a reboot. Being able to support using an existing hugetlbfs by specifying its path would be a reasonable compromise and should be easy to implement.)
Comment 1 Daniel Berrange 2008-10-03 15:31:52 EDT
I definitely do not want to expose the -mem-path implementation detail to users or virt-manager. They simply do not have the knowledge neccessary to understand what this means, nor how to leverage it. If we're to make use of this capability we need to figure out a better way to handle it in libvirt and/or kvm, preferably such that it 'just works' as it does in Xen world without any admin config.

Changing to libvirt, because this definitely isn't the job of virt-manager
Comment 2 James Ralston 2008-10-06 21:55:38 EDT
Well, the thing about hugepages is, you pretty much have to allocate them at system startup.  But memory reserved for hugepages can't be used normally by regular processes.

Also, it's not clear that virtualization could be the exclusive user of hugepages; there might be other things that could want to use hugepages.

Honestly, I think to *really* do this right, there needs to be some sort of system startup hook to allocate hugepages and mount hugetlsfs in a standard location (e.g., /hugepages).  There also needs to be some sort of configuration program (e.g., system-config-memory) to set the amount of memory allocated as hugepages.  (Maybe that could be folder into an existing utility.)

Then, when starting a virtual guest, libvirt could inspect /proc/meminfo to see if enough non-reserved hugepages are available to satisfy the virtual guest's memory size.  If yes, then libvirt automatically includes –mem-path on the command line; if not, then it doesn't.

Optionally, libvirt could expose tunables to virt-manager, so that users could, say, fine-tune which virtual guests grab hugepages, limit the number of hugepages that libvirt will attempt to grab (e.g., if other processes in the system were also wanting to use hugepages), et. al.

Thoughts?
Comment 3 Noura El hawary 2008-11-26 01:33:17 EST
This bug appears to have been reported against 'rawhide' during the Fedora 10 development cycle.
Changing version to '10'.

More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Comment 4 Mark McLoughlin 2009-08-04 03:23:29 EDT
Moving to rawhide since this is an RFE. See also:

  https://fedoraproject.org/wiki/Features/KVM_Huge_Page_Backed_Memory
Comment 5 Mark McLoughlin 2009-08-19 13:26:55 EDT
We're still planning on this for Fedora 12

See bug #515741 for a related kernel issue
Comment 6 Daniel Berrange 2009-08-26 11:00:49 EDT
Latest upstream patches

http://www.redhat.com/archives/libvir-list/2009-August/msg00480.html
Comment 7 Daniel Berrange 2009-09-15 06:43:27 EDT
Merged upstream and in rawhide

http://libvirt.org/git/?p=libvirt.git;a=commit;h=d823a05aef2baa0927e792322f1af45e18c325bd