Bug 515285
Summary: | RFE: support configuration of hugepages for guests | ||||||
---|---|---|---|---|---|---|---|
Product: | Red Hat Enterprise Linux 6 | Reporter: | Daniel Berrangé <berrange> | ||||
Component: | libvirt | Assignee: | Daniel Veillard <veillard> | ||||
Status: | CLOSED CURRENTRELEASE | QA Contact: | Virtualization Bugs <virt-bugs> | ||||
Severity: | medium | Docs Contact: | |||||
Priority: | medium | ||||||
Version: | 6.0 | CC: | ajia, berrange, crobinso, hbrock, jialiu, jjarvis, kmonroe, nelhawar, syeghiay, troels, veillard, virt-maint, xen-maint | ||||
Target Milestone: | beta | Keywords: | FutureFeature | ||||
Target Release: | 6.0 | ||||||
Hardware: | All | ||||||
OS: | Linux | ||||||
Whiteboard: | |||||||
Fixed In Version: | libvirt-0.7.1-2.el6.1 | Doc Type: | Enhancement | ||||
Doc Text: | Story Points: | --- | |||||
Clone Of: | 465532 | Environment: | |||||
Last Closed: | 2010-07-02 19:24:49 UTC | Type: | --- | ||||
Regression: | --- | Mount Type: | --- | ||||
Documentation: | --- | CRM: | |||||
Verified Versions: | Category: | --- | |||||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
Cloudforms Team: | --- | Target Upstream Version: | |||||
Embargoed: | |||||||
Bug Depends On: | 537389 | ||||||
Bug Blocks: | 356741, 554559 | ||||||
Attachments: |
|
Description
Daniel Berrangé
2009-08-03 15:34:33 UTC
cf initial patches from John Cooper http://www.redhat.com/archives/libvir-list/2009-July/msg00753.html This request was evaluated by Red Hat Product Management for inclusion in a Red Hat Enterprise Linux major release. Product Management has requested further review of this request by Red Hat Engineering, for potential inclusion in a Red Hat Enterprise Linux Major release. This request is not yet committed for inclusion. IBM is also interested in seeing the feature in RHEL 6 ------- Comment From ratliff.com 2009-08-18 14:58 EDT------- 1. Feature Overview: Feature Id: [202016] a. Name of Feature: libvirt control hugepages b. Feature Description Have libvirt provide controls over the use of hugepages to back VMs 2. Feature Details: Sponsor: LTC Architectures: x86 x86_64 Arch Specificity: Both Delivery Mechanism: Direct from community Category: Xen Request Type: Package - Feature from Upstream d. Upstream Acceptance: In Progress Sponsor Priority 2 f. Severity: Medium IBM Confidential: no Code Contribution: 3rd party code g. Component Version Target: latest version of libvirt 3. Business Case On systems with hardware paging support (EPT/NPT) using largepages through hugetlbfs to back the VM can provide a significant perforamce improvement for some workloads. Currently this isn't exposed through libvirt as a guest configurable option. Add support for this will allow managemnt software greater flexibility to control quest config in performance sensitive areas. 4. Primary contact at Red Hat: John Jarvis jjarvis 5. Primary contacts at Partner: Project Management Contact: Stephanie Glass, sglass.com, 512-838-9284 Technical contact(s): Ryan Harper, raharper.com IBM Manager: Warren Grunbok II, grunbok.com An updated patch has been submitted upstream with greater capabilities http://www.redhat.com/archives/libvir-list/2009-August/msg00480.html This was included in the 0.7.1 release of libvirt Now built in RHEL-6 version libvirt-0.7.1-2.el6.1 Fixed in 'libvirt-0.7.1-2.el6.1'. 'libvirt-0.7.1-2.el6.2' included in compose 'RHEL6.0-20091029.0'. Moving to ON_QA. Created attachment 368341 [details]
Domain config xml file
Test this defect with libvirt-0.7.1-2.el6.2 on x86_64, but failed. [root@dhcp-66-70-119 ~]# uname -a Linux dhcp-66-70-119.nay.redhat.com 2.6.31-33.el6.x86_64 #1 SMP Thu Oct 29 12:00 :41 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux [root@dhcp-66-70-119 ~]# more /etc/redhat-release Red Hat Enterprise Linux release 6.0 Alpha (Santiago) [root@dhcp-66-70-119 ~]# lsmod |grep kvm kvm_amd 45056 0 kvm 192512 1 kvm_amd [root@dhcp-66-70-119 ~]# rpm -qa|grep libvirt libvirt-0.7.1-2.el6.2.x86_64 libvirt-client-0.7.1-2.el6.2.x86_64 libvirt-python-0.7.1-2.el6.2.x86_64 libvirt-devel-0.7.1-2.el6.2.x86_64 libvirt-cim-0.5.6-2.el6.x86_64 =============Configure guest with hugepage support via libvirt============ domain config xml file: ... ... ... <memoryBacking> <hugepages/> </memoryBacking> ... ... ... Note: Domain config xml file(rhel5.4_i386.xml) has been attached. [root@dhcp-66-70-119 ~]# mkdir /dev/hugepages [root@dhcp-66-70-119 ~]# mount -t hugetlbfs hugtlbfs /dev/hugepages [root@dhcp-66-70-119 ~]# sysctl vm.nr_hugepages=516 vm.nr_hugepages = 516 [root@dhcp-66-70-119 ~]# cat /proc/mounts |grep hugtlb hugtlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime 0 0 [root@dhcp-66-70-119 ~]# service libvirtd restart Stopping libvirtd daemon: [ OK ] Starting libvirtd daemon: [ OK ] [root@dhcp-66-70-119 ~]# virsh start rhel5.4_i386 Domain rhel5.4_i386 started [root@dhcp-66-70-119 ~]# ps -ef|grep qemu qemu 2771 1 43 02:39 ? 00:00:45 /usr/bin/qemu-kvm -S -M pc-0.11 -cpu qemu32 -m 0 -mem-path /dev/hugepages/libvirt/qemu -smp 1 -name rhel5.4_i386 -uuid 1cd779e0-5bd9-e8bb-0b9b-4d4bedd28a8b -monitor unix:/var/lib/libvirt/qemu/ rhel5.4_i386.monitor,server,nowait -boot c -drive file=/var/lib/libvirt/images/r hel5.4_i386.img,if=ide,index=0,boot=on,format=raw -drive file=,if=ide,media=cdro m,index=2 -net nic,macaddr=52:54:00:26:dd:a4,vlan=0,name=nic.0 -net tap,fd=16,vl an=0,name=tap.0 -serial pty -parallel none -usb -vnc 127.0.0.1:0 -k en-us -vga c irrus NOTE: Seem hugepage support has been passed using "-mem-path" by libvirt. [root@dhcp-66-70-119 ~]# more /proc/meminfo |grep Hug HugePages_Total: 516 HugePages_Free: 516 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB NOTE: Looks like the hugepage is never used, so test failed. ========start kvm guest with hugepage support directly using qemu-kvm========= [root@dhcp-66-70-119 ~]# qemu-kvm -hda /var/lib/libvirt/images/rhel5.4_i386.img -name aaa -m 100 -mem-path /dev/hugepages/libvirt/qemu [root@dhcp-66-70-119 ~]# more /proc/meminfo |grep Hug HugePages_Total: 516 HugePages_Free: 458 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB NOTE: If start kvm guest directly using qemu-kvm command with "-mem-path" parameter, hugepage support works fine. I know why I still reproduced the defect, because in my steps, there is a typo in the command that I typed. "mount -t hugetlbfs hugtlbfs /dev/hugepages" should be correct to "mount -t hugetlbfs hugetlbfs /dev/hugepages" After I correct the typo, hugepage support for kvm guest works fine. Sorry for the inconvenience. I checked the defect again. For libvirt-0.7.1-2.el6.2 package, if selinux is turn on, the test failed as comment #12; if selinux is turn off, the test will passed. So the root cause of my test fail is selinux, so please ignore the comment #13. Confirmed that this is SELinux policy denying use of hugepages, and opened this bug to track resolution of that problem https://bugzilla.redhat.com/show_bug.cgi?id=537389 ------- Comment From agl.com 2010-06-07 16:55 EDT------- The SELinux policy has been corrected in Snap6. Marking as Verified. Verified this defect with libvirt-0.8.1-10.el6.x86_64 on nightly build (RHEL6.0-20100622.n.0), and PASSED. Steps: 1. #mkdir /dev/hugepages 2. #mount -t hugetlbfs hugetlbfs /dev/hugepages 3. #sysctl vm.nr_hugepages=100 4. #service libvirtd restart 5. cat meminfo before guest start # more /proc/meminfo |grep Huge AnonHugePages: 16384 kB HugePages_Total: 100 HugePages_Free: 100 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB 6. Run a guest with the following xml added to domain xml. <memoryBacking><hugepages/></memoryBacking> 7. Start the domain. 8. Check HugePages_Free in meminfo again, confirm the guest is using hugepages. # more /proc/meminfo |grep Huge AnonHugePages: 86016 kB HugePages_Total: 100 HugePages_Free: 99 HugePages_Rsvd: 7 HugePages_Surp: 0 Hugepagesize: 2048 kB Red Hat Enterprise Linux Beta 2 is now available and should resolve the problem described in this bug report. This report is therefore being closed with a resolution of CURRENTRELEASE. You may reopen this bug report if the solution does not work for you. |