Bug 663264

Summary: Poor hugepage performance with enabled_ept=0
Product: Red Hat Enterprise Linux 5 Reporter: Wang Liao <wliao>
Component: kvmAssignee: Marcelo Tosatti <mtosatti>
Status: CLOSED WONTFIX QA Contact: Virtualization Bugs <virt-bugs>
Severity: medium Docs Contact:
Priority: low    
Version: 5.6CC: mkenneth, tburke, virt-maint
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2010-12-15 11:28:25 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Wang Liao 2010-12-15 07:15:07 UTC
Description of problem:
With kvm_intel module inserted with enabled_ept=0, utilizing hugepage in guest encounters performance decrease compared with original 4kb page.

Version-Release number of selected component (if applicable):
RHEL5.6-Server-20101208.1
kvm-83-219.el5

How reproducible:
always

Steps to Reproduce:
1.Insert module disabling ept
rmmod kvm_intel
modprobe kvm_intel enable_ept=0
2.Boot guest without hugepage
/usr/libexec/qemu-kvm -m 8G -smp 8 -drive file=/var/lib/libvirt/images/rhel5.6.img,cache=none,format=raw,if=virtio,boot=on,werror=stop -drive file=/root/rhel5_add.img,cache=none,format=raw,if=virtio,boot=off,werror=stop -net nic,vlan=1,macaddr=00:11:ff:33:54:55,model=virtio -net tap,vlan=1 -boot c -uuid `uuidgen` -rtc-td-hack -no-kvm-pit-reinjection -balloon none -monitor stdio -vnc :1
3.Compile kernel with 16 jobs
time make -j 16
4.Mount hugetlbfs
mount -t hugetlbfs none /mnt/kvm_hugepages
5.Allocate hugepages
echo 5000 > /proc/sys/vm/nr_hugepages
6.Boot guest with hugepage
 /usr/libexec/qemu-kvm -m 8G -smp 8 -drive file=/var/lib/libvirt/images/rhel5.6.img,cache=none,format=raw,if=virtio,boot=on,werror=stop -drive file=/root/rhel5_add.img,cache=none,format=raw,if=virtio,boot=off,werror=stop -net nic,vlan=1,macaddr=00:11:ff:33:54:55,model=virtio -net tap,vlan=1 -boot c -uuid `uuidgen` -rtc-td-hack -no-kvm-pit-reinjection -balloon none -monitor stdio -vnc :1 -mem-path /mnt/kvm_hugepage/
7.Compile kernel with 16 jobs
time make -j 16
 
Actual results:
Time elapsed with guest hugepage is longer than that without hugepage.
(Detailed in Additional info)

Expected results:

Time elapsed with guest hugepage should be less than that without hugepage.

Additional info:

Environment:

CPU:
processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 44
model name	: Intel(R) Xeon(R) CPU           E5620  @ 2.40GHz
stepping	: 2
cpu MHz		: 1596.000
cache size	: 12288 KB
physical id	: 1
siblings	: 4
core id		: 10
cpu cores	: 4
apicid		: 52
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx pdpe1gb rdtscp lm constant_tsc ida nonstop_tsc arat pni monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 4787.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

Mem:
MemTotal:     16423836 kB

Guest kernel:
2.6.18-225.el5

Detailed result:
+----------------------------+----------+----------+----------+
 |                                   |  real      |   user    |   sys     |
+----------------------------+----------+----------+----------+
 |EPT=1 hugepage=0   |3m30s   |21m40s  |3m55s   |
+----------------------------+----------+----------+----------+
 |EPT=1 hugepage=1   |3m25s   |21m14s  | 3m18s  |
+----------------------------+----------+----------+----------+
 |EPT=0 hugepage=0   |7m37s   |44m5s    | 14m14s|
+----------------------------+----------+----------+----------+
 |EPT=0 hugepage=1   |9m15s   |52m1s    |14m7s   |
+----------------------------+----------+----------+----------+
 |Native Host                 |3m11s   |20m16s |2m11s    |
+----------------------------+----------+----------+----------+