Bug 1251744 - [scale] test ETP effect on performance
Summary: [scale] test ETP effect on performance
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine
Version: 3.6.0
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ovirt-3.6.3
: 3.6.0
Assignee: Pavel Stehlik
QA Contact: Eldad Marciano
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-08-09 14:03 UTC by Doron Fediuck
Modified: 2016-03-14 12:49 UTC (History)
10 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-03-14 12:49:05 UTC
oVirt Team: Infra
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
Test result (3.40 MB, application/msword)
2016-03-03 11:46 UTC, Yuri Obshansky
no flags Details

Description Doron Fediuck 2015-08-09 14:03:24 UTC
Based on a ML thread by Daniel Berrange we need to test whether this 
has a similar effect in RHEV when using Extended Page Table (EPT):

Tim Bell at CERN published a blog today where he is claiming that they actually see a performance /slowdown/ when EPT is enabled in KVM in their particular workload/benchmarks.

  "This is a 6% performance improvement with EPT off."

  http://openstack-in-production.blogspot.co.uk/2015/08/ept-and-ksm-for-high-throughput.html

Comment 1 Oved Ourfali 2015-08-10 06:19:25 UTC
Please specify what and how to test.
As far as I understand this is test-only bug?
If so, move to ON_QA, and put the TestOnly keyword.

Comment 2 Doron Fediuck 2015-08-10 06:40:01 UTC
1. Check that ept is currently on (this is the default):
# cat /sys/module/kvm_intel/parameters/ept
Y

2. Run a test utilizing cpu and memory and record the results.

3. Make sure you have no running VMs and kvm_intel module not being used.

4. Set a kvm config ept to N:
# cat >/etc/modprobe.d/kvm-intel.conf <<EOF
options kvm_intel ept=n
EOF

5. Reload the kernel module and verify ept state:
# modprobe -r kvm_intel
# modprobe  kvm_intel
# cat /sys/module/kvm_intel/parameters/ept
N

6. Run the same test you did in step 2 and record the results.

7. Restore ept state (kil VMs, change 'n' to 'y' in step 4, and run step 5).

8. Compare the results from step 2 and 6.

Comment 5 Yuri Obshansky 2016-03-03 11:45:44 UTC
Test performed on 
RHEVM: 3.6.3-0.1
KVM Version: 2.3.0 - 31.el7_2.7
LIBVIRT Version: libvirt-1.2.17-13.el7_2.3
VDSM Version: vdsm-4.17.19-0.el7ev

Test flow:
1. Check ETP settings
# cat /sys/module/kvm_intel/parameters/ept
Y
2. Populate and start 100 VMs on Hypervisor.
Run Load Test using JMeter 
100 concurrent threads simulated different actions on 100 VMs
which are running on single hypervisor/host during 2 hours
Each VM - 1 CPU, 1 G RAM
3. Save monitored statistics
4. Shutdown and remove VMs
4. Update ETL settings
# cat >/etc/modprobe.d/kvm-intel.conf <<EOF
options kvm_intel ept=n
EOF
5. Reload the kernel module and verify ept state:
# modprobe -r kvm_intel
# modprobe  kvm_intel
# cat /sys/module/kvm_intel/parameters/ept
N
6. Populate and start 100 VMs on Hypervisor and rerun Load Test.
7. Save monitored statistics
8, Analyse results.

Test showed us that we have opposite effect when change ETP to N.
The better result we reached while EPT is enabled in KVM.

Metrics name	                Module name	        EPT = Y	 EPT = N
CPU Utilization Server (%)	Engine node - System	4.42	 4.06
	                        Engine node - User	12.95	 6.8
	                        Host – System	        7.38	 11.78
	                        Host – User	        3.38	 3.8
CPU Utilization Process (%)	Engine node – Ovirt	99.83	 46.18
	                        Host node – LIbvirt	19.64	 28.04
	                        Host node – VDSM	28.39	 30.44
Memory Util Server (Mb)	        Engine node - Used	27681.43 28608.76
	                        Host - Used	        35155.52 34572.57
Memory Util Process (Mb)	Engine node – Ovirt	6904.62	6984.05
	                        Host node – LIbvirt	32.36	32.34
	                        Host node – VDSM	542.48	665.1
Network Utilization (Mb)	Engine node – Recv	3.71	2.28
	                        Engine node – Sent	3.96	2.41
	                        Host – Recv	        57.02	50.21
	                        Host – Sent	        1.89	2.21
Disk Utilization (b/sec)	Engine node – Reads	0	0
	                        Engine node – Writes	131.33	113.94
	                        Host – Reads	        0.02	0.01
	                        Host – Writes	        211.16	326.43

See attached detailed document

Comment 6 Yuri Obshansky 2016-03-03 11:46:29 UTC
Created attachment 1132731 [details]
Test result

Comment 7 Eyal Edri 2016-03-14 12:49:05 UTC
RHEV 3.6 GA was released.
This bug didn't have any required doc text, closing.


Note You need to log in before you can comment on or make changes to this bug.