Bug 1251744 - [scale] test ETP effect on performance
[scale] test ETP effect on performance
Status: CLOSED CURRENTRELEASE
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine (Show other bugs)
3.6.0
Unspecified Unspecified
unspecified Severity high
: ovirt-3.6.3
: 3.6.0
Assigned To: Pavel Stehlik
Eldad Marciano
: TestOnly
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-08-09 10:03 EDT by Doron Fediuck
Modified: 2016-03-14 08:49 EDT (History)
10 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-03-14 08:49:05 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: Infra
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Test result (3.40 MB, application/msword)
2016-03-03 06:46 EST, Yuri Obshansky
no flags Details

  None (edit)
Description Doron Fediuck 2015-08-09 10:03:24 EDT
Based on a ML thread by Daniel Berrange we need to test whether this 
has a similar effect in RHEV when using Extended Page Table (EPT):

Tim Bell at CERN published a blog today where he is claiming that they actually see a performance /slowdown/ when EPT is enabled in KVM in their particular workload/benchmarks.

  "This is a 6% performance improvement with EPT off."

  http://openstack-in-production.blogspot.co.uk/2015/08/ept-and-ksm-for-high-throughput.html
Comment 1 Oved Ourfali 2015-08-10 02:19:25 EDT
Please specify what and how to test.
As far as I understand this is test-only bug?
If so, move to ON_QA, and put the TestOnly keyword.
Comment 2 Doron Fediuck 2015-08-10 02:40:01 EDT
1. Check that ept is currently on (this is the default):
# cat /sys/module/kvm_intel/parameters/ept
Y

2. Run a test utilizing cpu and memory and record the results.

3. Make sure you have no running VMs and kvm_intel module not being used.

4. Set a kvm config ept to N:
# cat >/etc/modprobe.d/kvm-intel.conf <<EOF
options kvm_intel ept=n
EOF

5. Reload the kernel module and verify ept state:
# modprobe -r kvm_intel
# modprobe  kvm_intel
# cat /sys/module/kvm_intel/parameters/ept
N

6. Run the same test you did in step 2 and record the results.

7. Restore ept state (kil VMs, change 'n' to 'y' in step 4, and run step 5).

8. Compare the results from step 2 and 6.
Comment 5 Yuri Obshansky 2016-03-03 06:45:44 EST
Test performed on 
RHEVM: 3.6.3-0.1
KVM Version: 2.3.0 - 31.el7_2.7
LIBVIRT Version: libvirt-1.2.17-13.el7_2.3
VDSM Version: vdsm-4.17.19-0.el7ev

Test flow:
1. Check ETP settings
# cat /sys/module/kvm_intel/parameters/ept
Y
2. Populate and start 100 VMs on Hypervisor.
Run Load Test using JMeter 
100 concurrent threads simulated different actions on 100 VMs
which are running on single hypervisor/host during 2 hours
Each VM - 1 CPU, 1 G RAM
3. Save monitored statistics
4. Shutdown and remove VMs
4. Update ETL settings
# cat >/etc/modprobe.d/kvm-intel.conf <<EOF
options kvm_intel ept=n
EOF
5. Reload the kernel module and verify ept state:
# modprobe -r kvm_intel
# modprobe  kvm_intel
# cat /sys/module/kvm_intel/parameters/ept
N
6. Populate and start 100 VMs on Hypervisor and rerun Load Test.
7. Save monitored statistics
8, Analyse results.

Test showed us that we have opposite effect when change ETP to N.
The better result we reached while EPT is enabled in KVM.

Metrics name	                Module name	        EPT = Y	 EPT = N
CPU Utilization Server (%)	Engine node - System	4.42	 4.06
	                        Engine node - User	12.95	 6.8
	                        Host – System	        7.38	 11.78
	                        Host – User	        3.38	 3.8
CPU Utilization Process (%)	Engine node – Ovirt	99.83	 46.18
	                        Host node – LIbvirt	19.64	 28.04
	                        Host node – VDSM	28.39	 30.44
Memory Util Server (Mb)	        Engine node - Used	27681.43 28608.76
	                        Host - Used	        35155.52 34572.57
Memory Util Process (Mb)	Engine node – Ovirt	6904.62	6984.05
	                        Host node – LIbvirt	32.36	32.34
	                        Host node – VDSM	542.48	665.1
Network Utilization (Mb)	Engine node – Recv	3.71	2.28
	                        Engine node – Sent	3.96	2.41
	                        Host – Recv	        57.02	50.21
	                        Host – Sent	        1.89	2.21
Disk Utilization (b/sec)	Engine node – Reads	0	0
	                        Engine node – Writes	131.33	113.94
	                        Host – Reads	        0.02	0.01
	                        Host – Writes	        211.16	326.43

See attached detailed document
Comment 6 Yuri Obshansky 2016-03-03 06:46 EST
Created attachment 1132731 [details]
Test result
Comment 7 Eyal Edri 2016-03-14 08:49:05 EDT
RHEV 3.6 GA was released.
This bug didn't have any required doc text, closing.

Note You need to log in before you can comment on or make changes to this bug.