Bug 1779821

Summary: sap-hana tuned profile needs to change for RHEL8 to take advantage of scheduler changes
Product: Red Hat Enterprise Linux 8 Reporter: djdumas
Component: tunedAssignee: Jaroslav Škarvada <jskarvad>
Status: CLOSED ERRATA QA Contact: Robin Hack <rhack>
Severity: high Docs Contact:
Priority: unspecified    
Version: 8.2CC: cpelland, djdumas, dshaks, fdanapfe, hannsj_uhl, jeder, jskarvad, olysonek, psklenar, rhack, toneata, wchadwic
Target Milestone: rcKeywords: Patch, ZStream
Target Release: 8.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: tuned-2.13.0-1.el8 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1780679 1780682 (view as bug list) Environment:
Last Closed: 2020-04-28 16:59:38 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1780679, 1780682    

Description djdumas 2019-12-04 19:27:03 UTC
Description of problem: Value of parameters set in throughput-performance profile are no longer appropriate for best performance of sap-hana.  While working on a certification performance issue, the following settings were found to improve performance.

Kernel.sched_min_granularity_ns from 10000000 to 3000000
Kernel.sched_wakeup_granularity_ns from 15000000 to 4000000

The 10000000 / 15000000 settings are currently inherited from throughput-performance.

We also found that readahead parameter was better left at the default of 128K rather than the 4096K that is set in throughput-performance.

One performance investigation is outlined in https://bugzilla.redhat.com/show_bug.cgi?id=1745111 for Power9,  another example is in https://bugzilla.redhat.com/show_bug.cgi?id=1753205 for x86_64.


Version-Release number of selected component (if applicable):
tuned-2.13.0-0.1.rc1.el8.noarch

How reproducible:
Every time

Steps to Reproduce:
1. Load SAP workload fsperf and scsi based I/O
2. Run 4K random overwrite test - once with lower sched_ settings, change to higher settings and run again
3. Compare results

Actual results:  Results don't meet SAP KPIs with higher (throughput-performance based) sched_ settings

Expected results:  Results meet and exceed SAP KPIs with lower sched_settings


Additional info:  Recommending no longer inheriting throughput-performance profile and putting required settings directly in sap-hana profile

Comment 2 John Shakshober 2019-12-06 14:39:17 UTC
This is a RHEL8.2 blocker request as it is gating an SAP cert.  The testing for this is also being requested for the low_latency tuned in 8.2
Its been tested very well on by both SAP workloads and with the Perf team / kernel QE perf team.

Comment 5 Jaroslav Škarvada 2019-12-06 15:47:01 UTC
https://github.com/redhat-performance/tuned/pull/231

Comment 7 djdumas 2019-12-06 16:28:30 UTC
Hi Jaroslav,

There are a few other settings that we do need to carry over from throughput-performance profile into the new sap-hana profile.  The [cpu] settings are necessary from a performance perspective.
The vm. parameters have been set for all the tests that we have done so would like to include them as well.


[cpu]
governor=performance
energy_perf_bias=performance
min_perf_pct=100

[sysctl]
vm.dirty_ratio = 40
vm.dirty_background_ratio = 10
vm.swappiness=10


Thanks!
Dave

Comment 8 Jaroslav Škarvada 2019-12-06 16:40:56 UTC
(In reply to djdumas from comment #7)
> Hi Jaroslav,
> 
> There are a few other settings that we do need to carry over from
> throughput-performance profile into the new sap-hana profile.  The [cpu]
> settings are necessary from a performance perspective.
> The vm. parameters have been set for all the tests that we have done so
> would like to include them as well.
> 
> 
> [cpu]
> governor=performance
> energy_perf_bias=performance
> min_perf_pct=100
> 
> [sysctl]
> vm.dirty_ratio = 40
> vm.dirty_background_ratio = 10
> vm.swappiness=10
> 
> 
> Thanks!
> Dave

Hi Dave,

please check the PR from the comment 5 (direct link: https://github.com/redhat-performance/tuned/pull/231/commits/52156a00b590ef401b38dcc65720b344d67e86f6) whether it's OK for you. It's the single tuning for all architectures.

It's simple profile, so I directly copied the settings from the throughput-performance profile to the sap-hana not to complicate things, but if the following works better for you, I could change it, e.g.:
- set readahead explicitly to the 128kB (now it's left on its defaults)
- keep throughput-performance included, and just override what's needed in sap-hana

Jaroslav

Comment 9 djdumas 2019-12-06 19:01:09 UTC
(In reply to Jaroslav Škarvada from comment #8)
> 
> Hi Dave,
> 
> please check the PR from the comment 5 (direct link:
> https://github.com/redhat-performance/tuned/pull/231/commits/
> 52156a00b590ef401b38dcc65720b344d67e86f6) whether it's OK for you. It's the
> single tuning for all architectures.

Hi Jaroslav,
Yes, this is exactly what we were looking for - sorry I didn't even notice that in the bugzilla until you pointed it out.
Thanks!

Comment 15 errata-xmlrpc 2020-04-28 16:59:38 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:1883