Bug 1459146

Summary: [RFE] Have "tuned-adm recommend" match OCP 3.5 installed master/node profile
Product: Red Hat Enterprise Linux 7 Reporter: Thom Carlin <tcarlin>
Component: tunedAssignee: Jaroslav Škarvada <jskarvad>
Status: CLOSED ERRATA QA Contact: Johnny Liu <jialiu>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 7.5CC: jeder, jialiu, jmencak, jskarvad, mifiedle, olysonek, psklenar, salmy, sdodson, tcarlin, tcerna, xtian
Target Milestone: rcKeywords: FutureFeature, Patch, Upstream
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
URL: aos-scalability-36
Whiteboard:
Fixed In Version: tuned-2.9.0-0.1.rc1.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1504475 (view as bug list) Environment:
Last Closed: 2018-04-10 16:04:16 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1724792, 1465887, 1467576, 1485946, 1504475    

Description Thom Carlin 2017-06-06 12:18:21 UTC
Description of enhancement:

For a OCP 3.5 master/node running virtualized, enhance "tuned-adm recommend" to match the default installed by OCP "atomic-openshift-node-guest" instead of "virtual-guest"

Version-Release number of selected component (if applicable):

2.7.1-3.el7_3.2

Additional info:

# tuned-adm recommend
virtual-guest

# tuned-adm list
Available profiles:
- atomic-openshift-node-guest
- atomic-openshift-node-host  - Optimize bare metal systems running the OpenShift
- balanced                    - General non-specialized tuned profile
- desktop                     - Optmize for the desktop use-case
- latency-performance         - Optimize for deterministic performance at the cost of increased power consumption
- network-latency             - Optimize for deterministic performance at the cost of increased power consumption, focused on low latency network performance
- network-throughput          - Optimize for streaming network throughput.  Generally only necessary on older CPUs or 40G+ networks.
- powersave                   - Optimize for low power consumption
- throughput-performance      - Broadly applicable tuning that provides excellent performance across a variety of common server workloads.  This is the default profile for RHEL7.
- virtual-guest               - Optimize for running inside a virtual guest.
- virtual-host                - Optimize for running KVM guests
Current active profile: atomic-openshift-node-guest

# cat /etc/system-release-cpe
cpe:/o:redhat:enterprise_linux:7.3:ga:server

# virt-what
kvm

OCP master/node systems were RHEL 3.5 running on RHV.

Comment 4 Jaroslav Škarvada 2017-06-06 12:49:55 UTC
Currently upstream recommend.conf contains preferences only for upstream profiles, but atomic-openshift-node-guest is not in upstream. I can see two possible solutions:

a) merge atomic-openshift-node-guest into Tuned upstream and add its record into the upstream recommend.conf. In such case we need to find a way how to reliable detect OCP 3.5 (the /etc/system-release-cpe doesn't seem to be helpful), or

b) pack /etc/tuned/recommend.conf into the package providing atomic-openshift-node-guest. In such case it's not Tuned bug, but such approach doesn't scale well, because multiple 3rd party packages cannot install multiple /etc/tuned/recommend.conf. In Tuned we could address it by e.g. providing /etc/tuned/recommend.d directory where multiple 3rd party packages could install their recommend.conf

Comment 8 Scott Dodson 2017-06-06 15:46:56 UTC
To be clear, when tuned-profiles-atomic-openshift-node is installed it does congifure the desired profile using scripts in its pre/post config. I understand it's desirable for `tuned-adm recommend` to provide the same recommendation as the pre/post scripts configure.

Aside from making recommend work, is there something that's causing the active profile to be reset?

Here's the scripts btw :
# rpm -q --scripts tuned-profiles-atomic-openshift-node 
postinstall scriptlet (using /bin/sh):
recommended=`/usr/sbin/tuned-adm recommend`
if [[ "${recommended}" =~ guest ]] ; then
  /usr/sbin/tuned-adm profile atomic-openshift-node-guest > /dev/null 2>&1
else
  /usr/sbin/tuned-adm profile atomic-openshift-node-host > /dev/null 2>&1
fi

preuninstall scriptlet (using /bin/sh):
# reset the tuned profile to the recommended profile
# $1 = 0 when we're being removed > 0 during upgrades
if [ "$1" = 0 ]; then
  recommended=`/usr/sbin/tuned-adm recommend`
  /usr/sbin/tuned-adm profile $recommended > /dev/null 2>&1
fi

Comment 9 Thom Carlin 2017-06-06 16:03:03 UTC
It seems that https://bugzilla.redhat.com/show_bug.cgi?id=1459146#c0 did not describe the situation as clearly as it could:
1) OCP 3.5 master and nodes are running in "thick" mode in RHEL 7.3
2) Master/nodes are VMs on RHV 4.0 hypervisors
3) The active profile is *not* reset -- it is "atomic-openshift-node-guest"
   a) virt-what returns "kvm"
   b) /etc/system-release-cpe contains "cpe:/o:redhat:enterprise_linux:7.3:ga:server"
   c) This causes "tuned-adm recommend" to return "virtual-guest"
4) As https://bugzilla.redhat.com/show_bug.cgi?id=1459146#c8 shows, the installer uses atomic-openshift-node-guest (due to 3).  I did verify the scripts matched the listing above.

The tuned-adm profile has not been changed from the original installation.

Is atomic-openshift-node-host the expected profile for this situation?

Comment 10 Scott Dodson 2017-06-06 16:09:54 UTC
atomic-openshift-node-guest is the profile I'd expect for this host, if it were bare metal then it'd be atomic-openshift-node-host

Comment 11 Jaroslav Škarvada 2017-06-06 16:58:52 UTC
(In reply to Scott Dodson from comment #8)
> Aside from making recommend work, is there something that's causing the
> active profile to be reset?
>
I am not sure whether I understand the question correctly, but if /etc/tuned/active_profile doesn't exist or is empty when Tuned is started, it sets the same profile as 'tuned-adm recommend' prints .

Comment 12 Jaroslav Škarvada 2017-06-06 17:08:20 UTC
(In reply to Thom Carlin from comment #9)
> It seems that https://bugzilla.redhat.com/show_bug.cgi?id=1459146#c0 did not
> describe the situation as clearly as it could:
> 1) OCP 3.5 master and nodes are running in "thick" mode in RHEL 7.3
> 2) Master/nodes are VMs on RHV 4.0 hypervisors
> 3) The active profile is *not* reset -- it is "atomic-openshift-node-guest"
>    a) virt-what returns "kvm"
>    b) /etc/system-release-cpe contains
> "cpe:/o:redhat:enterprise_linux:7.3:ga:server"
>    c) This causes "tuned-adm recommend" to return "virtual-guest"
> 4) As https://bugzilla.redhat.com/show_bug.cgi?id=1459146#c8 shows, the
> installer uses atomic-openshift-node-guest (due to 3).  I did verify the
> scripts matched the listing above.
> 

Unfortunately the "fingerprint" (virt-what and /etc/system-release-cpe) doesn't seem to be enough to deduce that it's OCP and that atomic-openshift-node* should be used. Is there anything more which could be used for unique identification (across other products)?

If not, it also shouldn't be problem, we could use e.g. the following for /etc/tuned/recommend.d/ocp.conf (when implemented)

[atomic-openshift-node-guest]
virt=.+

[atomic-openshift-node-host]

I.e. if tuned-profiles-atomic-openshift-node is installed, it will set atomic-openshift-node-host on bare metal and atomic-openshift-node-guest otherwise. I.e. the tuned-profiles-atomic-openshift-node specific recommend.conf (/etc/tuned/recommend.d/ocp.conf in this case) will override the base recommend.conf.

Comment 17 Jaroslav Škarvada 2017-08-09 14:16:30 UTC
The Tuned side of this feature has been merged upstream:
https://github.com/redhat-performance/tuned/pull/60

Rules can be installed into:
/usr/lib/tuned/recommend.d/
or
/etc/tuned/recommend.d/

For packages installed from Red Hat controlled repositories it's preferred to drop rules into:
/usr/lib/tuned/recommend.d/

The /etc/tuned/recommend.d/ can be used for override and is meant for third party packages or admin overrides.

Comment 25 Fedora Update System 2017-10-13 14:19:48 UTC
tuned-2.9.0-0.1.rc1.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2017-d9c6b990df

Comment 26 Fedora Update System 2017-10-13 22:24:43 UTC
tuned-2.9.0-0.1.rc1.fc26 has been pushed to the Fedora 26 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-d9c6b990df

Comment 27 Fedora Update System 2017-10-13 23:24:33 UTC
tuned-2.9.0-0.1.rc1.fc27 has been pushed to the Fedora 27 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-5f0849d207

Comment 34 Johnny Liu 2017-10-20 02:38:42 UTC
/usr/lib/tuned/recommend.d is already installed by tuned-2.9.0-0.1.rc1.el7.noarch.

About the left issues described in https://bugzilla.redhat.com/show_bug.cgi?id=1459146#c28, I open clone this bug to openshift product to track it.

Comment 35 Fedora Update System 2017-10-29 21:06:04 UTC
tuned-2.9.0-1.fc27 has been submitted as an update to Fedora 27. https://bodhi.fedoraproject.org/updates/FEDORA-2017-0e45ce4685

Comment 36 Fedora Update System 2017-10-29 21:12:14 UTC
tuned-2.9.0-1.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2017-c30e9bd1ea

Comment 39 errata-xmlrpc 2018-04-10 16:04:16 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:0879