Bug 1826167

Summary:	Sysctl parameters set by tuned can not be overwritten by parameters set via /etc/sysctl{.conf,.d}
Product:	OpenShift Container Platform	Reporter:	Jiří Mencák <jmencak>
Component:	Node Tuning Operator	Assignee:	Jiří Mencák <jmencak>
Status:	CLOSED ERRATA	QA Contact:	Simon <skordas>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	4.3.z	CC:	rhowe, sejug, skordas
Target Milestone:	---
Target Release:	4.3.z
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:	Cause: Tuned pods did not mount /etc/sysctl.{conf,d/} from the host. Consequence: Settings provided by the host can be overriden by tuned profiles even though this is no longer the standard behaviour of tuned as shipped in RHEL 7.3. Fix: Mount /etc/sysctl.{conf,d/} from the host in tuned pods. Result: Tuned profiles no longer override the host sysctl settings in /etc/sysctl.{conf,d/}.	Story Points:	---
Clone Of:	1825375	Environment:
Last Closed:	2020-05-11 21:20:39 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1825375
Bug Blocks:

Comment 3 Simon 2020-04-27 20:05:16 UTC

Verification positive!

: oc project openshift-cluster-node-tuning-operator
Already on project "openshift-cluster-node-tuning-operator" on server "https://api.skordas-427-nto.perf-testing.devcluster.openshift.com:6443".

: worker1=$(oc get nodes --no-headers | awk '/worker/ {print $1}' | head -n1)

: worker1_tuned=$(oc get pods -o wide --no-headers | grep $worker1 | cut -d' ' -f1)

: echo "chroot /host /bin/bash -c 'cat /etc/sysctl.d/inotify.conf'" | oc debug node/$worker1
Starting pod/ip-10-0-134-50us-east-2computeinternal-debug ...
To use host binaries, run `chroot /host`
Pod IP: 10.0.134.50
If you don't see a command prompt, try pressing enter.

fs.inotify.max_user_watches = 65536
fs.inotify.max_user_instances = 8192

Removing debug pod ...

: oc exec $worker1_tuned -- mount | grep /etc/sysctl
/dev/mapper/coreos-luks-root-nocrypt on /etc/sysctl.d type xfs (ro,relatime,seclabel,attr2,inode64,prjquota)
/dev/mapper/coreos-luks-root-nocrypt on /etc/sysctl.conf type xfs (ro,relatime,seclabel,attr2,inode64,prjquota)

: echo "chroot /host /bin/bash -c 'sysctl kernel.pid_max'" | oc debug node/$worker1
Starting pod/ip-10-0-134-50us-east-2computeinternal-debug ...
To use host binaries, run `chroot /host`
Pod IP: 10.0.134.50
If you don't see a command prompt, try pressing enter.
kernel.pid_max = 4194304

Removing debug pod ...

: touch override.yaml

: vim override.yaml 

: oc create -f override.yaml 
tuned.tuned.openshift.io/override created

: oc label node $worker1 tuned.openshift.io/override=
node/ip-10-0-134-50.us-east-2.compute.internal labeled

: oc label node $worker1 tuned.openshift.io/override=
node/ip-10-0-134-50.us-east-2.compute.internal labeled

: oc logs $worker1_tuned | tail -n9
I0427 20:02:25.438760   14755 openshift-tuned.go:337] Sending HUP to PID 14992
2020-04-27 20:02:25,438 INFO     tuned.daemon.daemon: stopping tuning
2020-04-27 20:02:26,054 INFO     tuned.daemon.daemon: terminating Tuned, rolling back all changes
2020-04-27 20:02:26,056 INFO     tuned.daemon.daemon: Running in automatic mode, checking what profile is recommended for your configuration.
2020-04-27 20:02:26,057 INFO     tuned.daemon.daemon: Using 'override' profile
2020-04-27 20:02:26,057 INFO     tuned.profiles.loader: loading profile: override
2020-04-27 20:02:26,057 INFO     tuned.daemon.daemon: starting tuning
2020-04-27 20:02:26,058 INFO     tuned.plugins.plugin_sysctl: reapplying system sysctl
2020-04-27 20:02:26,059 INFO     tuned.daemon.daemon: static tuning from profile 'override' applied

: echo "chroot /host /bin/bash -c 'sysctl fs.inotify.max_user_instances'" | oc debug node/$worker1
Starting pod/ip-10-0-134-50us-east-2computeinternal-debug ...
To use host binaries, run `chroot /host`
Pod IP: 10.0.134.50
If you don't see a command prompt, try pressing enter.
fs.inotify.max_user_instances = 8192

Removing debug pod ...

: echo "chroot /host /bin/bash -c 'sysctl fs.inotify.max_user_watches'" | oc debug node/$worker1
Starting pod/ip-10-0-134-50us-east-2computeinternal-debug ...
To use host binaries, run `chroot /host`
Pod IP: 10.0.134.50
If you don't see a command prompt, try pressing enter.
fs.inotify.max_user_watches = 65536

Removing debug pod ...

: echo "chroot /host /bin/bash -c 'sysctl kernel.pid_max'" | oc debug node/$worker1
Starting pod/ip-10-0-134-50us-east-2computeinternal-debug ...
To use host binaries, run `chroot /host`
Pod IP: 10.0.134.50
If you don't see a command prompt, try pressing enter.
kernel.pid_max = 1048576

Removing debug pod ...

: oc get clusterversions.config.openshift.io 
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.3.0-0.nightly-2020-04-27-081123   True        False         100m    Cluster version is 4.3.0-0.nightly-2020-04-27-081123

Comment 5 errata-xmlrpc 2020-05-11 21:20:39 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:2006