Bug 1826167 - Sysctl parameters set by tuned can not be overwritten by parameters set via /etc/sysctl{.conf,.d}
Summary: Sysctl parameters set by tuned can not be overwritten by parameters set via /...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Node Tuning Operator
Version: 4.3.z
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: 4.3.z
Assignee: Jiří Mencák
QA Contact: Simon
URL:
Whiteboard:
Depends On: 1825375
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-04-21 07:09 UTC by Jiří Mencák
Modified: 2020-05-11 21:20 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: Tuned pods did not mount /etc/sysctl.{conf,d/} from the host. Consequence: Settings provided by the host can be overriden by tuned profiles even though this is no longer the standard behaviour of tuned as shipped in RHEL 7.3. Fix: Mount /etc/sysctl.{conf,d/} from the host in tuned pods. Result: Tuned profiles no longer override the host sysctl settings in /etc/sysctl.{conf,d/}.
Clone Of: 1825375
Environment:
Last Closed: 2020-05-11 21:20:39 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift cluster-node-tuning-operator pull 128 0 None closed Bug 1826167: Mount host's /etc/sysctl.{conf,d/} to enable sysctl override from the host. 2020-09-29 23:50:39 UTC
Red Hat Product Errata RHBA-2020:2006 0 None None None 2020-05-11 21:20:51 UTC

Comment 3 Simon 2020-04-27 20:05:16 UTC
Verification positive!

: oc project openshift-cluster-node-tuning-operator
Already on project "openshift-cluster-node-tuning-operator" on server "https://api.skordas-427-nto.perf-testing.devcluster.openshift.com:6443".

: worker1=$(oc get nodes --no-headers | awk '/worker/ {print $1}' | head -n1)

: worker1_tuned=$(oc get pods -o wide --no-headers | grep $worker1 | cut -d' ' -f1)

: echo "chroot /host /bin/bash -c 'cat /etc/sysctl.d/inotify.conf'" | oc debug node/$worker1
Starting pod/ip-10-0-134-50us-east-2computeinternal-debug ...
To use host binaries, run `chroot /host`
Pod IP: 10.0.134.50
If you don't see a command prompt, try pressing enter.

fs.inotify.max_user_watches = 65536
fs.inotify.max_user_instances = 8192

Removing debug pod ...

: oc exec $worker1_tuned -- mount | grep /etc/sysctl
/dev/mapper/coreos-luks-root-nocrypt on /etc/sysctl.d type xfs (ro,relatime,seclabel,attr2,inode64,prjquota)
/dev/mapper/coreos-luks-root-nocrypt on /etc/sysctl.conf type xfs (ro,relatime,seclabel,attr2,inode64,prjquota)

: echo "chroot /host /bin/bash -c 'sysctl kernel.pid_max'" | oc debug node/$worker1
Starting pod/ip-10-0-134-50us-east-2computeinternal-debug ...
To use host binaries, run `chroot /host`
Pod IP: 10.0.134.50
If you don't see a command prompt, try pressing enter.
kernel.pid_max = 4194304

Removing debug pod ...

: touch override.yaml

: vim override.yaml 

: oc create -f override.yaml 
tuned.tuned.openshift.io/override created

: oc label node $worker1 tuned.openshift.io/override=
node/ip-10-0-134-50.us-east-2.compute.internal labeled

: oc label node $worker1 tuned.openshift.io/override=
node/ip-10-0-134-50.us-east-2.compute.internal labeled

: oc logs $worker1_tuned | tail -n9
I0427 20:02:25.438760   14755 openshift-tuned.go:337] Sending HUP to PID 14992
2020-04-27 20:02:25,438 INFO     tuned.daemon.daemon: stopping tuning
2020-04-27 20:02:26,054 INFO     tuned.daemon.daemon: terminating Tuned, rolling back all changes
2020-04-27 20:02:26,056 INFO     tuned.daemon.daemon: Running in automatic mode, checking what profile is recommended for your configuration.
2020-04-27 20:02:26,057 INFO     tuned.daemon.daemon: Using 'override' profile
2020-04-27 20:02:26,057 INFO     tuned.profiles.loader: loading profile: override
2020-04-27 20:02:26,057 INFO     tuned.daemon.daemon: starting tuning
2020-04-27 20:02:26,058 INFO     tuned.plugins.plugin_sysctl: reapplying system sysctl
2020-04-27 20:02:26,059 INFO     tuned.daemon.daemon: static tuning from profile 'override' applied

: echo "chroot /host /bin/bash -c 'sysctl fs.inotify.max_user_instances'" | oc debug node/$worker1
Starting pod/ip-10-0-134-50us-east-2computeinternal-debug ...
To use host binaries, run `chroot /host`
Pod IP: 10.0.134.50
If you don't see a command prompt, try pressing enter.
fs.inotify.max_user_instances = 8192

Removing debug pod ...

: echo "chroot /host /bin/bash -c 'sysctl fs.inotify.max_user_watches'" | oc debug node/$worker1
Starting pod/ip-10-0-134-50us-east-2computeinternal-debug ...
To use host binaries, run `chroot /host`
Pod IP: 10.0.134.50
If you don't see a command prompt, try pressing enter.
fs.inotify.max_user_watches = 65536

Removing debug pod ...

: echo "chroot /host /bin/bash -c 'sysctl kernel.pid_max'" | oc debug node/$worker1
Starting pod/ip-10-0-134-50us-east-2computeinternal-debug ...
To use host binaries, run `chroot /host`
Pod IP: 10.0.134.50
If you don't see a command prompt, try pressing enter.
kernel.pid_max = 1048576

Removing debug pod ...

: oc get clusterversions.config.openshift.io 
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.3.0-0.nightly-2020-04-27-081123   True        False         100m    Cluster version is 4.3.0-0.nightly-2020-04-27-081123

Comment 5 errata-xmlrpc 2020-05-11 21:20:39 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:2006


Note You need to log in before you can comment on or make changes to this bug.