Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1917442

Summary: Unable to set protectKernelDefaults
Product: OpenShift Container Platform Reporter: Juan Antonio Osorio <josorior>
Component: Node Tuning OperatorAssignee: Jiří Mencák <jmencak>
Status: CLOSED NOTABUG QA Contact: Mike Fiedler <mifiedle>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 4.7CC: sejug
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-01-18 15:13:19 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Juan Antonio Osorio 2021-01-18 13:50:25 UTC
Description of problem:

Setting the `protectKernelDefaults` kubelet argument as adviced by the CIS benchmark is non-trivial.

Initially, I tried to do this with a MachineConfig-based two-step approach that I documented here: https://jaosorior.dev/2020/protectkerneldefaults-in-openshift/

However, I was recently adviced that this should really be done with the Node Tuning Operator instead.

The idea being that I would create a profile that would set the relevant sysctls that the kubelet needs to start. I would then set the kubelet parameter with a `KubeletConfig` object and it "should work"


Version-Release number of selected component (if applicable):
4.7 & 4.6

Steps to Reproduce:
1. Spawn a cluster and get credentials for it
2. Create the following objects:
---
apiVersion: tuned.openshift.io/v1
kind: Tuned
metadata:
  name: kubelet-defaults
  namespace: openshift-cluster-node-tuning-operator
spec:
  profile:
  - data: |
      [main]
      summary=Defaults for running OpenShift nodes

      [sysctl]
      kernel.keys.root_maxbytes=25000000
      kernel.keys.root_maxkeys=1000000
      kernel.panic=10
      kernel.panic_on_oops=1
      vm.overcommit_memory=1
      vm.panic_on_oom=0
    name: openshift-kubelet-defaults
  recommend:
  - match:
    - label: node-role.kubernetes.io/master
    - label: node-role.kubernetes.io/worker
    - label: node-role.kubernetes.io/infra
    operand:
      debug: false
    priority: 75
    profile: openshift-kubelet-defaults
---
apiVersion: machineconfiguration.openshift.io/v1
kind: KubeletConfig
metadata:
  name: master-protect-kernel-defaults
spec:
  machineConfigPoolSelector:
    matchLabels:
      pools.operator.machineconfiguration.openshift.io/master: ""
  kubeletConfig:
    protectKernelDefaults: true
---
apiVersion: machineconfiguration.openshift.io/v1
kind: KubeletConfig
metadata:
  name: worker-protect-kernel-defaults
spec:
  machineConfigPoolSelector:
    matchLabels:
      pools.operator.machineconfiguration.openshift.io/worker: ""
  kubeletConfig:
    protectKernelDefaults: true

Actual results:
The MachineConfigOperator start rolling in the changes but the nodes never come up (the kubelet crashes as the sysctl's haven't been set by the operator as they should).

Expected results:
The nodes come up with the relevant parameters set.

Additional info:
If this is expected... then we should document the two-step approach that I mentioned in the blog-post.

Comment 1 Jiří Mencák 2021-01-18 15:13:19 UTC
Thank you for your report, however, this is not NTO's bug.  NTO is responsible for wrapping around and enabling the Tuned daemon
to run in the OpenShift environment and achieve as much compatibility with its functionality as possible.

Your report is a feature request and those are handled by Jira cards.  However, based on Ryan's comment in
https://bugzilla.redhat.com/show_bug.cgi?id=1898933#c3 , #c6
and looking into this a bit more, we believe there is not much to gain in terms of security by implementing this
as you'll still have to use CAP_SYS_ADMIN by the kubelet and a lot to lose in terms of complexity and potential issues
already pointed out in 1898933.