Bug 1898933

Description of problem:

Currently, the "protectKernelDefaults" flag in the kubelet is left unset. This flag by default is "false" which enables the kubelet to change sysctl's to what it needs. These sysctl's are not many. Namely, they are these:

```
kernel.keys.root_maxbytes=25000000
kernel.keys.root_maxkeys=1000000
kernel.panic=10
kernel.panic_on_oops=1
vm.overcommit_memory=1
vm.panic_on_oom=0
```

While this works just fine, the CIS benchmark's control 4.2.6 recommends setting this to true.

This means that the Kubelet will expect the values to be set already set beforehand, else, the kubelet will error out.

Ideally, we should:

* Include these sysctl's as defaults in RHCOS (via the hyperkube rpm, maybe?)
* Set `protectKernelDefaults` to `true` in the default kubelet configuration
* Ensure that the kubelet is not able to set sysctl's in general (by setting `ProtectKernelTunables=true` in the kubelet's systemd unit... thought I'm not sure if this is currently possible)

This would improve our security by limiting one attack vector in case of a kubelet compromise.


Version-Release number of selected component (if applicable):

* All

How reproducible:

* Always

Steps to Reproduce:
1. Deploy an OpenShift cluster
2. Look at /etc/kubernetes/kubelet.conf on any host

Actual results:

$ oc debug node/$NODE -- grep protectKernelDefaults /host/etc/kubernetes/kubelet.conf
Creating debug namespace/openshift-debug-node-jkmvf ...
Starting pod/ip-10-0-128-17ec2internal-debug ...
To use host binaries, run `chroot /host`

Removing debug pod ...
Removing debug namespace/openshift-debug-node-jkmvf ...
error: non-zero exit code from debug container


Expected results:
$ oc debug node/$NODE -- grep protectKernelDefaults /host/etc/kubernetes/kubelet.conf
Creating debug namespace/openshift-debug-node-ltvnd ...
Starting pod/ip-10-0-137-231ec2internal-debug ...
To use host binaries, run `chroot /host`
  "protectKernelDefaults": true,

Removing debug pod ...
Removing debug namespace/openshift-debug-node-ltvnd ...

Additional info:

Currently, it takes a two-step process to be able to set this parameter:

* First: Create a MachineConfig object that sets the sysctl's (Wait for the pools to update)
* Second: Create a Kubeletconfig object that sets the parameter

It two steps because in cases of applying this as a day-2 operation, the kubelet runs before the MachineConfigDaemon, and so, the Kubelet will fail to start since it won't be able to set the sysctl's... and these wouldn't be set yet by the MCD.

I tried to document the whole process here: https://jaosorior.dev/2020/protectkerneldefaults-in-openshift/