Bug 1972559

Sorry for the delay in replying. I think this is just a matter of tailoring the profile to your environment, although I agree that the documentation and exposing the variables is currently not the best.

I'm working on testing a tailored profile that would match your customer's kubelet config.

Sorry again for the late reply. Using the customer's kubelet config, this tailoring file worked for me:

apiVersion: compliance.openshift.io/v1alpha1
kind: TailoredProfile
metadata:
  name: cis-node-bz
spec:
  extends: ocp4-moderate-node
  title: CIS node tailored for BZ-1972559
  setValues:
    # evictionHard
    - name: ocp4-var-kubelet-evictionhard-imagefs-available
      rationale: evictionHard.imagefs.available = 15%
      value: 15%
    - name: ocp4-var-kubelet-evictionhard-nodefs-inodesfree
      rationale: evictionHard.imagefs.inodesFree = 5%
      value: 5%
    - name: ocp4-var-kubelet-evictionhard-nodefs-available
      rationale: evictionHard.imagefs.nodefs.available = 10%
      value: 10%
    - name: ocp4-var-kubelet-evictionhard-memory-available
      rationale: evictionHard.memory.available = 100Mi
      value: 100Mi
    # evictionSoft
    - name: ocp4-var-kubelet-evictionsoft-imagefs-available
      rationale: evictionSoft.imagefs.available = 15%
      value: 15%
    - name: ocp4-var-kubelet-evictionsoft-nodefs-inodesfree
      rationale: evictionSoft.imagefs.inodesFree = 5%
      value: 5%
    - name: ocp4-var-kubelet-evictionsoft-nodefs-available
      rationale: evictionSoft.imagefs.nodefs.available = 10%
      value: 10%
    - name: ocp4-var-kubelet-evictionsoft-memory-available
      rationale: evictionSoft.memory.available = 100Mi
      value: 100Mi
  disableRules:
    - name: ocp4-kubelet-eviction-thresholds-set-hard-imagefs-inodesfree
      rationale: The customer's kubelet doesn't seem to set this
    - name: ocp4-kubelet-eviction-thresholds-set-soft-imagefs-inodesfree
      rationale: The customer's kubelet doesn't seem to set this

As you can see, the tailorig file sets the values of the variables to the values that the customer is using. I then created the TailoredProfile out of the file:
$ oc apply -f tailoring/tailored-cis-node-bz.yam
which created a tailored profile:
$ oc get tailoredprofile
NAME          STATE
cis-node-bz   READY

And then I used that in a ScanSettingBinding:
$ cat tailoring/bindings-cis-node-bz.yaml
apiVersion: compliance.openshift.io/v1alpha1
kind: ScanSettingBinding
metadata:
  name: cis-bz
  namespace: openshift-compliance
profiles:
  # Node checks
  - name: cis-node-bz
    kind: TailoredProfile
    apiGroup: compliance.openshift.io/v1alpha1
  # Cluster checks
  - name: ocp4-cis
    kind: Profile
    apiGroup: compliance.openshift.io/v1alpha1
settingsRef:
  name: default
  kind: ScanSetting
  apiGroup: compliance.openshift.io/v1alpha1

Using this file, all the checks for the kubelet parameters passed using the customer's kubelet:
oc get ccr -l 'compliance.openshift.io/scan-name=cis-node-bz-worker' | grep kubelet
cis-node-bz-worker-file-groupowner-kubelet-conf                             PASS             medium
cis-node-bz-worker-file-owner-kubelet-conf                                  PASS             medium
cis-node-bz-worker-file-permissions-kubelet-conf                            PASS             medium
cis-node-bz-worker-kubelet-anonymous-auth                                   PASS             medium
cis-node-bz-worker-kubelet-authorization-mode                               PASS             medium
cis-node-bz-worker-kubelet-configure-client-ca                              PASS             medium
cis-node-bz-worker-kubelet-configure-event-creation                         PASS             medium
cis-node-bz-worker-kubelet-configure-tls-cipher-suites                      PASS             medium
cis-node-bz-worker-kubelet-disable-hostname-override                        PASS             medium
cis-node-bz-worker-kubelet-enable-cert-rotation                             PASS             medium
cis-node-bz-worker-kubelet-enable-client-cert-rotation                      PASS             medium
cis-node-bz-worker-kubelet-enable-iptables-util-chains                      PASS             medium
cis-node-bz-worker-kubelet-enable-protect-kernel-defaults                   FAIL             medium
cis-node-bz-worker-kubelet-enable-server-cert-rotation                      PASS             medium
cis-node-bz-worker-kubelet-enable-streaming-connections                     PASS             medium
cis-node-bz-worker-kubelet-eviction-thresholds-set-hard-imagefs-available   PASS             medium
cis-node-bz-worker-kubelet-eviction-thresholds-set-hard-memory-available    PASS             medium
cis-node-bz-worker-kubelet-eviction-thresholds-set-hard-nodefs-available    PASS             medium
cis-node-bz-worker-kubelet-eviction-thresholds-set-hard-nodefs-inodesfree   PASS             medium
cis-node-bz-worker-kubelet-eviction-thresholds-set-soft-imagefs-available   PASS             medium
cis-node-bz-worker-kubelet-eviction-thresholds-set-soft-memory-available    PASS             medium
cis-node-bz-worker-kubelet-eviction-thresholds-set-soft-nodefs-available    PASS             medium
cis-node-bz-worker-kubelet-eviction-thresholds-set-soft-nodefs-inodesfree   PASS             medium

except the protect-kernel-defaults one, but I guess if the customer doesn't need to comply with that recommendation, they can also remove that rule from the tailoring.

Some additional notes:

 1) when I was playing with the tailoring, I realized that changing the TailoredProfile CR is not reflected in the configMap that Compliance Operator generates from the TailoredProfile for use by the OpenScap scanner. This is a bug and I'm going to fix it. In the meantime, deleting the Tailoredprofile and recreating is a workaround. Tracked by https://issues.redhat.com/browse/CMP-1008

 2) I realize and acknowledge that the documentation around the variables and the tuning is very poor. Feel free to file a docs bug (and CC me so I can provide some concrent information there and our docs writers don't have to dive too deep). I also filed to follow-up tickets https://issues.redhat.com/browse/CMP-1006 and https://issues.redhat.com/browse/CMP-1007 to make the variables easier to expose and consume. In the meantime, I'm afraid the best way is to just run "oc get variables.compliance" and then "oc describe" the variables to see what is used and what values are available. Also https://github.com/openshift/compliance-operator/blob/master/doc/tutorials/workshop/content/exercises/05-tailoring-profiles.md#disable-rules-in-a-profile might help somewhat.

 3) the protect-kernel-defaults one is quite tricky. Just setting the variable to true would crash kubelet with the default sysctls. The sysctls can be set with a MC file found here: https://github.com/ComplianceAsCode/content/blob/master/ocp-resources/kubelet-sysctls-mc.yaml But nonetheless I would recommend to first test things out with some test cluster as kubelet not coming up in prod can be bad.

One more note, we're also tracking setting the kubelet config in a nicer manner in https://issues.redhat.com/browse/CMP-912

[Bug_Verification]

This looks good. It checks the value in the kubelet threshold checks for all kubelet-eviction rules.

Verified on:
4.8.0-0.nightly-2021-08-23-234834 + compliance-operator.v0.1.39


$ oc get clusterversion
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.8.0-0.nightly-2021-08-23-234834   True        False         6h15m   Cluster version is 4.8.0-0.nightly-2021-08-23-234834

$ oc get csv
NAME                              DISPLAY                            VERSION    REPLACES   PHASE
compliance-operator.v0.1.39       Compliance Operator                0.1.39                Succeeded
elasticsearch-operator.5.1.1-42   OpenShift Elasticsearch Operator   5.1.1-42              Succeeded

$ oc get machineconfigpool  --show-labels
NAME     CONFIG                                             UPDATED   UPDATING   DEGRADED   MACHINECOUNT   READYMACHINECOUNT   UPDATEDMACHINECOUNT   DEGRADEDMACHINECOUNT   AGE     LABELS
master   rendered-master-bd244583a02138b1ea2968089313c2b0   True      False      False      3              3                   3                     0                      5h29m   machineconfiguration.openshift.io/mco-built-in=,operator.machineconfiguration.openshift.io/required-for-upgrade=,pools.operator.machineconfiguration.openshift.io/master=
worker   rendered-worker-0eaa90354bf63581601be7b39a0ba89c   True      False      False      3              3                   3                     0                      5h29m   machineconfiguration.openshift.io/mco-built-in=,pools.operator.machineconfiguration.openshift.io/worker=

$ oc get nodes
NAME                                         STATUS   ROLES    AGE     VERSION
ip-10-0-135-188.us-east-2.compute.internal   Ready    worker   5h40m   v1.21.1+9807387
ip-10-0-148-205.us-east-2.compute.internal   Ready    master   5h46m   v1.21.1+9807387
ip-10-0-177-244.us-east-2.compute.internal   Ready    master   5h46m   v1.21.1+9807387
ip-10-0-181-138.us-east-2.compute.internal   Ready    worker   5h38m   v1.21.1+9807387
ip-10-0-204-255.us-east-2.compute.internal   Ready    worker   5h40m   v1.21.1+9807387
ip-10-0-211-145.us-east-2.compute.internal   Ready    master   5h46m   v1.21.1+9807387

$ oc label machineconfigpool worker cis-hardening=true
machineconfigpool.machineconfiguration.openshift.io/worker labeled

$ oc get machineconfigpool  --show-labels
NAME     CONFIG                                             UPDATED   UPDATING   DEGRADED   MACHINECOUNT   READYMACHINECOUNT   UPDATEDMACHINECOUNT   DEGRADEDMACHINECOUNT   AGE     LABELS
master   rendered-master-bd244583a02138b1ea2968089313c2b0   True      False      False      3              3                   3                     0                      5h50m   machineconfiguration.openshift.io/mco-built-in=,operator.machineconfiguration.openshift.io/required-for-upgrade=,pools.operator.machineconfiguration.openshift.io/master=
worker   rendered-worker-0eaa90354bf63581601be7b39a0ba89c   True      False      False      3              3                   3                     0                      5h50m   cis-hardening=true,machineconfiguration.openshift.io/mco-built-in=,pools.operator.machineconfiguration.openshift.io/worker=

$ oc project default
Now using project "default" on server "https://api.pdhamdhe-2348.qe.devcluster.openshift.com:6443".

$ cat worker-kube-config.yaml 
apiVersion: machineconfiguration.openshift.io/v1
kind: KubeletConfig
metadata:
  name: cis-hardening
spec:
  machineConfigPoolSelector:
    matchLabels:
      cis-hardening: "true"
  kubeletConfig:
    eventRecordQPS: 5
    tlsCipherSuites:
    - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
    - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
    - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
    - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
    protectKernelDefaults: false
    evictionSoftGracePeriod:
      memory.available:  "5m"
      nodefs.available:  "5m"
      nodefs.inodesFree: "5m"
      imagefs.available: "5m"
    evictionHard:
      memory.available:  "100Mi"
      nodefs.available:  "10%"
      nodefs.inodesFree: "5%"
      imagefs.available: "15%"
    evictionSoft:
      memory.available:  "100Mi"
      nodefs.available:  "10%"
      nodefs.inodesFree: "5%"
      imagefs.available: "15%"
      
$ oc create -f worker-kube-config.yaml
kubeletconfig.machineconfiguration.openshift.io/cis-hardening created
      
$ oc get mcp -w
NAME     CONFIG                                             UPDATED   UPDATING   DEGRADED   MACHINECOUNT   READYMACHINECOUNT   UPDATEDMACHINECOUNT   DEGRADEDMACHINECOUNT   AGE
master   rendered-master-bd244583a02138b1ea2968089313c2b0   True      False      False      3              3                   3                     0                      5h55m
worker   rendered-worker-0eaa90354bf63581601be7b39a0ba89c   False     True       False      3              1                   1                     0                      5h55m
worker   rendered-worker-0eaa90354bf63581601be7b39a0ba89c   False     True       False      3              2                   2                     0                      5h56m
worker   rendered-worker-0eaa90354bf63581601be7b39a0ba89c   False     True       False      3              2                   2                     0                      5h56m
worker   rendered-worker-9f03acf618ae1c8d022a0b8cdb2bc204   True      False      False      3              3                   3                     0                      5h58m

$ oc get mcp
NAME     CONFIG                                             UPDATED   UPDATING   DEGRADED   MACHINECOUNT   READYMACHINECOUNT   UPDATEDMACHINECOUNT   DEGRADEDMACHINECOUNT   AGE
master   rendered-master-bd244583a02138b1ea2968089313c2b0   True      False      False      3              3                   3                     0                      6h3m
worker   rendered-worker-9f03acf618ae1c8d022a0b8cdb2bc204   True      False      False      3              3                   3                     0                      6h3m

$ oc get nodes
NAME                                         STATUS   ROLES    AGE     VERSION
ip-10-0-135-188.us-east-2.compute.internal   Ready    worker   5h58m   v1.21.1+9807387
ip-10-0-148-205.us-east-2.compute.internal   Ready    master   6h4m    v1.21.1+9807387
ip-10-0-177-244.us-east-2.compute.internal   Ready    master   6h4m    v1.21.1+9807387
ip-10-0-181-138.us-east-2.compute.internal   Ready    worker   5h56m   v1.21.1+9807387
ip-10-0-204-255.us-east-2.compute.internal   Ready    worker   5h58m   v1.21.1+9807387
ip-10-0-211-145.us-east-2.compute.internal   Ready    master   6h4m    v1.21.1+9807387

$  oc debug -q node/ip-10-0-204-255.us-east-2.compute.internal -- jq -r '.evictionHard."imagefs.available"' /host/etc/kubernetes/kubelet.conf
15%

$ oc project openshift-compliance
Now using project "openshift-compliance" on server "https://api.pdhamdhe-2348.qe.devcluster.openshift.com:6443".

$ oc debug -q node/ip-10-0-204-255.us-east-2.compute.internal
sh-4.4# cat /host/etc/kubernetes/kubelet.conf
{
  "kind": "KubeletConfiguration",
  "apiVersion": "kubelet.config.k8s.io/v1beta1",
  "staticPodPath": "/etc/kubernetes/manifests",
  "syncFrequency": "0s",
  "fileCheckFrequency": "0s",
  "httpCheckFrequency": "0s",
  "tlsCipherSuites": [
    "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256"
  ],
  "tlsMinVersion": "VersionTLS12",
  "rotateCertificates": true,
  "serverTLSBootstrap": true,
  "authentication": {
    "x509": {
      "clientCAFile": "/etc/kubernetes/kubelet-ca.crt"
    },
    "webhook": {
      "cacheTTL": "0s"
    },
    "anonymous": {
      "enabled": false
    }
  },
  "authorization": {
    "webhook": {
      "cacheAuthorizedTTL": "0s",
      "cacheUnauthorizedTTL": "0s"
    }
  },
  "eventRecordQPS": 5,
  "clusterDomain": "cluster.local",
  "clusterDNS": [
    "172.30.0.10"
  ],
  "streamingConnectionIdleTimeout": "0s",
  "nodeStatusUpdateFrequency": "0s",
  "nodeStatusReportFrequency": "0s",
  "imageMinimumGCAge": "0s",
  "volumeStatsAggPeriod": "0s",
  "systemCgroups": "/system.slice",
  "cgroupRoot": "/",
  "cgroupDriver": "systemd",
  "cpuManagerReconcilePeriod": "0s",
  "runtimeRequestTimeout": "0s",
  "maxPods": 250,
  "kubeAPIQPS": 50,
  "kubeAPIBurst": 100,
  "serializeImagePulls": false,
  "evictionHard": {
    "imagefs.available": "15%",
    "memory.available": "100Mi",
    "nodefs.available": "10%",
    "nodefs.inodesFree": "5%"
  },
  "evictionSoft": {
    "imagefs.available": "15%",
    "memory.available": "100Mi",
    "nodefs.available": "10%",
    "nodefs.inodesFree": "5%"
  },
  "evictionSoftGracePeriod": {
    "imagefs.available": "5m",
    "memory.available": "5m",
    "nodefs.available": "5m",
    "nodefs.inodesFree": "5m"
  },
  "evictionPressureTransitionPeriod": "0s",
  "featureGates": {
    "APIPriorityAndFairness": true,
    "DownwardAPIHugePages": true,
    "LegacyNodeRoleBehavior": false,
    "NodeDisruptionExclusion": true,
    "RotateKubeletServerCertificate": true,
    "ServiceNodeExclusion": true,
    "SupportPodPidsLimit": true
  },
  "containerLogMaxSize": "50Mi",
  "systemReserved": {
    "ephemeral-storage": "1Gi"
  },
  "logging": {},
  "shutdownGracePeriod": "0s",
  "shutdownGracePeriodCriticalPods": "0s"
}
sh-4.4# exit

$ oc get pods
NAME                                            READY   STATUS    RESTARTS   AGE
compliance-operator-bb9f644cc-xwfnq             1/1     Running   1          4h14m
ocp4-openshift-compliance-pp-6d7c7db4bd-jwnnq   1/1     Running   0          4h13m
rhcos4-openshift-compliance-pp-c7b548bd-9hqvz   1/1     Running   0          4h13m

$ oc create -f - << EOF
> apiVersion: compliance.openshift.io/v1alpha1
> kind: TailoredProfile
> metadata:
>   name: cis-node-bz
> spec:
>   extends: ocp4-moderate-node
>   title: CIS node tailored for BZ-1972559
>   setValues:
>     # evictionHard
>     - name: ocp4-var-kubelet-evictionhard-imagefs-available
>       rationale: evictionHard.imagefs.available = 15%
>       value: 15%
>     - name: ocp4-var-kubelet-evictionhard-nodefs-inodesfree
>       rationale: evictionHard.imagefs.inodesFree = 5%
>       value: 5%
>     - name: ocp4-var-kubelet-evictionhard-nodefs-available
>       rationale: evictionHard.imagefs.nodefs.available = 10%
>       value: 10%
>     - name: ocp4-var-kubelet-evictionhard-memory-available
>       rationale: evictionHard.memory.available = 100Mi
>       value: 100Mi
>     # evictionSoft
>     - name: ocp4-var-kubelet-evictionsoft-imagefs-available
>       rationale: evictionSoft.imagefs.available = 15%
>       value: 15%
>     - name: ocp4-var-kubelet-evictionsoft-nodefs-inodesfree
>       rationale: evictionSoft.imagefs.inodesFree = 5%
>       value: 5%
>     - name: ocp4-var-kubelet-evictionsoft-nodefs-available
>       rationale: evictionSoft.imagefs.nodefs.available = 10%
>       value: 10%
>     - name: ocp4-var-kubelet-evictionsoft-memory-available
>       rationale: evictionSoft.memory.available = 100Mi
>       value: 100Mi
>   disableRules:
>     - name: ocp4-kubelet-eviction-thresholds-set-hard-imagefs-inodesfree
>       rationale: The customer's kubelet doesn't seem to set this
>     - name: ocp4-kubelet-eviction-thresholds-set-soft-imagefs-inodesfree
>       rationale: The customer's kubelet doesn't seem to set this
> EOF
tailoredprofile.compliance.openshift.io/cis-node-bz created

$ oc get TailoredProfile
NAME          STATE
cis-node-bz   READY

$ oc create -f - << EOF
> apiVersion: compliance.openshift.io/v1alpha1
> kind: ScanSettingBinding
> metadata:
>   name: cis-bz
>   namespace: openshift-compliance
> profiles:
>   # Node checks
>   - name: cis-node-bz
>     kind: TailoredProfile
>     apiGroup: compliance.openshift.io/v1alpha1
>   # Cluster checks
>   - name: ocp4-cis
>     kind: Profile
>     apiGroup: compliance.openshift.io/v1alpha1
> settingsRef:
>   name: default
>   kind: ScanSetting
>   apiGroup: compliance.openshift.io/v1alpha1
> EOF
scansettingbinding.compliance.openshift.io/cis-bz created

]$ oc get suite -w
NAME     PHASE     RESULT
cis-bz   PENDING   NOT-AVAILABLE
cis-bz   LAUNCHING   NOT-AVAILABLE
cis-bz   LAUNCHING   NOT-AVAILABLE
cis-bz   LAUNCHING   NOT-AVAILABLE
cis-bz   RUNNING     NOT-AVAILABLE
 
$ oc get suite 
NAME     PHASE   RESULT
cis-bz   DONE    NON-COMPLIANT

$ oc get pods
NAME                                                    READY   STATUS      RESTARTS   AGE
aggregator-pod-cis-node-bz-master                       0/1     Completed   0          78s
aggregator-pod-cis-node-bz-worker                       0/1     Completed   0          89s
aggregator-pod-ocp4-cis                                 0/1     Completed   0          78s
compliance-operator-bb9f644cc-xwfnq                     1/1     Running     1          4h19m
ocp4-cis-api-checks-pod                                 0/2     Completed   0          109s
ocp4-openshift-compliance-pp-6d7c7db4bd-jwnnq           1/1     Running     0          4h17m
openscap-pod-03609e5ad2bea14ed075cccdd5ec9dc470c9e40f   0/2     Completed   0          116s
openscap-pod-64df8668170b580a0de1163d34c8c127ae9e4891   0/2     Completed   0          116s
openscap-pod-6d6aa5225203ba17dda78abbf29ffdc912653a70   0/2     Completed   0          116s
openscap-pod-9b5f85e315dce20ab418c80fb2aef3bd16331c04   0/2     Completed   0          108s
openscap-pod-d1bca77b2de582fe7acc13256246eba50ae9b6e5   0/2     Completed   0          108s
openscap-pod-eb8b23263548944696ea25c82e9b3fa33da451b9   0/2     Completed   0          109s
rhcos4-openshift-compliance-pp-c7b548bd-9hqvz           1/1     Running     0          4h17m

$ oc get ccr -l 'compliance.openshift.io/scan-name=cis-node-bz-worker' | grep kubelet-eviction
cis-node-bz-worker-kubelet-eviction-thresholds-set-hard-imagefs-available   PASS             medium
cis-node-bz-worker-kubelet-eviction-thresholds-set-hard-memory-available    PASS             medium
cis-node-bz-worker-kubelet-eviction-thresholds-set-hard-nodefs-available    PASS             medium
cis-node-bz-worker-kubelet-eviction-thresholds-set-hard-nodefs-inodesfree   PASS             medium
cis-node-bz-worker-kubelet-eviction-thresholds-set-soft-imagefs-available   PASS             medium
cis-node-bz-worker-kubelet-eviction-thresholds-set-soft-memory-available    PASS             medium
cis-node-bz-worker-kubelet-eviction-thresholds-set-soft-nodefs-available    PASS             medium
cis-node-bz-worker-kubelet-eviction-thresholds-set-soft-nodefs-inodesfree   PASS             medium

$ for rule in $(oc get ccr -l 'compliance.openshift.io/scan-name=cis-node-bz-worker' | grep kubelet-eviction | awk '{print $1}'); do echo -e "\n\n >>>>> Print instruction for" $rule ;oc get ccr $rule -ojsonpath={.instructions}; done


 >>>>> Print instruction for cis-node-bz-worker-kubelet-eviction-thresholds-set-hard-imagefs-available
Run the following command on the kubelet node(s):
$ oc debug -q node/$NODE -- jq -r '.evictionHard."imagefs.available"' /host/etc/kubernetes/kubelet.conf
and make sure it outputs a value.

 >>>>> Print instruction for cis-node-bz-worker-kubelet-eviction-thresholds-set-hard-memory-available
Run the following command on the kubelet node(s):
$ oc debug -q node/$NODE -- jq -r '.evictionHard."memory.available"' /host/etc/kubernetes/kubelet.conf
and make sure it outputs a value.

 >>>>> Print instruction for cis-node-bz-worker-kubelet-eviction-thresholds-set-hard-nodefs-available
Run the following command on the kubelet node(s):
$ oc debug -q node/$NODE -- jq -r '.evictionHard."nodefs.available"' /host/etc/kubernetes/kubelet.conf
and make sure it outputs a value.

 >>>>> Print instruction for cis-node-bz-worker-kubelet-eviction-thresholds-set-hard-nodefs-inodesfree
Run the following command on the kubelet node(s):
$ oc debug -q node/$NODE -- jq -r '.evictionHard."nodefs.inodesFree"' /host/etc/kubernetes/kubelet.conf
and make sure it outputs a value.

 >>>>> Print instruction for cis-node-bz-worker-kubelet-eviction-thresholds-set-soft-imagefs-available
Run the following command on the kubelet node(s):
$ oc debug -q node/$NODE -- jq -r '.evictionSoft."imagefs.available"' /host/etc/kubernetes/kubelet.conf
and make sure it outputs a value.

 >>>>> Print instruction for cis-node-bz-worker-kubelet-eviction-thresholds-set-soft-memory-available
Run the following command on the kubelet node(s):
$ oc debug -q node/$NODE -- jq -r '.evictionSoft."memory.available"' /host/etc/kubernetes/kubelet.conf
and make sure it outputs a value.

 >>>>> Print instruction for cis-node-bz-worker-kubelet-eviction-thresholds-set-soft-nodefs-available
Run the following command on the kubelet node(s):
$ oc debug -q node/$NODE -- jq -r '.evictionSoft."nodefs.available"' /host/etc/kubernetes/kubelet.conf
and make sure it outputs a value.

 >>>>> Print instruction for cis-node-bz-worker-kubelet-eviction-thresholds-set-soft-nodefs-inodesfree
Run the following command on the kubelet node(s):
$ oc debug -q node/$NODE -- jq -r '.evictionSoft."nodefs.inodesFree"' /host/etc/kubernetes/kubelet.conf
and make sure it outputs a value.

$ oc get nodes
NAME                                         STATUS   ROLES    AGE     VERSION
ip-10-0-135-188.us-east-2.compute.internal   Ready    worker   6h20m   v1.21.1+9807387
ip-10-0-148-205.us-east-2.compute.internal   Ready    master   6h26m   v1.21.1+9807387
ip-10-0-177-244.us-east-2.compute.internal   Ready    master   6h26m   v1.21.1+9807387
ip-10-0-181-138.us-east-2.compute.internal   Ready    worker   6h19m   v1.21.1+9807387
ip-10-0-204-255.us-east-2.compute.internal   Ready    worker   6h21m   v1.21.1+9807387
ip-10-0-211-145.us-east-2.compute.internal   Ready    master   6h26m   v1.21.1+9807387

$ for NODE in $(oc get node -lnode-role.kubernetes.io/worker= --no-headers |awk '{print $1}'); do echo -n "$NODE "; oc debug -q node/$NODE -- jq -r '.evictionHard."imagefs.available"' /host/etc/kubernetes/kubelet.conf; done
ip-10-0-135-188.us-east-2.compute.internal 15%
ip-10-0-181-138.us-east-2.compute.internal 15%
ip-10-0-204-255.us-east-2.compute.internal 15%

$ for NODE in $(oc get node -lnode-role.kubernetes.io/worker= --no-headers |awk '{print $1}'); do echo -n "$NODE "; oc debug -q node/$NODE -- jq -r '.evictionHard."nodefs.inodesFree"' /host/etc/kubernetes/kubelet.conf; done
ip-10-0-135-188.us-east-2.compute.internal 5%
ip-10-0-181-138.us-east-2.compute.internal 5%
ip-10-0-204-255.us-east-2.compute.internal 5%

$ for NODE in $(oc get node -lnode-role.kubernetes.io/worker= --no-headers |awk '{print $1}'); do echo -n "$NODE "; oc debug -q node/$NODE -- jq -r '.evictionSoft."memory.available"' /host/etc/kubernetes/kubelet.conf; done
ip-10-0-135-188.us-east-2.compute.internal 100Mi
ip-10-0-181-138.us-east-2.compute.internal 100Mi
ip-10-0-204-255.us-east-2.compute.internal 100Mi

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Compliance Operator bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:3214

Hi Shailendra,
you attached an active case to a bug that was resolved almost a year ago without any addition details. This is not really likely to trigger any case investigation from our end. If the customer is running a recent version of the operator, please open a new bug, not attach a case to a closed bug.

In general, this should not happen except for manual changes to the nodes (ssh + vi and such). The complianceCheckResult CRs should be labeled with nodes that differ from the rest, this is a good way to start an investigation.

The needinfo request[s] on this closed bug have been removed as they have been unresolved for 365 days