Description of problem: The remediation settings apply through kubeletconfig are not persistent on subsequent scan Version-Release number of selected component (if applicable): 4.10.0-0.nightly-2021-12-13-233752 + compliance-operator.v0.1.46 How reproducible: Always Steps to Reproduce: 1. Install compliance operator 2. Create scansetting and scansettingbinding object $ oc create -f - << EOF > apiVersion: compliance.openshift.io/v1alpha1 > autoApplyRemediations: true > autoUpdateRemediations: true > kind: ScanSetting > metadata: > name: auto-apply > namespace: openshift-compliance > rawResultStorage: > nodeSelector: > node-role.kubernetes.io/master: "" > pvAccessModes: > - ReadWriteOnce > rotation: 3 > size: 1Gi > tolerations: > - effect: NoSchedule > key: node-role.kubernetes.io/master > operator: Exists > roles: > - worker > scanTolerations: > - operator: Exists > schedule: 0 1 * * * > strictNodeScan: true > EOF scansetting.compliance.openshift.io/auto-apply created $ oc create -f - << EOF apiVersion: compliance.openshift.io/v1alpha1 kind: ScanSettingBinding metadata: name: pci-test profiles: - name: ocp4-pci-dss kind: Profile apiGroup: compliance.openshift.io/v1alpha1 - name: ocp4-pci-dss-node kind: Profile apiGroup: compliance.openshift.io/v1alpha1 settingsRef: name: auto-apply kind: ScanSetting apiGroup: compliance.openshift.io/v1alpha1 EOF scansettingbinding.compliance.openshift.io/pci-test created 3. Check scan status $ oc get pods NAME READY STATUS RESTARTS AGE aggregator-pod-ocp4-pci-dss 0/1 Completed 0 38s aggregator-pod-ocp4-pci-dss-node-worker 0/1 Completed 0 48s compliance-operator-58c774df7c-sn2gk 1/1 Running 1 (6h38m ago) 6h38m ocp4-openshift-compliance-pp-7d6c9cfd7b-4wrc8 1/1 Running 0 6h37m ocp4-pci-dss-api-checks-pod 0/2 Completed 0 69s openscap-pod-0a1d12439e7b5b37debe8a0fab53a25b7a7e1c05 0/2 Completed 0 69s openscap-pod-4fc74548ac1530f88db0369adf44cb356d53ed74 0/2 Completed 0 69s openscap-pod-6bc69aaa990a3b596c88c21c4e87f198d7cc662e 0/2 Completed 0 69s rhcos4-openshift-compliance-pp-8446c97876-2xmv4 1/1 Running 0 6h37m $ oc get suite NAME PHASE RESULT pci-test DONE NON-COMPLIANT $ oc get scan NAME PHASE RESULT ocp4-pci-dss DONE NON-COMPLIANT ocp4-pci-dss-node-worker DONE NON-COMPLIANT 4. Check FAIL rules count $ oc get ccr -lcompliance.openshift.io/check-status=FAIL NAME STATUS SEVERITY ocp4-pci-dss-audit-log-forwarding-enabled FAIL medium ocp4-pci-dss-configure-network-policies-namespaces FAIL high ocp4-pci-dss-file-integrity-exists FAIL medium ocp4-pci-dss-file-integrity-notification-enabled FAIL medium ocp4-pci-dss-kubeadmin-removed FAIL medium ocp4-pci-dss-node-worker-kubelet-configure-event-creation FAIL medium ocp4-pci-dss-node-worker-kubelet-configure-tls-cipher-suites FAIL medium ocp4-pci-dss-node-worker-kubelet-enable-iptables-util-chains FAIL medium ocp4-pci-dss-node-worker-kubelet-enable-protect-kernel-defaults FAIL medium ocp4-pci-dss-node-worker-kubelet-enable-protect-kernel-sysctl FAIL medium ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-hard-imagefs-available FAIL medium ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-hard-imagefs-inodesfree FAIL medium ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-hard-memory-available FAIL medium ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-hard-nodefs-available FAIL medium ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-hard-nodefs-inodesfree FAIL medium ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-soft-imagefs-available FAIL medium ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-soft-imagefs-inodesfree FAIL medium ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-soft-memory-available FAIL medium ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-soft-nodefs-available FAIL medium ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-soft-nodefs-inodesfree FAIL medium 5. Check all remediation gets applied expect ocp4-kubelet-enable-protect-kernel-defaults $ oc get complianceremediations NAME STATE ocp4-pci-dss-node-worker-kubelet-configure-event-creation Applied ocp4-pci-dss-node-worker-kubelet-configure-tls-cipher-suites Applied ocp4-pci-dss-node-worker-kubelet-enable-iptables-util-chains Applied ocp4-pci-dss-node-worker-kubelet-enable-protect-kernel-defaults MissingDependencies ocp4-pci-dss-node-worker-kubelet-enable-protect-kernel-sysctl Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-hard-imagefs-available Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-hard-imagefs-available-1 Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-hard-imagefs-inodesfree Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-hard-imagefs-inodesfree-1 Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-hard-memory-available Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-hard-memory-available-1 Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-hard-nodefs-available Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-hard-nodefs-available-1 Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-hard-nodefs-inodesfree Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-hard-nodefs-inodesfree-1 Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-soft-imagefs-available Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-soft-imagefs-available-1 Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-soft-imagefs-available-2 Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-soft-imagefs-inodesfree Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-soft-imagefs-inodesfree-1 Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-soft-imagefs-inodesfree-2 Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-soft-memory-available Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-soft-memory-available-1 Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-soft-memory-available-2 Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-soft-nodefs-available Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-soft-nodefs-available-1 Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-soft-nodefs-available-2 Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-soft-nodefs-inodesfree Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-soft-nodefs-inodesfree-1 Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-soft-nodefs-inodesfree-2 Applied 6. Check nodes status through machineConfigPool $ oc get mcp -w NAME CONFIG UPDATED UPDATING DEGRADED MACHINECOUNT READYMACHINECOUNT UPDATEDMACHINECOUNT DEGRADEDMACHINECOUNT AGE master rendered-master-0192a3dd220d7ccdf12a7b93a748e204 True False False 3 3 3 0 7h12m worker rendered-worker-c2b060c3f151374aff219b11fe2a70e1 False True False 3 0 0 0 7h12m worker rendered-worker-c2b060c3f151374aff219b11fe2a70e1 False True False 3 1 1 0 7h13m worker rendered-worker-c2b060c3f151374aff219b11fe2a70e1 False True False 3 1 1 0 7h13m worker rendered-worker-c2b060c3f151374aff219b11fe2a70e1 False True False 3 2 2 0 7h16m worker rendered-worker-c2b060c3f151374aff219b11fe2a70e1 False True False 3 2 2 0 7h16m worker rendered-worker-86c9421ad3323fee63e2b2dd5b6310a8 True False False 3 3 3 0 7h17m $ oc get mcp NAME CONFIG UPDATED UPDATING DEGRADED MACHINECOUNT READYMACHINECOUNT UPDATEDMACHINECOUNT DEGRADEDMACHINECOUNT AGE master rendered-master-0192a3dd220d7ccdf12a7b93a748e204 True False False 3 3 3 0 7h27m worker rendered-worker-86c9421ad3323fee63e2b2dd5b6310a8 True False False 3 3 3 0 7h27m 7. Rerun scan and check scan status along with FAIL rules count $ oc-compliance rerun-now compliancesuite/pci-test Rerunning scans from 'pci-test': ocp4-pci-dss, ocp4-pci-dss-node-worker Re-running scan 'openshift-compliance/ocp4-pci-dss' Re-running scan 'openshift-compliance/ocp4-pci-dss-node-worker' $ oc get suite -w NAME PHASE RESULT pci-test RUNNING NOT-AVAILABLE pci-test RUNNING NOT-AVAILABLE pci-test AGGREGATING NOT-AVAILABLE pci-test AGGREGATING NOT-AVAILABLE pci-test DONE NON-COMPLIANT pci-test DONE NON-COMPLIANT $ oc debug -q node/ip-10-0-153-40.us-east-2.compute.internal -- jq -r '.evictionHard."imagefs.available"' /host/etc/kubernetes/kubelet.conf W1214 16:48:46.149043 58277 warnings.go:70] would violate "latest" version of "baseline" PodSecurity profile: host namespaces (hostNetwork=true, hostPID=true), hostPath volumes (volume "host"), privileged (container "container-00" must not set securityContext.privileged=true) 10% $ oc get kubeletconfig NAME AGE compliance-operator-kubelet-worker 2m19s $ oc get mcp -w NAME CONFIG UPDATED UPDATING DEGRADED MACHINECOUNT READYMACHINECOUNT UPDATEDMACHINECOUNT DEGRADEDMACHINECOUNT AGE master rendered-master-0192a3dd220d7ccdf12a7b93a748e204 True False False 3 3 3 0 7h31m worker rendered-worker-86c9421ad3323fee63e2b2dd5b6310a8 False True False 3 1 1 0 7h31m worker rendered-worker-86c9421ad3323fee63e2b2dd5b6310a8 False True False 3 2 2 0 7h32m worker rendered-worker-86c9421ad3323fee63e2b2dd5b6310a8 False True False 3 2 2 0 7h32m worker rendered-worker-e6404be47931a5b39b99fd625441ee59 True False False 3 3 3 0 7h34m $ oc get mcp NAME CONFIG UPDATED UPDATING DEGRADED MACHINECOUNT READYMACHINECOUNT UPDATEDMACHINECOUNT DEGRADEDMACHINECOUNT AGE master rendered-master-0192a3dd220d7ccdf12a7b93a748e204 True False False 3 3 3 0 7h34m worker rendered-worker-e6404be47931a5b39b99fd625441ee59 True False False 3 3 3 0 7h34m $ oc debug -q node/ip-10-0-153-40.us-east-2.compute.internal -- jq -r '.evictionHard."imagefs.available"' /host/etc/kubernetes/kubelet.conf W1214 16:52:43.064639 58834 warnings.go:70] would violate "latest" version of "baseline" PodSecurity profile: host namespaces (hostNetwork=true, hostPID=true), hostPath volumes (volume "host"), privileged (container "container-00" must not set securityContext.privileged=true) null $ oc get ccr -lcompliance.openshift.io/check-status=FAIL NAME STATUS SEVERITY ocp4-pci-dss-audit-log-forwarding-enabled FAIL medium ocp4-pci-dss-configure-network-policies-namespaces FAIL high ocp4-pci-dss-file-integrity-exists FAIL medium ocp4-pci-dss-file-integrity-notification-enabled FAIL medium ocp4-pci-dss-kubeadmin-removed FAIL medium ocp4-pci-dss-node-worker-kubelet-enable-protect-kernel-defaults FAIL medium 8. Rerun scan to check the rules are PASS which applied remediation through machineConfigand and also FAIL rules count $ oc-compliance rerun-now compliancesuite/pci-test Rerunning scans from 'pci-test': ocp4-pci-dss, ocp4-pci-dss-node-worker Re-running scan 'openshift-compliance/ocp4-pci-dss' Re-running scan 'openshift-compliance/ocp4-pci-dss-node-worker' $ oc get suite -w NAME PHASE RESULT pci-test LAUNCHING NOT-AVAILABLE pci-test RUNNING NOT-AVAILABLE pci-test RUNNING NOT-AVAILABLE pci-test AGGREGATING NOT-AVAILABLE pci-test AGGREGATING NOT-AVAILABLE pci-test DONE NON-COMPLIANT pci-test DONE NON-COMPLIANT $ oc debug -q node/ip-10-0-153-40.us-east-2.compute.internal -- jq -r '.evictionHard."imagefs.available"' /host/etc/kubernetes/kubelet.conf W1214 16:55:17.042853 59136 warnings.go:70] would violate "latest" version of "baseline" PodSecurity profile: host namespaces (hostNetwork=true, hostPID=true), hostPath volumes (volume "host"), privileged (container "container-00" must not set securityContext.privileged=true) null $ oc get ccr -lcompliance.openshift.io/check-status=FAIL NAME STATUS SEVERITY ocp4-pci-dss-audit-log-forwarding-enabled FAIL medium ocp4-pci-dss-configure-network-policies-namespaces FAIL high ocp4-pci-dss-file-integrity-exists FAIL medium ocp4-pci-dss-file-integrity-notification-enabled FAIL medium ocp4-pci-dss-kubeadmin-removed FAIL medium ocp4-pci-dss-node-worker-kubelet-configure-event-creation FAIL medium ocp4-pci-dss-node-worker-kubelet-configure-tls-cipher-suites FAIL medium ocp4-pci-dss-node-worker-kubelet-enable-iptables-util-chains FAIL medium ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-hard-imagefs-available FAIL medium <--- ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-hard-imagefs-inodesfree FAIL medium ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-hard-memory-available FAIL medium ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-hard-nodefs-available FAIL medium ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-hard-nodefs-inodesfree FAIL medium ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-soft-imagefs-available FAIL medium ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-soft-imagefs-inodesfree FAIL medium ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-soft-memory-available FAIL medium ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-soft-nodefs-available FAIL medium ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-soft-nodefs-inodesfree FAIL medium Actual results: The rules applied remediation through kubeletconfig are reporting scan status FAIL after third scan and all parameters setting are getting removed from kubeletconfig. $ oc get kubeletconfig compliance-operator-kubelet-worker -o yaml apiVersion: machineconfiguration.openshift.io/v1 kind: KubeletConfig metadata: annotations: compliance.openshift.io/remediation: "" creationTimestamp: "2021-12-14T11:43:54Z" finalizers: - 99-worker-generated-kubelet generation: 1 labels: compliance.openshift.io/scan-name: ocp4-cis-node-worker compliance.openshift.io/suite: my-cis-ssb name: compliance-operator-kubelet-worker resourceVersion: "198251" uid: aa4f6db7-3e73-44ca-9a4a-96f72269c16c spec: kubeletConfig: protectKernelDefaults: true machineConfigPoolSelector: matchLabels: pools.operator.machineconfiguration.openshift.io/worker: "" status: conditions: - lastTransitionTime: "2021-12-14T11:43:54Z" message: Success status: "True" type: Success Expected results: Those rules applied remediation through kubeletconfig should not report the scan status FAIL after third scan and the parameters setting should not get removed from kubeletconfig. Additional info:
Fixed by: https://github.com/openshift/compliance-operator/pull/761
[Bug_Verification] Looks good, The remediations applied through kubeletconfig are persistent now on subsequent scans. Verified on: 4.10.0-0.nightly-2021-12-21-130047 + compliance-operator.v0.1.47 $ oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.10.0-0.nightly-2021-12-21-130047 True False 26m Cluster version is 4.10.0-0.nightly-2021-12-21-130047 $ oc get csv NAME DISPLAY VERSION REPLACES PHASE compliance-operator.v0.1.47 Compliance Operator 0.1.47 Succeeded elasticsearch-operator.5.3.2-5 OpenShift Elasticsearch Operator 5.3.2-5 Succeeded $ oc create -f - << EOF > apiVersion: compliance.openshift.io/v1alpha1 > autoApplyRemediations: true > autoUpdateRemediations: true > kind: ScanSetting > metadata: > name: auto-apply > namespace: openshift-compliance > rawResultStorage: > nodeSelector: > node-role.kubernetes.io/master: "" > pvAccessModes: > - ReadWriteOnce > rotation: 3 > size: 1Gi > tolerations: > - effect: NoSchedule > key: node-role.kubernetes.io/master > operator: Exists > roles: > - worker > scanTolerations: > - operator: Exists > schedule: 0 1 * * * > strictNodeScan: true > EOF scansetting.compliance.openshift.io/auto-apply created $ oc get ss NAME AGE auto-apply 8s default 14m default-auto-apply 14m $ oc create -f - << EOF > apiVersion: compliance.openshift.io/v1alpha1 > kind: ScanSettingBinding > metadata: > name: pci-test > profiles: > - name: ocp4-pci-dss > kind: Profile > apiGroup: compliance.openshift.io/v1alpha1 > - name: ocp4-pci-dss-node > kind: Profile > apiGroup: compliance.openshift.io/v1alpha1 > settingsRef: > name: auto-apply > kind: ScanSetting > apiGroup: compliance.openshift.io/v1alpha1 > EOF scansettingbinding.compliance.openshift.io/pci-test created $ oc get suite -w NAME PHASE RESULT pci-test RUNNING NOT-AVAILABLE pci-test RUNNING NOT-AVAILABLE pci-test AGGREGATING NOT-AVAILABLE pci-test AGGREGATING NOT-AVAILABLE pci-test DONE NON-COMPLIANT pci-test DONE NON-COMPLIANT $ oc get pods NAME READY STATUS RESTARTS AGE aggregator-pod-ocp4-pci-dss 0/1 Completed 0 93s aggregator-pod-ocp4-pci-dss-node-worker 0/1 Completed 0 103s compliance-operator-55fd995f9-7z9pf 1/1 Running 1 (17m ago) 18m ocp4-openshift-compliance-pp-54f5ffdd5b-5z6x6 1/1 Running 0 17m ocp4-pci-dss-api-checks-pod 0/2 Completed 0 2m14s openscap-pod-79a2f19388137a99d83a844f7b1d94e7dff0c3ae 0/2 Completed 0 2m13s openscap-pod-f0ca9b67d9d780b66b78f4e3d1fc9f11828c0093 0/2 Completed 0 2m13s rhcos4-openshift-compliance-pp-868bf9bd9b-q6xgx 1/1 Running 0 17m $ oc get suite NAME PHASE RESULT pci-test DONE NON-COMPLIANT $ oc get scan NAME PHASE RESULT ocp4-pci-dss DONE NON-COMPLIANT ocp4-pci-dss-node-worker DONE NON-COMPLIANT $ oc get mcp -w NAME CONFIG UPDATED UPDATING DEGRADED MACHINECOUNT READYMACHINECOUNT UPDATEDMACHINECOUNT DEGRADEDMACHINECOUNT AGE master rendered-master-66fc6d9f1a59b6f3b3c3fe23219d9f9b True False False 3 3 3 0 57m worker rendered-worker-dbaec782c061fae1ae3f3dea7a0e1d12 False True False 3 0 0 0 57m $ oc get ccr -lcompliance.openshift.io/check-status=FAIL NAME STATUS SEVERITY ocp4-pci-dss-api-server-encryption-provider-cipher FAIL medium ocp4-pci-dss-api-server-encryption-provider-config FAIL medium ocp4-pci-dss-api-server-no-adm-ctrl-plugins-disabled FAIL medium ocp4-pci-dss-audit-log-forwarding-enabled FAIL medium ocp4-pci-dss-configure-network-policies-namespaces FAIL high ocp4-pci-dss-file-integrity-exists FAIL medium ocp4-pci-dss-file-integrity-notification-enabled FAIL medium ocp4-pci-dss-kubeadmin-removed FAIL medium ocp4-pci-dss-node-worker-kubelet-configure-event-creation FAIL medium ocp4-pci-dss-node-worker-kubelet-configure-tls-cipher-suites FAIL medium ocp4-pci-dss-node-worker-kubelet-enable-iptables-util-chains FAIL medium ocp4-pci-dss-node-worker-kubelet-enable-protect-kernel-defaults FAIL medium ocp4-pci-dss-node-worker-kubelet-enable-protect-kernel-sysctl FAIL medium ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-hard-imagefs-available FAIL medium ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-hard-imagefs-inodesfree FAIL medium ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-hard-memory-available FAIL medium ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-hard-nodefs-available FAIL medium ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-hard-nodefs-inodesfree FAIL medium ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-soft-imagefs-available FAIL medium ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-soft-imagefs-inodesfree FAIL medium ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-soft-memory-available FAIL medium ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-soft-nodefs-available FAIL medium ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-soft-nodefs-inodesfree FAIL medium $ oc get complianceremediations NAME STATE ocp4-pci-dss-api-server-encryption-provider-cipher Applied ocp4-pci-dss-api-server-encryption-provider-config Applied ocp4-pci-dss-node-worker-kubelet-configure-event-creation Applied ocp4-pci-dss-node-worker-kubelet-configure-tls-cipher-suites Applied ocp4-pci-dss-node-worker-kubelet-enable-iptables-util-chains Applied ocp4-pci-dss-node-worker-kubelet-enable-protect-kernel-defaults MissingDependencies ocp4-pci-dss-node-worker-kubelet-enable-protect-kernel-sysctl Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-hard-imagefs-available Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-hard-imagefs-available-1 Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-hard-imagefs-inodesfree Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-hard-imagefs-inodesfree-1 Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-hard-memory-available Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-hard-memory-available-1 Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-hard-nodefs-available Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-hard-nodefs-available-1 Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-hard-nodefs-inodesfree Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-hard-nodefs-inodesfree-1 Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-soft-imagefs-available Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-soft-imagefs-available-1 Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-soft-imagefs-available-2 Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-soft-imagefs-inodesfree Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-soft-imagefs-inodesfree-1 Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-soft-imagefs-inodesfree-2 Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-soft-memory-available Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-soft-memory-available-1 Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-soft-memory-available-2 Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-soft-nodefs-available Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-soft-nodefs-available-1 Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-soft-nodefs-available-2 Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-soft-nodefs-inodesfree Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-soft-nodefs-inodesfree-1 Applied ocp4-pci-dss-node-worker-kubelet-eviction-thresholds-set-soft-nodefs-inodesfree-2 Applied $ oc get kubeletconfig NAME AGE compliance-operator-kubelet-worker 2m28s $ oc get mc 75-ocp4-pci-dss-node-worker-kubelet-enable-protect-kernel-sysctl NAME GENERATEDBYCONTROLLER IGNITIONVERSION AGE 75-ocp4-pci-dss-node-worker-kubelet-enable-protect-kernel-sysctl 3.1.0 3m $ oc get mcp -w NAME CONFIG UPDATED UPDATING DEGRADED MACHINECOUNT READYMACHINECOUNT UPDATEDMACHINECOUNT DEGRADEDMACHINECOUNT AGE master rendered-master-66fc6d9f1a59b6f3b3c3fe23219d9f9b True False False 3 3 3 0 59m worker rendered-worker-dbaec782c061fae1ae3f3dea7a0e1d12 False True False 3 1 1 0 59m worker rendered-worker-dbaec782c061fae1ae3f3dea7a0e1d12 False True False 3 2 2 0 60m worker rendered-worker-dbaec782c061fae1ae3f3dea7a0e1d12 False True False 3 2 2 0 60m worker rendered-worker-02aff05f91ecda4299d283bfbf7feadf True False False 3 3 3 0 62m $ oc get mcp NAME CONFIG UPDATED UPDATING DEGRADED MACHINECOUNT READYMACHINECOUNT UPDATEDMACHINECOUNT DEGRADEDMACHINECOUNT AGE master rendered-master-66fc6d9f1a59b6f3b3c3fe23219d9f9b True False False 3 3 3 0 62m worker rendered-worker-02aff05f91ecda4299d283bfbf7feadf True False False 3 3 3 0 62m $ oc get nodes NAME STATUS ROLES AGE VERSION ip-10-0-128-176.us-east-2.compute.internal Ready master 64m v1.22.1+6859754 ip-10-0-133-98.us-east-2.compute.internal Ready worker 59m v1.22.1+6859754 ip-10-0-164-181.us-east-2.compute.internal Ready worker 59m v1.22.1+6859754 ip-10-0-168-101.us-east-2.compute.internal Ready master 63m v1.22.1+6859754 ip-10-0-192-53.us-east-2.compute.internal Ready master 64m v1.22.1+6859754 ip-10-0-201-172.us-east-2.compute.internal Ready worker 59m v1.22.1+6859754 $ oc debug -q node/ip-10-0-133-98.us-east-2.compute.internal -- jq -r '.evictionHard."imagefs.available"' /host/etc/kubernetes/kubelet.conf 10% $ oc describe kubeletconfig compliance-operator-kubelet-worker Name: compliance-operator-kubelet-worker Namespace: Labels: compliance.openshift.io/scan-name=ocp4-pci-dss-node-worker compliance.openshift.io/suite=pci-test Annotations: compliance.openshift.io/remediation: API Version: machineconfiguration.openshift.io/v1 Kind: KubeletConfig Metadata: Creation Timestamp: 2021-12-22T07:33:21Z Finalizers: 99-worker-generated-kubelet Generation: 19 Managed Fields: API Version: machineconfiguration.openshift.io/v1 Fields Type: FieldsV1 fieldsV1: f:metadata: f:finalizers: .: v:"99-worker-generated-kubelet": Manager: machine-config-controller Operation: Update Time: 2021-12-22T07:33:21Z API Version: machineconfiguration.openshift.io/v1 Fields Type: FieldsV1 fieldsV1: f:status: .: f:conditions: Manager: machine-config-controller Operation: Update Subresource: status Time: 2021-12-22T07:33:21Z API Version: machineconfiguration.openshift.io/v1 Fields Type: FieldsV1 fieldsV1: f:metadata: f:annotations: .: f:compliance.openshift.io/remediation: f:labels: .: f:compliance.openshift.io/scan-name: f:compliance.openshift.io/suite: f:spec: .: f:kubeletConfig: .: f:eventRecordQPS: f:evictionHard: f:evictionPressureTransitionPeriod: f:evictionSoft: f:evictionSoftGracePeriod: f:makeIPTablesUtilChains: f:tlsCipherSuites: f:machineConfigPoolSelector: .: f:matchLabels: .: f:pools.operator.machineconfiguration.openshift.io/worker: Manager: compliance-operator Operation: Update Time: 2021-12-22T07:33:24Z Resource Version: 46593 UID: 508dc675-d581-4f12-95ac-c5dc0fd2445f Spec: Kubelet Config: Event Record QPS: 10 Eviction Hard: imagefs.available: 10% imagefs.inodesFree: 5% memory.available: 200Mi nodefs.available: 5% nodefs.inodesFree: 4% Eviction Pressure Transition Period: 0s Eviction Soft: imagefs.available: 15% imagefs.inodesFree: 10% memory.available: 500Mi nodefs.available: 10% nodefs.inodesFree: 5% Eviction Soft Grace Period: imagefs.available: 1m30s imagefs.inodesFree: 1m30s memory.available: 1m30s nodefs.available: 1m30s nodefs.inodesFree: 1m30s Make IP Tables Util Chains: true Tls Cipher Suites: TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 Machine Config Pool Selector: Match Labels: pools.operator.machineconfiguration.openshift.io/worker: Status: Conditions: Last Transition Time: 2021-12-22T07:33:28Z Message: Success Status: True Type: Success Events: <none> $ oc get complianceremediations ocp4-pci-dss-node-worker-kubelet-enable-protect-kernel-defaults NAME STATE ocp4-pci-dss-node-worker-kubelet-enable-protect-kernel-defaults MissingDependencies $ oc-compliance rerun-now compliancesuite/pci-test Rerunning scans from 'pci-test': ocp4-pci-dss, ocp4-pci-dss-node-worker Re-running scan 'openshift-compliance/ocp4-pci-dss' Re-running scan 'openshift-compliance/ocp4-pci-dss-node-worker' $ oc get suite -w NAME PHASE RESULT pci-test AGGREGATING NOT-AVAILABLE pci-test AGGREGATING NOT-AVAILABLE pci-test DONE NON-COMPLIANT pci-test DONE NON-COMPLIANT $ oc get complianceremediations ocp4-pci-dss-node-worker-kubelet-enable-protect-kernel-defaults NAME STATE ocp4-pci-dss-node-worker-kubelet-enable-protect-kernel-defaults Applied $ oc debug -q node/ip-10-0-133-98.us-east-2.compute.internal -- jq -r '.evictionHard."imagefs.available"' /host/etc/kubernetes/kubelet.conf 10% $ oc get ccr -lcompliance.openshift.io/check-status=FAIL NAME STATUS SEVERITY ocp4-pci-dss-api-server-no-adm-ctrl-plugins-disabled FAIL medium ocp4-pci-dss-audit-log-forwarding-enabled FAIL medium ocp4-pci-dss-configure-network-policies-namespaces FAIL high ocp4-pci-dss-file-integrity-exists FAIL medium ocp4-pci-dss-file-integrity-notification-enabled FAIL medium ocp4-pci-dss-kubeadmin-removed FAIL medium ocp4-pci-dss-node-worker-kubelet-enable-protect-kernel-defaults FAIL medium $ oc describe kubeletconfig compliance-operator-kubelet-worker Name: compliance-operator-kubelet-worker Namespace: Labels: compliance.openshift.io/scan-name=ocp4-pci-dss-node-worker compliance.openshift.io/suite=pci-test Annotations: compliance.openshift.io/remediation: API Version: machineconfiguration.openshift.io/v1 Kind: KubeletConfig Metadata: Creation Timestamp: 2021-12-22T07:33:21Z Finalizers: 99-worker-generated-kubelet Generation: 20 Managed Fields: API Version: machineconfiguration.openshift.io/v1 Fields Type: FieldsV1 fieldsV1: f:metadata: f:finalizers: .: v:"99-worker-generated-kubelet": Manager: machine-config-controller Operation: Update Time: 2021-12-22T07:33:21Z API Version: machineconfiguration.openshift.io/v1 Fields Type: FieldsV1 fieldsV1: f:status: .: f:conditions: Manager: machine-config-controller Operation: Update Subresource: status Time: 2021-12-22T07:33:21Z API Version: machineconfiguration.openshift.io/v1 Fields Type: FieldsV1 fieldsV1: f:metadata: f:annotations: .: f:compliance.openshift.io/remediation: f:labels: .: f:compliance.openshift.io/scan-name: f:compliance.openshift.io/suite: f:spec: .: f:kubeletConfig: .: f:eventRecordQPS: f:evictionHard: f:evictionPressureTransitionPeriod: f:evictionSoft: f:evictionSoftGracePeriod: f:makeIPTablesUtilChains: f:protectKernelDefaults: f:tlsCipherSuites: f:machineConfigPoolSelector: .: f:matchLabels: .: f:pools.operator.machineconfiguration.openshift.io/worker: Manager: compliance-operator Operation: Update Time: 2021-12-22T07:42:42Z Resource Version: 54028 UID: 508dc675-d581-4f12-95ac-c5dc0fd2445f Spec: Kubelet Config: Event Record QPS: 10 Eviction Hard: imagefs.available: 10% imagefs.inodesFree: 5% memory.available: 200Mi nodefs.available: 5% nodefs.inodesFree: 4% Eviction Pressure Transition Period: 0s Eviction Soft: imagefs.available: 15% imagefs.inodesFree: 10% memory.available: 500Mi nodefs.available: 10% nodefs.inodesFree: 5% Eviction Soft Grace Period: imagefs.available: 1m30s imagefs.inodesFree: 1m30s memory.available: 1m30s nodefs.available: 1m30s nodefs.inodesFree: 1m30s Make IP Tables Util Chains: true Protect Kernel Defaults: true Tls Cipher Suites: TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 Machine Config Pool Selector: Match Labels: pools.operator.machineconfiguration.openshift.io/worker: Status: Conditions: Last Transition Time: 2021-12-22T07:42:42Z Message: Success Status: True Type: Success Events: <none> $ oc get mcp -w NAME CONFIG UPDATED UPDATING DEGRADED MACHINECOUNT READYMACHINECOUNT UPDATEDMACHINECOUNT DEGRADEDMACHINECOUNT AGE master rendered-master-66fc6d9f1a59b6f3b3c3fe23219d9f9b True False False 3 3 3 0 66m worker rendered-worker-02aff05f91ecda4299d283bfbf7feadf False True False 3 0 0 0 66m worker rendered-worker-02aff05f91ecda4299d283bfbf7feadf False True False 3 1 1 0 67m worker rendered-worker-02aff05f91ecda4299d283bfbf7feadf False True False 3 1 1 0 67m worker rendered-worker-02aff05f91ecda4299d283bfbf7feadf False True False 3 2 2 0 69m worker rendered-worker-02aff05f91ecda4299d283bfbf7feadf False True False 3 2 2 0 69m worker rendered-worker-62ee9a7137536a8d3c8c416efccf0d65 True False False 3 3 3 0 71m $ oc get mcp NAME CONFIG UPDATED UPDATING DEGRADED MACHINECOUNT READYMACHINECOUNT UPDATEDMACHINECOUNT DEGRADEDMACHINECOUNT AGE master rendered-master-66fc6d9f1a59b6f3b3c3fe23219d9f9b True False False 3 3 3 0 104m worker rendered-worker-62ee9a7137536a8d3c8c416efccf0d65 True False False 3 3 3 0 104m $ oc-compliance rerun-now compliancesuite/pci-test Rerunning scans from 'pci-test': ocp4-pci-dss, ocp4-pci-dss-node-worker Re-running scan 'openshift-compliance/ocp4-pci-dss' Re-running scan 'openshift-compliance/ocp4-pci-dss-node-worker' $ oc get suite -w NAME PHASE RESULT pci-test RUNNING NOT-AVAILABLE pci-test RUNNING NOT-AVAILABLE pci-test AGGREGATING NOT-AVAILABLE pci-test AGGREGATING NOT-AVAILABLE pci-test DONE NON-COMPLIANT pci-test DONE NON-COMPLIANT $ oc describe kubeletconfig compliance-operator-kubelet-worker Name: compliance-operator-kubelet-worker Namespace: Labels: compliance.openshift.io/scan-name=ocp4-pci-dss-node-worker compliance.openshift.io/suite=pci-test Annotations: compliance.openshift.io/remediation: API Version: machineconfiguration.openshift.io/v1 Kind: KubeletConfig Metadata: Creation Timestamp: 2021-12-22T07:33:21Z Finalizers: 99-worker-generated-kubelet Generation: 20 Managed Fields: API Version: machineconfiguration.openshift.io/v1 Fields Type: FieldsV1 fieldsV1: f:metadata: f:finalizers: .: v:"99-worker-generated-kubelet": Manager: machine-config-controller Operation: Update Time: 2021-12-22T07:33:21Z API Version: machineconfiguration.openshift.io/v1 Fields Type: FieldsV1 fieldsV1: f:status: .: f:conditions: Manager: machine-config-controller Operation: Update Subresource: status Time: 2021-12-22T07:33:21Z API Version: machineconfiguration.openshift.io/v1 Fields Type: FieldsV1 fieldsV1: f:metadata: f:annotations: .: f:compliance.openshift.io/remediation: f:labels: .: f:compliance.openshift.io/scan-name: f:compliance.openshift.io/suite: f:spec: .: f:kubeletConfig: .: f:eventRecordQPS: f:evictionHard: f:evictionPressureTransitionPeriod: f:evictionSoft: f:evictionSoftGracePeriod: f:makeIPTablesUtilChains: f:protectKernelDefaults: f:tlsCipherSuites: f:machineConfigPoolSelector: .: f:matchLabels: .: f:pools.operator.machineconfiguration.openshift.io/worker: Manager: compliance-operator Operation: Update Time: 2021-12-22T07:42:42Z Resource Version: 54028 UID: 508dc675-d581-4f12-95ac-c5dc0fd2445f Spec: Kubelet Config: Event Record QPS: 10 Eviction Hard: imagefs.available: 10% imagefs.inodesFree: 5% memory.available: 200Mi nodefs.available: 5% nodefs.inodesFree: 4% Eviction Pressure Transition Period: 0s Eviction Soft: imagefs.available: 15% imagefs.inodesFree: 10% memory.available: 500Mi nodefs.available: 10% nodefs.inodesFree: 5% Eviction Soft Grace Period: imagefs.available: 1m30s imagefs.inodesFree: 1m30s memory.available: 1m30s nodefs.available: 1m30s nodefs.inodesFree: 1m30s Make IP Tables Util Chains: true Protect Kernel Defaults: true Tls Cipher Suites: TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 Machine Config Pool Selector: Match Labels: pools.operator.machineconfiguration.openshift.io/worker: Status: Conditions: Last Transition Time: 2021-12-22T07:42:42Z Message: Success Status: True Type: Success Events: <none> $ oc debug -q node/ip-10-0-133-98.us-east-2.compute.internal -- jq -r '.evictionHard."imagefs.available"' /host/etc/kubernetes/kubelet.conf oc get ccr -lcompliance.openshift.io/check-status=FAIL 10% $ oc get ccr -lcompliance.openshift.io/check-status=FAIL NAME STATUS SEVERITY ocp4-pci-dss-api-server-no-adm-ctrl-plugins-disabled FAIL medium ocp4-pci-dss-audit-log-forwarding-enabled FAIL medium ocp4-pci-dss-configure-network-policies-namespaces FAIL high ocp4-pci-dss-file-integrity-exists FAIL medium ocp4-pci-dss-file-integrity-notification-enabled FAIL medium ocp4-pci-dss-kubeadmin-removed FAIL medium $ oc get ccr ocp4-pci-dss-node-worker-kubelet-enable-protect-kernel-defaults NAME STATUS SEVERITY ocp4-pci-dss-node-worker-kubelet-enable-protect-kernel-defaults PASS medium
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (OpenShift Compliance Operator bug fix and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2022:0014