Bug 2105153
| Summary: | Rule ocp4-kubelet-enable-client-cert-rotation is not working as expected | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | zhou ying <yinzhou> |
| Component: | Compliance Operator | Assignee: | Vincent Shen <wenshen> |
| Status: | CLOSED ERRATA | QA Contact: | |
| Severity: | medium | Docs Contact: | |
| Priority: | medium | ||
| Version: | 4.11 | CC: | lbragsta, mrogers, wenshen, xiyuan |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: |
If this bug requires documentation, please select an appropriate Doc Type value.
|
Story Points: | --- |
| Clone Of: | Environment: | ||
| Last Closed: | 2022-11-02 16:00:53 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
It appears we have two rules for CIS 4.2.11, one rule `ocp4-cis-node-master-kubelet-enable-cert-rotation` we check if `rotateCertificates` is not set to false, and the other rule `ocp4-cis-node-master-kubelet-enable-cert-rotation` we have is to check if `RotateKubeletClientCertificate` is not set to false. The second rule will pass if `RotateKubeletClientCertificate` is not set. I will fix the instruction in the second rule. Verification pass with 4.12.0-0.nightly-2022-09-20-095559 + compliance-operator.v0.1.55
$ oc apply -f -<<EOF
apiVersion: compliance.openshift.io/v1alpha1
kind: ScanSettingBinding
metadata:
name: test
profiles:
- apiGroup: compliance.openshift.io/v1alpha1
kind: Profile
name: ocp4-cis
- apiGroup: compliance.openshift.io/v1alpha1
kind: Profile
name: ocp4-cis-node
settingsRef:
apiGroup: compliance.openshift.io/v1alpha1
kind: ScanSetting
name: default
EOF
scansettingbinding.compliance.openshift.io/test created
$ oc get scan
NAME PHASE RESULT
ocp4-cis DONE NON-COMPLIANT
ocp4-cis-node-master DONE NON-COMPLIANT
ocp4-cis-node-worker DONE NON-COMPLIANT
$ oc get compliancecheckresults | grep rotation
ocp4-cis-kubelet-enable-cert-rotation PASS medium
ocp4-cis-kubelet-enable-client-cert-rotation PASS medium
ocp4-cis-kubelet-enable-server-cert-rotation PASS medium
$ oc get ccr ocp4-cis-kubelet-enable-cert-rotation -o=jsonpath={.instructions}
Run the following command on the kubelet node(s):
$ sudo grep rotateCertificates /etc/kubernetes/kubelet.conf
The output should return nothing or true.
$ for NODE_NAME in $(oc get nodes -ojsonpath='{.items[*].metadata.name}'); do oc debug node/$NODE_NAME -- chroot /host grep rotateCertificates /etc/kubernetes/kubelet.conf; done
Starting pod/xiyuan23-3-r6v2t-master-0-debug ...
To use host binaries, run `chroot /host`
"rotateCertificates": true,
Removing debug pod ...
Starting pod/xiyuan23-3-r6v2t-master-1-debug ...
To use host binaries, run `chroot /host`
"rotateCertificates": true,
Removing debug pod ...
Starting pod/xiyuan23-3-r6v2t-master-2-debug ...
To use host binaries, run `chroot /host`
"rotateCertificates": true,
Removing debug pod ...
Starting pod/xiyuan23-3-r6v2t-worker-southcentralus1-2xhzl-debug ...
To use host binaries, run `chroot /host`
"rotateCertificates": true,
Removing debug pod ...
Starting pod/xiyuan23-3-r6v2t-worker-southcentralus2-6vbhx-debug ...
To use host binaries, run `chroot /host`
"rotateCertificates": true,
Removing debug pod ...
Starting pod/xiyuan23-3-r6v2t-worker-southcentralus3-swz7f-debug ...
To use host binaries, run `chroot /host`
"rotateCertificates": true,
Removing debug pod ...
$ oc get ccr ocp4-cis-kubelet-enable-client-cert-rotation -o=jsonpath={.instructions}
Run the following command on the kubelet node(s):
$ for NODE_NAME in $(oc get nodes -ojsonpath='{.items[*].metadata.name}'); do oc get --raw /api/v1/nodes/$NODE_NAME/proxy/configz | jq '.kubeletconfig|.kind="KubeletConfiguration"|.apiVersion="kubelet.config.k8s.io/v1beta1"' | grep RotateKubeletClientCertificate; done
The output should return nothing or true.
$ for NODE_NAME in $(oc get nodes -ojsonpath='{.items[*].metadata.name}'); do oc get --raw /api/v1/nodes/$NODE_NAME/proxy/configz | jq '.kubeletconfig|.kind="KubeletConfiguration"|.apiVersion="kubelet.config.k8s.io/v1beta1"' | grep RotateKubeletClientCertificate; done
$
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (OpenShift Compliance Operator bug fix update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2022:6657 |
Description of problem: Rule ocp4-kubelet-enable-client-cert-rotation is not working as expected Version-Release number of selected component (if applicable): oc get csv NAME DISPLAY VERSION REPLACES PHASE compliance-operator.v0.1.53 Compliance Operator 0.1.53 Succeeded How reproducible: always Steps to Reproduce: 1. Generate a compliancesuite through a new ScanSetting CR: $ oc create -f - << EOF apiVersion: compliance.openshift.io/v1alpha1 kind: ScanSettingBinding metadata: name: my-ssb-r profiles: - name: ocp4-cis kind: Profile apiGroup: compliance.openshift.io/v1alpha1 - name: ocp4-cis-node kind: Profile apiGroup: compliance.openshift.io/v1alpha1 settingsRef: name: default-auto-apply kind: ScanSetting apiGroup: compliance.openshift.io/v1alpha1 EOF 2 check compliancecheckresult for： oc get compliancecheckresults | grep kubelet-enable-client-cert ocp4-cis-node-master-kubelet-enable-client-cert-rotation PASS medium ocp4-cis-node-worker-kubelet-enable-client-cert-rotation PASS medium 3. Check the instructions for kubelet-enable-client-cert: Actual results: The instructions for kubelet-enable-client-cert is not correct: oc get compliancecheckresults ocp4-cis-node-master-kubelet-enable-client-cert-rotation -o=jsonpath={.instructions} Run the following command on the kubelet node(s): $ sudo grep RotateKubeletClientCertificate /etc/kubernetes/kubelet.conf The output should return true. # oc get compliancecheckresults ocp4-cis-node-worker-kubelet-enable-client-cert-rotation -o=jsonpath={.instructions} Run the following command on the kubelet node(s): $ sudo grep RotateKubeletClientCertificate /etc/kubernetes/kubelet.conf The output should return true. oc debug node/yinzhouco81-9t744-worker-southcentralus2-cjbrm -- chroot /host sudo grep RotateKubeletClientCertificate /etc/kubernetes/kubelet.conf Warning: would violate PodSecurity "restricted:latest": host namespaces (hostNetwork=true, hostPID=true), privileged (container "container-00" must not set securityContext.privileged=true), allowPrivilegeEscalation != false (container "container-00" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "container-00" must set securityContext.capabilities.drop=["ALL"]), restricted volume types (volume "host" uses restricted volume type "hostPath"), runAsNonRoot != true (pod or container "container-00" must set securityContext.runAsNonRoot=true), runAsUser=0 (container "container-00" must not set runAsUser=0), seccompProfile (pod or container "container-00" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost") Starting pod/yinzhouco81-9t744-worker-southcentralus2-cjbrm-debug ... To use host binaries, run `chroot /host` Removing debug pod ... Expected results: Per the instruction, the rule should fail. Additional info: Maybe need to check rotateCertificates not the RotateKubeletClientCertificate.