Description of problem: We want the cluster-kube-apiserver-operator to raise alerts in case there is a violation to the applicable Pod Security admission profile so that we are able to measure how many clusters would be affected by turning the PSa to enforce "restricted" profile globally in the cluster. Version-Release number of selected component (if applicable): 4.11 How reproducible: always Steps to Reproduce: 1. check whether there is an alert after "creating" a workload that violates its namespace-local Pod Security admission configuration Actual results: No alerts are being fired. Expected results: There should be an alert in the openshift-kube-apiserver namespace. Additional info:
The current alert would very likely trigger on many clusters as its Prometheus query is too broad, moving back to ASSIGNED.
Setting blocker+ as this would likely cause alerts in most clusters.
Verified on 4.11.0-0.nightly-2022-06-30-005428 1. Create and label a namespace with pod-security.kubernetes.io/enforce=restricted $ oc new-project testproj $ oc label ns testproj pod-security.kubernetes.io/enforce=restricted 2. Check the following query in Openshift console (prometheus) metrics sum(increase(pod_security_evaluations_total{resource="pod"}[1m])) by(decision,mode,policy_level) 3. Create a sample pod that should get denied due to scc restrictions $ oc create -f - <<EOF apiVersion: v1 kind: Pod metadata: name: testpod spec: containers: - image: quay.io/openshifttest/hello-openshift:openshift name: node-hello securityContext: privileged: true runAsUser: 0 EOF Error from server (Forbidden): error when creating "STDIN": pods "testpod" is forbidden: violates PodSecurity "restricted:latest": privileged (container "node-hello" must not set securityContext.privileged=true), allowPrivilegeEscalation != false (container "node-hello" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "node-hello" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or container "node-hello" must set securityContext.runAsNonRoot=true), runAsUser=0 (container "node-hello" must not set runAsUser=0), seccompProfile (pod or container "node-hello" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost") 4. Check the query in Openshift console (prometheus) metrics again sum(increase(pod_security_evaluations_total{resource="pod"}[1m])) by(decision,mode,policy_level) values should change to reflect denial to create pod Moving to verified
You should verify with alert fired. ` sum(increase(pod_security_evaluations_total{resource="pod"}[1m])) by(decision,mode,policy_level) ` is only tips to help understand whether the alert's condition satisfaction includes your test operation's triggering.
Verified on 4.11.0-0.nightly-2022-06-30-005428 1. Create and label a namespace with pod-security.kubernetes.io/enforce=restricted $ oc new-project testproj $ oc label ns testproj pod-security.kubernetes.io/enforce=restricted 2. Create a sample pod that should get denied due to scc restrictions $ oc create -f - <<EOF apiVersion: v1 kind: Pod metadata: name: testpod spec: containers: - image: quay.io/openshifttest/hello-openshift:openshift name: node-hello securityContext: privileged: true runAsUser: 0 EOF Error from server (Forbidden): error when creating "STDIN": pods "testpod" is forbidden: violates PodSecurity "restricted:latest": privileged (container "node-hello" must not set securityContext.privileged=true), allowPrivilegeEscalation != false (container "node-hello" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "node-hello" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or container "node-hello" must set securityContext.runAsNonRoot=true), runAsUser=0 (container "node-hello" must not set runAsUser=0), seccompProfile (pod or container "node-hello" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost") 3. Check the Alerts in Openshift console Actual PodSecurityViolation alert is being fired Expected PodSecurityViolation alert should be in a firing state Moving to verified
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Important: OpenShift Container Platform 4.11.0 bug fix and security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:5069