Bug 2100155

Summary:	kube-apiserver-operator should raise an alert when there is a Pod Security admission violation
Product:	OpenShift Container Platform	Reporter:	Standa Laznicka <slaznick>
Component:	apiserver-auth	Assignee:	Standa Laznicka <slaznick>
Status:	CLOSED ERRATA	QA Contact:	Yash Tripathi <ytripath>
Severity:	urgent	Docs Contact:
Priority:	urgent
Version:	4.11	CC:	mfojtik, surbania, wlewis, xxia
Target Milestone:	---	Keywords:	FastFix, NeedsTestCase
Target Release:	4.11.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:
Clones:	2101444 (view as bug list)		Environment:
Last Closed:	2022-08-10 11:19:11 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	2101444
Bug Blocks:

Description Standa Laznicka 2022-06-22 14:55:17 UTC

Description of problem:
We want the cluster-kube-apiserver-operator to raise alerts in case there is a violation to the applicable Pod Security admission profile so that we are able to measure how many clusters would be affected by turning the PSa to enforce "restricted" profile globally in the cluster.

Version-Release number of selected component (if applicable):
4.11

How reproducible:
always

Steps to Reproduce:
1. check whether there is an alert after "creating" a workload that violates its namespace-local Pod Security admission configuration


Actual results:
No alerts are being fired.


Expected results:
There should be an alert in the openshift-kube-apiserver namespace.

Additional info:

Comment 3 Standa Laznicka 2022-06-27 09:54:13 UTC

The current alert would very likely trigger on many clusters as its Prometheus query is too broad, moving back to ASSIGNED.

Comment 4 Standa Laznicka 2022-06-27 09:55:11 UTC

Setting blocker+ as this would likely cause alerts in most clusters.

Comment 8 Yash Tripathi 2022-07-01 13:45:28 UTC

Verified on 4.11.0-0.nightly-2022-06-30-005428

1. Create and label a namespace with pod-security.kubernetes.io/enforce=restricted
$ oc new-project testproj
$ oc label ns testproj pod-security.kubernetes.io/enforce=restricted

2. Check the following query in Openshift console (prometheus) metrics
sum(increase(pod_security_evaluations_total{resource="pod"}[1m])) by(decision,mode,policy_level)

3. Create a sample pod that should get denied due to scc restrictions
$ oc create -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
  name: testpod
spec:
  containers:
  - image: quay.io/openshifttest/hello-openshift:openshift
    name: node-hello
    securityContext:
      privileged: true
      runAsUser: 0
EOF
Error from server (Forbidden): error when creating "STDIN": pods "testpod" is forbidden: violates PodSecurity "restricted:latest": privileged (container "node-hello" must not set securityContext.privileged=true), allowPrivilegeEscalation != false (container "node-hello" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "node-hello" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or container "node-hello" must set securityContext.runAsNonRoot=true), runAsUser=0 (container "node-hello" must not set runAsUser=0), seccompProfile (pod or container "node-hello" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")

4. Check the query in Openshift console (prometheus) metrics again
sum(increase(pod_security_evaluations_total{resource="pod"}[1m])) by(decision,mode,policy_level)

values should change to reflect denial to create pod
Moving to verified

Comment 9 Xingxing Xia 2022-07-01 14:35:27 UTC

You should verify with alert fired. ` sum(increase(pod_security_evaluations_total{resource="pod"}[1m])) by(decision,mode,policy_level) ` is only tips to help understand whether the alert's condition satisfaction includes your test operation's triggering.

Comment 10 Yash Tripathi 2022-07-04 09:50:01 UTC

Verified on 4.11.0-0.nightly-2022-06-30-005428

1. Create and label a namespace with pod-security.kubernetes.io/enforce=restricted
$ oc new-project testproj
$ oc label ns testproj pod-security.kubernetes.io/enforce=restricted

2. Create a sample pod that should get denied due to scc restrictions
$ oc create -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
  name: testpod
spec:
  containers:
  - image: quay.io/openshifttest/hello-openshift:openshift
    name: node-hello
    securityContext:
      privileged: true
      runAsUser: 0
EOF
Error from server (Forbidden): error when creating "STDIN": pods "testpod" is forbidden: violates PodSecurity "restricted:latest": privileged (container "node-hello" must not set securityContext.privileged=true), allowPrivilegeEscalation != false (container "node-hello" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "node-hello" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or container "node-hello" must set securityContext.runAsNonRoot=true), runAsUser=0 (container "node-hello" must not set runAsUser=0), seccompProfile (pod or container "node-hello" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")

3. Check the Alerts in Openshift console

Actual
PodSecurityViolation alert is being fired

Expected
PodSecurityViolation alert should be in a firing state
Moving to verified

Comment 11 errata-xmlrpc 2022-08-10 11:19:11 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: OpenShift Container Platform 4.11.0 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:5069