Bug 2133543

Summary: [pod security violation audit] Audit violation in "kube-rbac-proxy" container should be fixed
Product: Container Native Virtualization (CNV) Reporter: SATHEESARAN <sasundar>
Component: NetworkingAssignee: Petr Horáček <phoracek>
Status: CLOSED ERRATA QA Contact: Yossi Segev <ysegev>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 4.11.1CC: dcadzow, ellorent, kmajcher, oshoval, sbulage, stirabos, ysegev
Target Milestone: ---   
Target Release: 4.12.0   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: bundle v4.12.0-769 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-01-24 13:41:26 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2089744    

Description SATHEESARAN 2022-10-10 19:51:32 UTC
Description of problem:
-----------------------
Test run[1] that looks for 'pod security violation entries' in audit logs,against 4.11.1-20, found few audit violation.

[1] - https://main-jenkins-csb-cnvqe.apps.ocp-c1.prod.psi.redhat.com/view/cnv-tests%20runner/job/cnv-tests-runner/4297/consoleFull.

This bug is to fix violation in 'kube-rbac-proxy' container.

<snip>
'pod-security.kubernetes.io/audit-violations': 'would violate PodSecurity "restricted:latest": allowPrivilegeEscalation != false (containers "manager", "kube-rbac-proxy" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (containers "manager", "kube-rbac-proxy" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or containers "manager", "kube-rbac-proxy" must set securityContext.runAsNonRoot=true), seccompProfile (pod or containers "manager", "kube-rbac-proxy" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")'}}
</snip>

Version-Release number of selected component (if applicable):
-------------------------------------------------------------
4.11.1-20

How reproducible:
-----------------
Always

Expected results:
-----------------
No audit-violation to be found

Comment 1 Petr Horáček 2022-10-11 12:58:29 UTC
I believe that this was addressed via https://github.com/kubevirt/cluster-network-addons-operator/pull/1404 in v4.11.1-29. The problem is that errata is stuck on a build that is two weeks old.

Comment 2 Yossi Segev 2022-10-24 18:48:55 UTC
Still happens on
CNV (HCO bundle) v4.11.1-49 (IIB: 346131)
ose-kube-rbac-proxy: v4.11


Reproduction scenario:
I Ran the same tests that was run in the original bug scenario (https://main-jenkins-csb-cnvqe.apps.ocp-c1.prod.psi.redhat.com/view/cnv-tests%20runner/job/cnv-tests-runner/4297/consoleFull), with https://code.engineering.redhat.com/gerrit/c/cnv-tests/+/430017, which was also used in the original scenario, cherry-picked.
* I ran from the cluster executor rather than from the Jenkins test job.

$ poetry run pytest -svv -o log_cli=true tests/install_upgrade_operators/pod_security/test_pod_security_audit_log.py --bugzilla --jira --junit-xml xunit_results.xml --tc=region:USA --tb=native --cluster-sanity-skip-storage-check

Result (from the test output log):
'pod-security.kubernetes.io/audit-violations': 'would violate PodSecurity "restricted:latest": allowPrivilegeEscalation != false (containers "manager", "kube-rbac-proxy" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (containers "manager", "kube-rbac-proxy" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or containers "manager", "kube-rbac-proxy" must set securityContext.runAsNonRoot=true), seccompProfile (pod or containers "manager", "kube-rbac-proxy" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")'

Comment 3 Quique Llorente 2022-10-25 09:02:28 UTC
Upstream for 4.11 has the proper fixes https://github.com/kubevirt/cluster-network-addons-operator/blob/v0.76.2/manifests/cluster-network-addons/0.76.2/operator.yaml#L207-L217.

@ysegev Can you dump the cluster-network-addons-operator pod yaml ?

Comment 5 Quique Llorente 2022-10-25 10:38:50 UTC
This workload is fine it has the proper scc

openshift.io/scc: restricted-v2

We are facing a false negative here also the yaml has the proper security context thing (that's why it has proper scc)

Comment 6 Yossi Segev 2022-10-25 13:50:11 UTC
Hi Sas,

Quique suspects that the audit violations, which are reported here, are actually related to old pods from previous installation of CNV on the cluster, and not from the current CNV.
Is there a way of running the test without auditing old remainders?

Thanks,
Yossi

Comment 8 SATHEESARAN 2022-10-31 10:29:28 UTC
(In reply to Yossi Segev from comment #6)
> Hi Sas,
> 
> Quique suspects that the audit violations, which are reported here, are
> actually related to old pods from previous installation of CNV on the
> cluster, and not from the current CNV.
> Is there a way of running the test without auditing old remainders?
> 
> Thanks,
> Yossi

Hi Yossi,

I am unaware of any such mechanism to ignore the older audits.
Very naive thinking is to trigger the test with the new cluster and make sure not to
observe the pod violation in audit log

Comment 9 Yossi Segev 2022-11-10 10:58:19 UTC
I have just run this test on a fresh cluster (with CNV 4.11.1), and the same failure still occurs.

Comment 10 Petr Horáček 2022-11-10 14:06:27 UTC
After discussing this issue offline, we came to a conclusion that while the component should be safe with the default security context it gets from OpenShift, we need to explicitly set it to silence the audit log.

Comment 12 Satyajit Bulage 2022-12-22 11:46:10 UTC
Followed same steps from comment 2.

Still able to see the violation:

'pod-security.kubernetes.io/audit-violations': 'would violate PodSecurity "restricted:latest": allowPrivilegeEscalation != false (containers "manager", "kube-rbac-proxy" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (containers "manager", "kube-rbac-proxy" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or containers "manager", "kube-rbac-proxy" must set securityContext.runAsNonRoot=true), seccompProfile (pod or containers "manager", "kube-rbac-proxy" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")'

Installed CNAO version: v4.11.2-1
CNV version: 4.11.2-10
CNI plugin: 4.11.2-1

Failed ON_QA.

Comment 13 Yossi Segev 2023-01-02 17:32:35 UTC
Verified by running the same scenario from comment #2.

OCP version: 4.12.0-rc.2
CNV 4.12.0-769

Comment 15 Yossi Segev 2023-01-23 15:00:46 UTC
(In reply to Yossi Segev from comment #13)
> Verified by running the same scenario from comment #2.
> 
> OCP version: 4.12.0-rc.2
> CNV 4.12.0-769

Also verified on 2 upgraded clusters:
OCP+CNV 4.11->4.12
OCP+CNV 4.10->4.12 (EUS upgrade).

Comment 17 errata-xmlrpc 2023-01-24 13:41:26 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: OpenShift Virtualization 4.12.0 Images security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2023:0408