Bug 2057545

Summary: machine-config-operator pod is trying to use custom SecurityContextConstraint deployed by OpenShift Sandboxed containers Operator
Product: OpenShift Container Platform Reporter: Pradipta Banerjee <prbanerj>
Component: Machine Config OperatorAssignee: Yuval Kashtan <ykashtan>
Machine Config Operator sub component: Machine Config Operator QA Contact: Silvia Serafini <sserafin>
Status: CLOSED NOTABUG Docs Contact:
Severity: high    
Priority: unspecified CC: aos-bugs, cmeadors, fan-wxa, gkurz, jfreiman, josantos, kgarriso, mkalinin, mkrejci, rh-container, sgrunert, sserafin, wking, ykashtan
Version: 4.10-rc3   
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-06-01 15:54:10 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Pradipta Banerjee 2022-02-23 15:30:20 UTC 
Description of problem:
In an OpenShift environment with custom SCC (sandboxed-containers-operator-scc) created by sandboxed containers operator, the machine-config-operator (MCO) pod tries to use the custom SCC instead of using hostmount-anyuid SCC and errors out.
This issue is noticed when the MCO pod is restarted.



Version-Release number of MCO (Machine Config Operator) (if applicable):

Platform (AWS, VSphere, Metal, etc.): Metal

Are you certain that the root cause of the issue being reported is the MCO (Machine Config Operator)?
(Y/N/Not sure): 

How reproducible:

In one of the environments, the issue occurs frequently whenever the MCO pod restarts, and in another one we have failed to recreate the issue.
The difference between both the environments are the additional operators installed. The environment where we observe the issue has OpenShift Virtualization and Noobaa operators installed.

Steps to Reproduce:
1. Install Sandboxed Containers Operator, this creates the custom SCC
2. Install CNV, this restarts the master nodes 
3. Issue observed with MCO pod

Actual results:
Warning  Failed  15m (x3392 over 12h)  kubelet  (combined from similar events): Error: container create failed: time="2022-02-15T05:58:11Z" level=error msg="runc create failed: unable to start container process: error during container init: write /proc/self/attr/keycreate: invalid argument"

Expected results:
MCO POD should still be using hostmount-anyuid SCC and not the custom SCC

Additional info:
The issue is because the MCO container process tries to use the osc_monitor.process SeLinux type which is not present on the master node. This SeLinux type is part of the custom SCC and is installed only on specific worker nodes by the Sandboxed containers operator
Comment 2 Pradipta Banerjee 2022-03-01 11:01:35 UTC 
Custom SCC details
```
allowHostDirVolumePlugin: true
allowHostIPC: false
allowHostNetwork: false
allowHostPID: false
allowHostPorts: false
allowPrivilegeEscalation: false
allowPrivilegedContainer: false
allowedCapabilities:
- DAC_READ_OVERRIDE
apiVersion: security.openshift.io/v1
defaultAddCapabilities: null
fsGroup:
  type: RunAsAny
groups: []
kind: SecurityContextConstraints
metadata:
  name: sandboxed-containers-operator-scc  
priority: null
readOnlyRootFilesystem: false
requiredDropCapabilities:
- MKNOD
- FSETID
- KILL
- FOWNER
runAsUser:
  type: MustRunAsNonRoot
seLinuxContext:
  seLinuxOptions:
    type: osc_monitor.process
  type: MustRunAs
supplementalGroups:
  type: RunAsAny
users:
- system:serviceaccount:openshift-sandboxed-containers-operator:monitor
volumes:
- '*'
```

Simple recreate steps
```
oc  scale deployments/machine-config-operator -n openshift-machine-config-operator --replicas=0
oc  scale deployments/machine-config-operator -n openshift-machine-config-operator --replicas=1
oc get pods -n openshift-machine-config-operator -l k8s-app=machine-config-operator -o yaml | grep scc
      openshift.io/scc: sandboxed-containers-operator-scc
```
Comment 3 Cameron Meadors 2022-03-01 14:31:10 UTC 
Any update on this?
Comment 14 Francesco Giudici 2022-03-22 10:33:25 UTC 
*** Bug 2065085 has been marked as a duplicate of this bug. ***
Comment 22 Red Hat Bugzilla 2023-09-15 01:52:08 UTC 
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 365 days