Bug 1977920

Summary: Pod fails to run when a custom SCC with a specific set of volumes is used
Product: OpenShift Container Platform Reporter: Fabian Deutsch <fdeutsch>
Component: apiserver-authAssignee: Maru Newby <mnewby>
Status: CLOSED ERRATA QA Contact: liyao
Severity: urgent Docs Contact:
Priority: urgent    
Version: 4.8CC: alitke, aos-bugs, mfojtik, sttts, surbania, xxia, ycui
Target Milestone: ---Keywords: FastFix, Regression
Target Release: 4.9.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of:
: 1977924 (view as bug list) Environment:
Last Closed: 2021-10-18 17:37:23 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1977924    

Description Fabian Deutsch 2021-06-30 17:39:18 UTC
Description of problem:
A pod using a SA which has only access to an SCC with a specific (defined) set of volumes will fail to launch.

This is because 1.21 is moving to bound tokens which are using projected volumes.

Version-Release number of selected component (if applicable):


How reproducible:
Always 

Steps to Reproduce:
1. SA with custom SCC which has secrets + hostpath is set
2. Create pod with SA
3. Create pod

Actual results:
Pod fails to launch

CNV HPP fails to launch, see bug #1977179

Expected results:
Pod launches

Additional info:
This is addressed by adding th projected  volume to the custom SCC.
Hwoeveer, this is not possible in 4.7 as the volume type did not exist back then.

Comment 5 liyao 2021-07-02 03:25:52 UTC
Tested in cluster 4.9.0-0.nightly-2021-07-02-002856
1. create new SCC without 'projected' volume type
$ oc get scc hostaccess -o yaml > myscc.yaml
$ sed -i -e "/- projected/d" -e "s/hostaccess/myscc/" myscc.yaml
$ oc create -f myscc.yaml

2. add the new SCC to SA default
$ oc login -u testuser
$ oc new-project test
$ oc adm policy add-scc-to-user myscc -z default -n test --context admin

3. create pod and check whether pod can be launched successfully
$ cat > pod.yaml << EOF
apiVersion: v1
kind: Pod
metadata:
  name: testpod
spec:
  containers:
  - image: quay.io/openshifttest/hello-pod:latest
    name: testpod
    volumeMounts:
    - mountPath: /host
      name: myvolume
  volumes:
  - hostPath:
      path: /
    name: myvolume
EOF
$ oc create -f pod.yaml 

when using old build, error is returned
Error from server (Forbidden): error when creating "pod.yaml": pods "testpod" is forbidden: unable to validate against any security context constraint: [provider "anyuid": Forbidden: not usable by user or serviceaccount, spec.volumes[0]: Invalid value: "hostPath": hostPath volumes are not allowed to be used, provider "nonroot": Forbidden: not usable by user or serviceaccount, provider "hostmount-anyuid": Forbidden: not usable by user or serviceaccount, provider "machine-api-termination-handler": Forbidden: not usable by user or serviceaccount, provider "hostnetwork": Forbidden: not usable by user or serviceaccount, provider "hostaccess": Forbidden: not usable by user or serviceaccount, spec.volumes[1]: Invalid value: "projected": projected volumes are not allowed to be used, provider "node-exporter": Forbidden: not usable by user or serviceaccount, provider "privileged": Forbidden: not usable by user or serviceaccount]

when using new build, pod can be launched with running status
$ oc get pods -n test
NAME      READY   STATUS    RESTARTS   AGE
testpod   1/1     Running   0          33m

Comment 6 liyao 2021-07-02 06:15:32 UTC
add test whether projected volume type is added or not when 'secret' and 'projected' volume type are both removed from the custom SCC yaml
$ oc get scc hostaccess -o yaml > myscc1.yaml
$ sed -i -e "/- projected/d" -e "/- secret/d" -e "s/hostaccess/myscc1/" myscc1.yaml
$ oc create -f myscc1.yaml 
securitycontextconstraints.security.openshift.io/myscc1 created
$ oc get scc myscc1 -o yaml
/****snipped****/
volumes:
- configMap
- downwardAPI
- emptyDir
- hostPath
- persistentVolumeClaim

check the result, projected volume type is not added as expected

Comment 7 Maru Newby 2021-07-02 15:43:28 UTC
(In reply to liyao from comment #6)
> add test whether projected volume type is added or not when 'secret' and
> 'projected' volume type are both removed from the custom SCC yaml
> $ oc get scc hostaccess -o yaml > myscc1.yaml
> $ sed -i -e "/- projected/d" -e "/- secret/d" -e "s/hostaccess/myscc1/"
> myscc1.yaml
> $ oc create -f myscc1.yaml 
> securitycontextconstraints.security.openshift.io/myscc1 created
> $ oc get scc myscc1 -o yaml
> /****snipped****/
> volumes:
> - configMap
> - downwardAPI
> - emptyDir
> - hostPath
> - persistentVolumeClaim
> 
> check the result, projected volume type is not added as expected

What is this test for? I don't believe there was ever an expectation of automatic addition of 'projected' to an scc's volume sources.

Comment 8 Xingxing Xia 2021-07-05 03:46:25 UTC
(In reply to Maru Newby from comment #7)
> (In reply to liyao from comment #6)
Maru Newby, you're right. I looked at the PR's comment, thought the fix was to automatic support 'projected' by adding it in SCC object when 'secret' is present, thus did not look into the PR code of multiple files. Now looking into pkg/securitycontextconstraints/sccmatching/provider.go & pkg/securitycontextconstraints/util/util.go , got to know original thought is not true after checking the implemented changes in ValidatePodSecurityContext, allowsVolumeType, allowedVolumes functions and IsOnlyServiceAccountTokenSources.

Comment 9 Xingxing Xia 2021-07-05 06:03:07 UTC
Thus a minor negative test for comment 6 would be: using steps of comment 5 but using SCC of comment 6, the pod would still not be run, as same as before (i.e. this bevior is not affected by the PRs)

Comment 10 Maru Newby 2021-07-07 17:51:23 UTC
No doc update required - this was a bug introduced and fixed in 4.8 prior to release.

Comment 13 errata-xmlrpc 2021-10-18 17:37:23 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.9.0 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:3759