Bug 1977920 - Pod fails to run when a custom SCC with a specific set of volumes is used
Summary: Pod fails to run when a custom SCC with a specific set of volumes is used
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: apiserver-auth
Version: 4.8
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: ---
: 4.9.0
Assignee: Maru Newby
QA Contact: liyao
URL:
Whiteboard:
Depends On:
Blocks: 1977924
TreeView+ depends on / blocked
 
Reported: 2021-06-30 17:39 UTC by Fabian Deutsch
Modified: 2021-10-18 17:37 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
Clone Of:
: 1977924 (view as bug list)
Environment:
Last Closed: 2021-10-18 17:37:23 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift apiserver-library-go pull 52 0 None open Bug 1977920: Ensure scc compatibility with BoundServiceAccountTokenVolume 2021-07-01 08:31:31 UTC
Github openshift kubernetes pull 841 0 None open Bug 1977920: Ensure scc compatibility with BoundServiceAccountTokenVolume 2021-07-01 09:00:59 UTC
Red Hat Product Errata RHSA-2021:3759 0 None None None 2021-10-18 17:37:50 UTC

Description Fabian Deutsch 2021-06-30 17:39:18 UTC
Description of problem:
A pod using a SA which has only access to an SCC with a specific (defined) set of volumes will fail to launch.

This is because 1.21 is moving to bound tokens which are using projected volumes.

Version-Release number of selected component (if applicable):


How reproducible:
Always 

Steps to Reproduce:
1. SA with custom SCC which has secrets + hostpath is set
2. Create pod with SA
3. Create pod

Actual results:
Pod fails to launch

CNV HPP fails to launch, see bug #1977179

Expected results:
Pod launches

Additional info:
This is addressed by adding th projected  volume to the custom SCC.
Hwoeveer, this is not possible in 4.7 as the volume type did not exist back then.

Comment 5 liyao 2021-07-02 03:25:52 UTC
Tested in cluster 4.9.0-0.nightly-2021-07-02-002856
1. create new SCC without 'projected' volume type
$ oc get scc hostaccess -o yaml > myscc.yaml
$ sed -i -e "/- projected/d" -e "s/hostaccess/myscc/" myscc.yaml
$ oc create -f myscc.yaml

2. add the new SCC to SA default
$ oc login -u testuser
$ oc new-project test
$ oc adm policy add-scc-to-user myscc -z default -n test --context admin

3. create pod and check whether pod can be launched successfully
$ cat > pod.yaml << EOF
apiVersion: v1
kind: Pod
metadata:
  name: testpod
spec:
  containers:
  - image: quay.io/openshifttest/hello-pod:latest
    name: testpod
    volumeMounts:
    - mountPath: /host
      name: myvolume
  volumes:
  - hostPath:
      path: /
    name: myvolume
EOF
$ oc create -f pod.yaml 

when using old build, error is returned
Error from server (Forbidden): error when creating "pod.yaml": pods "testpod" is forbidden: unable to validate against any security context constraint: [provider "anyuid": Forbidden: not usable by user or serviceaccount, spec.volumes[0]: Invalid value: "hostPath": hostPath volumes are not allowed to be used, provider "nonroot": Forbidden: not usable by user or serviceaccount, provider "hostmount-anyuid": Forbidden: not usable by user or serviceaccount, provider "machine-api-termination-handler": Forbidden: not usable by user or serviceaccount, provider "hostnetwork": Forbidden: not usable by user or serviceaccount, provider "hostaccess": Forbidden: not usable by user or serviceaccount, spec.volumes[1]: Invalid value: "projected": projected volumes are not allowed to be used, provider "node-exporter": Forbidden: not usable by user or serviceaccount, provider "privileged": Forbidden: not usable by user or serviceaccount]

when using new build, pod can be launched with running status
$ oc get pods -n test
NAME      READY   STATUS    RESTARTS   AGE
testpod   1/1     Running   0          33m

Comment 6 liyao 2021-07-02 06:15:32 UTC
add test whether projected volume type is added or not when 'secret' and 'projected' volume type are both removed from the custom SCC yaml
$ oc get scc hostaccess -o yaml > myscc1.yaml
$ sed -i -e "/- projected/d" -e "/- secret/d" -e "s/hostaccess/myscc1/" myscc1.yaml
$ oc create -f myscc1.yaml 
securitycontextconstraints.security.openshift.io/myscc1 created
$ oc get scc myscc1 -o yaml
/****snipped****/
volumes:
- configMap
- downwardAPI
- emptyDir
- hostPath
- persistentVolumeClaim

check the result, projected volume type is not added as expected

Comment 7 Maru Newby 2021-07-02 15:43:28 UTC
(In reply to liyao from comment #6)
> add test whether projected volume type is added or not when 'secret' and
> 'projected' volume type are both removed from the custom SCC yaml
> $ oc get scc hostaccess -o yaml > myscc1.yaml
> $ sed -i -e "/- projected/d" -e "/- secret/d" -e "s/hostaccess/myscc1/"
> myscc1.yaml
> $ oc create -f myscc1.yaml 
> securitycontextconstraints.security.openshift.io/myscc1 created
> $ oc get scc myscc1 -o yaml
> /****snipped****/
> volumes:
> - configMap
> - downwardAPI
> - emptyDir
> - hostPath
> - persistentVolumeClaim
> 
> check the result, projected volume type is not added as expected

What is this test for? I don't believe there was ever an expectation of automatic addition of 'projected' to an scc's volume sources.

Comment 8 Xingxing Xia 2021-07-05 03:46:25 UTC
(In reply to Maru Newby from comment #7)
> (In reply to liyao from comment #6)
Maru Newby, you're right. I looked at the PR's comment, thought the fix was to automatic support 'projected' by adding it in SCC object when 'secret' is present, thus did not look into the PR code of multiple files. Now looking into pkg/securitycontextconstraints/sccmatching/provider.go & pkg/securitycontextconstraints/util/util.go , got to know original thought is not true after checking the implemented changes in ValidatePodSecurityContext, allowsVolumeType, allowedVolumes functions and IsOnlyServiceAccountTokenSources.

Comment 9 Xingxing Xia 2021-07-05 06:03:07 UTC
Thus a minor negative test for comment 6 would be: using steps of comment 5 but using SCC of comment 6, the pod would still not be run, as same as before (i.e. this bevior is not affected by the PRs)

Comment 10 Maru Newby 2021-07-07 17:51:23 UTC
No doc update required - this was a bug introduced and fixed in 4.8 prior to release.

Comment 13 errata-xmlrpc 2021-10-18 17:37:23 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.9.0 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:3759


Note You need to log in before you can comment on or make changes to this bug.