Bug 1277092
| Summary: | Owner can't get container information via downward-api volume plugin because of "Permission denied" | ||
|---|---|---|---|
| Product: | OKD | Reporter: | Qixuan Wang <qixuan.wang> |
| Component: | Containers | Assignee: | Paul Morie <pmorie> |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Chao Yang <chaoyang> |
| Severity: | medium | Docs Contact: | |
| Priority: | medium | ||
| Version: | 3.x | CC: | aos-bugs, dmcphers, mmccomas, pmorie, qixuan.wang |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2016-05-12 17:09:39 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
To create privileged pods, the creating user must be added to the privileged security context constraints object: oadm policy add-scc-to-user privileged your_username (In reply to Jordan Liggitt from comment #1) > To create privileged pods, the creating user must be added to the privileged > security context constraints object: > > oadm policy add-scc-to-user privileged your_username Yeah, `oadm policy add-scc-to-user privileged your_username` really work. But downloadapi volume can work if and only if privileged=true? And since downloadapi volume is created by origin, so 700 is expected? Yesterday we added user via "oc edit scc" under privileged section and found the problem. I think "oadm policy add-scc-to-user privileged your_username" plays the same role. Today I couldn't reproduce the problem although the new environment has the same version. It's weird. (In reply to Jordan Liggitt from comment #1) > To create privileged pods, the creating user must be added to the privileged > security context constraints object: > > oadm policy add-scc-to-user privileged your_username And looks like the directory is created with 777, but do not know why it is 700. https://github.com/openshift/origin/blob/master/Godeps/_workspace/src/k8s.io/kubernetes/pkg/volume/downwardapi/downwardapi.go#L285 Paul, any insight on the gid or downward API usage without making the container privileged? The downward api plugin is a read only volume. Read only volumes were not being altered by the ownership and selinux management code. That is being refactored to provide a better decision plane for read only volumes that may be managed by the kubelet. Upstream PR is https://github.com/kubernetes/kubernetes/pull/16614. Should be fixed now. Need rebase to origin to verify it. Checked today's build devenv-fedora_3134(openshift v1.1-764-g7e08c95, kubernetes v1.1.0-origin-1107-g4c8e6f4, etcd 2.1.2), the upstream code has not been rebased into origin, so moved the bug to ASSIGNED temporarily. Sorry about that, should be fixed now that the rebase has landed. Tested on devevn-rhel_3335(openshift v1.1.1-440-g79b4613, kubernetes v1.2.0-alpha.4-851-g4a65fa1, etcd 2.2.2) which the upstream PR#16614 has been in it. Come into a privileged container as root so there is not permission problem. But enter into a non-privilege container of as a non-root role, the permission problem is still there. Steps are listed bellow. Could you help me to confirm we are in the same page? Thanks. [root@ip-172-18-12-137 ~]# oc get pod NAME READY STATUS RESTARTS AGE pod-dapi-volume-not-privileged 1/1 Running 0 1m pod-dapi-volume-privileged 1/1 Running 0 2s [root@ip-172-18-12-137 ~]# oc exec -it pod-dapi-volume-not-privileged sh / $ ls -laR /etc /etc: total 16 drwxrwxrwt 3 0 0 220 Feb 5 06:58 . drwxr-xr-x 17 0 0 4096 Feb 5 06:58 .. drwx------ 2 0 0 120 Feb 5 06:58 ..2016_02_05_01_58_19604757689 lrwxrwxrwx 1 0 0 30 Feb 5 06:58 ..downwardapi -> ..2016_02_05_01_58_19604757689 lrwxrwxrwx 1 0 0 25 Feb 5 06:58 annotations -> ..downwardapi/annotations -rw-r--r-- 1 0 0 31 Feb 5 06:58 hostname -rw-r--r-- 1 0 0 225 Feb 5 06:58 hosts lrwxrwxrwx 1 0 0 20 Feb 5 06:58 labels -> ..downwardapi/labels lrwxrwxrwx 1 0 0 18 Feb 5 06:58 name -> ..downwardapi/name lrwxrwxrwx 1 0 0 23 Feb 5 06:58 namespace -> ..downwardapi/namespace -rw-r--r-- 1 0 0 140 Feb 5 06:58 resolv.conf /etc/..2016_02_05_01_58_19604757689: ls: can't open '/etc/..2016_02_05_01_58_19604757689': Permission denied total 0 [root@ip-172-18-12-137 ~]# oc exec -it pod-dapi-volume-privileged sh / # ls -laR /etc /etc: total 16 drwxrwxrwt 3 0 0 220 Feb 5 06:59 . drwxr-xr-x 17 0 0 4096 Feb 5 07:01 .. drwx------ 2 0 0 120 Feb 5 06:59 ..2016_02_05_01_59_29346186948 lrwxrwxrwx 1 0 0 30 Feb 5 06:59 ..downwardapi -> ..2016_02_05_01_59_29346186948 lrwxrwxrwx 1 0 0 25 Feb 5 06:59 annotations -> ..downwardapi/annotations -rw-r--r-- 1 0 0 27 Feb 5 06:59 hostname -rw-r--r-- 1 0 0 221 Feb 5 06:59 hosts lrwxrwxrwx 1 0 0 20 Feb 5 06:59 labels -> ..downwardapi/labels lrwxrwxrwx 1 0 0 18 Feb 5 06:59 name -> ..downwardapi/name lrwxrwxrwx 1 0 0 23 Feb 5 06:59 namespace -> ..downwardapi/namespace -rw-r--r-- 1 0 0 140 Feb 5 06:59 resolv.conf /etc/..2016_02_05_01_59_29346186948: total 16 drwx------ 2 0 0 120 Feb 5 06:59 . drwxrwxrwt 3 0 0 220 Feb 5 06:59 .. -rw-r--r-- 1 0 0 157 Feb 5 06:59 annotations -rw-r--r-- 1 0 0 35 Feb 5 06:59 labels -rw-r--r-- 1 0 0 26 Feb 5 06:59 name -rw-r--r-- 1 0 0 6 Feb 5 06:59 namespace You need to have fsGroup set in order for this to work with a non-root UID. You will need to use the 'anyuid' SCC or 'privileged' SCC, and put the fsGroup into the pod spec:
apiVersion: v1
kind: Pod
metadata:
name: test-pod
spec:
securityContext:
fsGroup: 1234
Tested on devenv-rhel_3422(openshift v1.1.2-258-gda69009, kubernetes v1.2.0-origin, etcd 2.2.2+git), the bug has been fixed, thanks for your help.
Here are verification steps.
# cat pod-dapi-volume.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-dapi-volume
labels:
region: r1
zone: z11
rack: a111
annotations:
build: one
builder: qe-one
spec:
securityContext:
fsGroup: 1234
containers:
- name: client-container
image: gcr.io/google_containers/busybox
command: ["sh", "-c", "while true; do if [[ -e /etc/labels ]]; then cat /etc/labels; fi; if [[ -e /etc/annotations ]]; then cat /etc/annotations; fi; sleep 5; done"]
securityContext:
privileged: false
volumeMounts:
- name: podinfo
mountPath: /etc
readOnly: false
volumes:
- name: podinfo
downwardAPI:
items:
- path: "labels"
fieldRef:
fieldPath: metadata.labels
- path: "annotations"
fieldRef:
fieldPath: metadata.annotations
- path: "name"
fieldRef:
fieldPath: metadata.name
- path: "namespace"
fieldRef:
fieldPath: metadata.namespace
[root@ip-172-18-1-152 ~]# oc exec -it pod-dapi-volume sh
/ $ ls -laR /etc
/etc:
total 16
drwxrwsrwt 3 0 1234 220 Feb 14 06:53 .
drwxr-xr-x 17 0 0 4096 Feb 14 06:53 ..
drwxrwsrwx 2 0 1234 120 Feb 14 06:53 ..2016_02_14_01_53_09964771296
lrwxrwxrwx 1 0 0 30 Feb 14 06:53 ..downwardapi -> ..2016_02_14_01_53_09964771296
lrwxrwxrwx 1 0 0 25 Feb 14 06:53 annotations -> ..downwardapi/annotations
-rw-r--r-- 1 0 0 16 Feb 14 06:53 hostname
-rw-r--r-- 1 0 0 210 Feb 14 06:53 hosts
lrwxrwxrwx 1 0 0 20 Feb 14 06:53 labels -> ..downwardapi/labels
lrwxrwxrwx 1 0 0 18 Feb 14 06:53 name -> ..downwardapi/name
lrwxrwxrwx 1 0 0 23 Feb 14 06:53 namespace -> ..downwardapi/namespace
-rw-r--r-- 1 0 0 139 Feb 14 06:53 resolv.conf
/etc/..2016_02_14_01_53_09964771296:
total 16
drwxrwsrwx 2 0 1234 120 Feb 14 06:53 .
drwxrwsrwt 3 0 1234 220 Feb 14 06:53 ..
-rwxrwsrwx 1 0 1234 157 Feb 14 06:53 annotations
-rwxrwsrwx 1 0 1234 35 Feb 14 06:53 labels
-rwxrwsrwx 1 0 1234 15 Feb 14 06:53 name
-rwxrwsrwx 1 0 1234 6 Feb 14 06:53 namespace
|
Description of problem: I want to enter a container as root, so I create a privileged pod with downward API volume plug-in, the privileged containers are not allowed to be created. So I have to set "privileged" as "false", the container could run, but container information isn't able to get because the owner can't read the file /etc/..2015_11_02_15_53_43215191314 although it has the permission. Plus, privileged pod without downward-api volume can be created. Version-Release number of selected component (if applicable): openshift v3.0.2.903-114-g2849767 kubernetes v1.2.0-alpha.1-1107-g4c8e6f4 etcd 2.1.2 How reproducible: Always Steps to Reproduce: 1. Create a privileged pod that consumes its labele/annotations via the downward API volume. # vi pod-dapi-volume.yaml apiVersion: v1 kind: Pod metadata: name: pod-dapi-volume labels: region: r1 zone: z11 rack: a111 annotations: build: one builder: qe-one spec: containers: - name: client-container image: gcr.io/google_containers/busybox command: ["sh", "-c", "while true; do if [[ -e /etc/labels ]]; then cat /etc/labels; fi; if [[ -e /etc/annotations ]]; then cat /etc/annotations; fi; sleep 5; done"] securityContext: privileged: true volumeMounts: - name: podinfo mountPath: /etc readOnly: false volumes: - name: podinfo downwardAPI: items: - path: "labels" fieldRef: fieldPath: metadata.labels - path: "annotations" fieldRef: fieldPath: metadata.annotations - path: "name" fieldRef: fieldPath: metadata.name - path: "namespace" fieldRef: fieldPath: metadata.namespace # oc create -f pod-dapi-volume.yaml 2. Set "privileged" as "false" and then create the pod. 3. Enter into the container and check container info # oc exec -it $pod_name sh # ls -laR /etc Actual results: 1. [root@dhcp-136-118 testv3]# oc create -f pod-dapi-volume.yaml Error from server: error when creating "pod-dapi-volume.yaml": Pod "pod-dapi-volume-new" is forbidden: unable to validate against any security context constraint: [provider restricted: .spec.containers[0].securityContext.privileged: invalid value 'true', Details: Privileged containers are not allowed] 3. [root@dhcp-136-118 testv3]# oc exec -it pod-dapi-volume-new sh / $ id uid=1000340000 gid=0 groups=1000340000 / $ ls -laR /etc /etc: total 16 drwxrwxrwt 3 0 0 220 Nov 2 07:53 . drwxr-xr-x 17 0 0 4096 Nov 2 07:53 .. drwx------ 2 0 0 120 Nov 2 07:53 ..2015_11_02_15_53_43215191314 lrwxrwxrwx 1 0 0 30 Nov 2 07:53 ..downwardapi -> ..2015_11_02_15_53_43215191314 lrwxrwxrwx 1 0 0 25 Nov 2 07:53 annotations -> ..downwardapi/annotations -rw-r--r-- 1 0 0 20 Nov 2 07:53 hostname -rw-r--r-- 1 0 0 1871 Nov 2 07:53 hosts lrwxrwxrwx 1 0 0 20 Nov 2 07:53 labels -> ..downwardapi/labels lrwxrwxrwx 1 0 0 18 Nov 2 07:53 name -> ..downwardapi/name lrwxrwxrwx 1 0 0 23 Nov 2 07:53 namespace -> ..downwardapi/namespace -rw-r--r-- 1 0 0 166 Nov 2 07:53 resolv.conf /etc/..2015_11_02_15_53_43215191314: ls: can't open '/etc/..2015_11_02_15_53_43215191314': Permission denied total 0 Expected results: [root@ip-172-18-3-115 work]# oc exec -it pod-dapi-volume sh / # ls -laR /etc /etc: total 16 drwxrwxrwt 3 0 0 220 Oct 21 11:28 . drwxr-xr-x 17 0 0 4096 Oct 21 11:31 .. drwx------ 2 0 0 120 Oct 21 11:28 ..2015_10_21_11_28_57085975213 lrwxrwxrwx 1 0 0 30 Oct 21 11:28 ..downwardapi -> ..2015_10_21_11_28_57085975213 lrwxrwxrwx 1 0 0 25 Oct 21 11:28 annotations -> ..downwardapi/annotations -rw-r--r-- 1 0 0 16 Oct 21 11:28 hostname -rw-r--r-- 1 0 0 2913 Oct 21 11:28 hosts lrwxrwxrwx 1 0 0 20 Oct 21 11:28 labels -> ..downwardapi/labels lrwxrwxrwx 1 0 0 18 Oct 21 11:28 name -> ..downwardapi/name lrwxrwxrwx 1 0 0 23 Oct 21 11:28 namespace -> ..downwardapi/namespace -rw-r--r-- 1 0 0 139 Oct 21 11:28 resolv.conf /etc/..2015_10_21_11_28_57085975213: total 16 drwx------ 2 0 0 120 Oct 21 11:28 . drwxrwxrwt 3 0 0 220 Oct 21 11:28 .. -rw-r--r-- 1 0 0 152 Oct 21 11:28 annotations -rw-r--r-- 1 0 0 35 Oct 21 11:28 labels -rw-r--r-- 1 0 0 15 Oct 21 11:28 name -rw-r--r-- 1 0 0 6 Oct 21 11:28 namespace Additional info: