Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1475137

Summary:	Permission denied for secrets mounting as files
Product:	OpenShift Container Platform	Reporter:	Takayoshi Tanaka <tatanaka>
Component:	Node	Assignee:	Andrew McDermott <amcdermo>
Status:	CLOSED DUPLICATE	QA Contact:	DeShuai Ma <dma>
Severity:	medium	Docs Contact:
Priority:	unspecified
Version:	3.6.0	CC:	aos-bugs, erich, jokerman, jpazdziora, mmccomas, nschuetz, rhowe, sjenning, tkimura
Target Milestone:	---
Target Release:	3.6.z
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2017-09-08 18:38:25 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Takayoshi Tanaka 2017-07-26 07:17:51 UTC

Description of problem:
When a user mounts secrets as files, mounted file can't be accessed by "permission denied" error.

Version-Release number of selected component (if applicable):

openshift v3.5.5.31
kubernetes v1.5.2+43a9be4


How reproducible:
In the customer side, always.
In my Lab, never. (the version is same)


Steps to Reproduce:
The customer created a public reproduction repository.
https://github.com/brainsnorkel/test-os-subpaths

$ git clone https://github.com/brainsnorkel/test-os-subpaths
$ cd test-os-subpaths
$ oc new-project
$ ./oc-create-all.sh
$ ./testest-mounts+envs.sh  

Actual results:
the quick
brown fox
jumps over
the lazy dog
cat: can't open '/opt/test-secret.txt': Permission denied


Expected results:
the quick
brown fox
jumps over
the lazy dog
this
is
a
secret
file
SEKRIT_ENV=‘verySecretPassword


Additional info:
File permission info:
$ oc rsh $(oc get pods | grep Running | awk '// {print $1}') ls -als /opt
ls: /opt/test-secret.txt: Permission denied
total 4
     0 drwxr-xr-x    2 root     root            50 Jul 26 01:14 .
     0 drwxr-xr-x   19 root     root           217 Jul 26 01:14 ..
     4 -rw-r--r--    1 root     10001600        44 Jul 26 01:14 test-text.txt

The secret environment variable from the same secret is available:

$ oc rsh $(oc get pods | grep Running | awk '// {print $1}') sh -c set | grep -i SEKRIT_ENV
SEKRIT_ENV='verySecretPassword'

However, I can get the expected result at the same version of OCP 3.5.5.31. Also, the customer can get the expected result at OSO v3.6.0-rc.0+98b3d56 and v1.5.0-rc.0

Comment 2 Ryan Howe 2017-08-04 23:41:08 UTC

        volumeMounts:
        - mountPath: /opt/test-secret.txt
          name: volume-sekrit
          subPath: somexml
    - name: volume-sekrit
      secret:
        defaultMode: 420
        secretName: sekrits



The root of this  issue is that the right selinux context is not set for the target.


drwxrwsrwt. root 1000080000 system_u:object_r:svirt_sandbox_file_t:s0:c4,c9 default-token-r5bmh
drwxrwsrwt. root 1000080000 system_u:object_r:tmpfs_t:s0     volume-sekrit

lrwxrwxrwx. root root system_u:object_r:svirt_sandbox_file_t:s0:c4,c9 /var/lib/origin/openshift.local.volumes/pods/5972781e-7967-11e7-96bd-fa163eb9684a/volumes/kubernetes.io~secret/volume-sekrit/somexml -> ..data/somexml
-rw-r--r--. root 1000080000 system_u:object_r:tmpfs_t:s0     /var/lib/origin/openshift.local.volumes/pods/5972781e-7967-11e7-96bd-fa163eb9684a/volumes/kubernetes.io~secret/volume-sekrit/..data/somexml


If I mount the secret 2x one with out the subpath it works as a workaround. 

Thinking its an error some where here in this code: 

https://github.com/kubernetes/kubernetes/blob/release-1.5/pkg/kubelet/kubelet_pods.go#L117-L150

Comment 3 Takayoshi Kimura 2017-08-08 08:05:20 UTC

It looks like a bug. With the subpath specified, only the subpath symlink gets relabeled. The symlink target, actual content file won't be relabeled and access denied.

Comment 4 Seth Jennings 2017-08-25 15:46:36 UTC

reducing to medium since customer issue is closed and there are workarounds.

Andrew, please take a look.  Probably just want to recreate first, then run it down.  Issue is with selinux labeling for secrets with using subPath.

Comment 7 Jan Pazdziora (Red Hat) 2017-09-08 09:02:51 UTC

There is upstream issue https://github.com/openshift/origin/issues/15750 referencing this bugzilla which talks about upgrades. I wonder if the fact that the issue can be seen on one instance and not on another one of the same version (do I read it right?) is due to the fact that one instance was upgraded from older version and the other one was not.

Comment 8 Seth Jennings 2017-09-08 18:38:25 UTC

Looks like a dup of 1481617

*** This bug has been marked as a duplicate of bug 1481617 ***