Bug 1272146

Summary: Mounted secrets unreadible with SELinux enabled
Product: [Fedora] Fedora Reporter: Thijs Elferink <thijs.elferink>
Component: kubernetesAssignee: Jan Chaloupka <jchaloup>
Status: CLOSED NOTABUG QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: unspecified    
Version: 23CC: bchilds, eparis, golang-updates, jcajka, jchaloup, lsm5, nhorman, vbatts, walters
Target Milestone: ---Keywords: SELinux
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1276080 (view as bug list) Environment:
Last Closed: 2016-06-23 14:21:49 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Thijs Elferink 2015-10-15 14:53:59 UTC
Description of problem:

On a freshly installed Fedora Atomic host (as well as on a CentOS Atomic host);
When mounting a secret in a pod, the mount shows up with garbled permissions and is inaccessible.

Version-Release number of selected component (if applicable):

ostree images (both have this problem):
  TIMESTAMP (UTC)         VERSION        ID             OSNAME                 REFSPEC
  2015-10-14 11:25:03     23.33          89be310d70     centos-atomic-host     fedora-atomic:fedora-atomic/f23/x86_64/docker-host
  2015-10-01 09:32:09     7.20151001     1e9838ce88     centos-atomic-host     centos-atomic-host:centos-atomic-host/7/x86_64/standard

kubernetes node description:
 Kernel Version:		4.2.3-300.fc23.x86_64
 OS Image:			Fedora 23 (Twenty Three)
 Container Runtime Version:	docker://1.7.0-dev.fc23
 Kubelet Version:		v1.1.0-alpha.0.1588+e44c8e6661c931
 Kube-Proxy Version:		v1.1.0-alpha.0.1588+e44c8e6661c931

How reproducible:

always

Steps to Reproduce:
1. fresh fedora/centos atomic host
2. deploy secret (kubectl create -f secret.json)

secret.json:

{
  "apiVersion": "v1",
  "kind": "Secret",
  "metadata" : {
    "name": "test-secret"
  },
  "type": "Opaque",
  "data": {
    "test-data":"dGVzdDEyMw=="
  }
}

3. deploy pod (kubectl create -f test-pod.yaml)

test-pod.yaml:

apiVersion: v1
kind: Pod
metadata:
  name: test-pod
spec:
  containers:
  - name: test
    image: busybox
    volumeMounts:
    - name: "test-volume"
      mountPath: "/test"
      readOnly: true
    command: ["sh"]
    args: ["-c 'ls -l /test/test-data; cat /test/test-data'"]

  volumes:
    - name: "test-volume"
      secret:
        secretName: "test-secret"


Actual results:

output:

-bash-4.2# kubectl logs test-pod
ls: cannot access /test/test-data: Permission denied
total 0
-????????? ? ? ? ?            ? test-data
cat: /test/test-data: Permission denied


Expected results:

output:

-bash-4.2# kubectl logs test-pod
total 4
-r--r--r--. 1 root root 7 Oct 15 08:08 test-data
test123


Additional info:

After disabling SELinux (setenforce 0) the secret is accessible.

Comment 1 Jan Chaloupka 2015-10-21 09:17:33 UTC
Hi Colin,

what selinux rpms are used in Atomic Host? Have you encountered with this issue before?

Thanks
Jan

Comment 6 Jan Chaloupka 2016-06-23 14:21:49 UTC
As mentioned in [1], the solution is to use pod-level SecurityContext:

apiVersion: v1
kind: Pod
metadata:
  name: test-pod
spec:
  containers:
  - name: test
    image: busybox
    volumeMounts:
    - name: "test-volume"
      mountPath: "/test"
      readOnly: true
    command:
    - "sh"
    - "-c"
    - |
      ls -l /test/test-data
      cat /test/test-data
  securityContext:
    seLinuxOptions:
       level: "s0:c123,c456"
  volumes:
    - name: "test-volume"
      secret:
        secretName: "test-secret"

SecurityContextDeny must be removed from /etc/kubernetes/apiserver to enable the SecurityContext. More about it [2], [3].

[1] https://github.com/projectatomic/adb-atomic-developer-bundle/issues/117#issuecomment-215313573
[2] http://kubernetes.io/docs/admin/admission-controllers/#securitycontextdeny
[3] http://kubernetes.io/docs/user-guide/security-context/