Bug 1577824

Summary: Reading data from persistent storage provided by csi will report "Permission denied" error
Product: OpenShift Container Platform Reporter: Qin Ping <piqin>
Component: StorageAssignee: Jan Safranek <jsafrane>
Status: CLOSED ERRATA QA Contact: Qin Ping <piqin>
Severity: high Docs Contact:
Priority: high    
Version: 3.10.0CC: aos-bugs, aos-storage-staff, bchilds
Target Milestone: ---   
Target Release: 3.10.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: No Doc Update
Doc Text:
undefined
 Story Points: ---
Clone Of: Environment:
Last Closed: 2018-07-30 19:15:23 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Qin Ping 2018-05-14 08:24:34 UTC 
Description of problem:
Reading data from persistent storage provided by csi will report "Permission denied" error

Version-Release number of selected component (if applicable):
oc v3.10.0-0.38.0
openshift v3.10.0-0.41.0
kubernetes v1.10.0+b81c8f8
csi-attacher-0.2.0-2.git27299be.el7.x86_64
csi-provisioner-0.2.0-1.el7.x86_64
csi-driver-registrar-0.2.0-1.el7.x86_64

How reproducible:
Deploy csi per https://github.com/openshift/openshift-docs/pull/8783/files

Steps to Reproduce:
1. Create PVC
# cat pvc.yaml 
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: pvc1 
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi

2. Create Pod
# cat pod.yaml 
apiVersion: v1
kind: Pod
metadata:
  name: web-server
spec:
  containers:
   - name: web-server
     image: nginx 
     volumeMounts:
       - mountPath: /var/lib/www/html
         name: mypvc
  volumes:
   - name: mypvc
     persistentVolumeClaim:
       claimName: pvc1
       readOnly: false

3. Read data from persistent storage provided by csi

Actual results:
# oc exec web-server -- ls /var/lib/www/html
ls: cannot open directory '/var/lib/www/html': Permission denied

Expected results:
Could read and write data successfully.

Master Log:

Node Log (of failed PODs):

PV Dump:

PVC Dump:

StorageClass Dump (if StorageClass used by PV/PVC):

Additional info:
Comment 1 Jan Safranek 2018-05-15 12:58:31 UTC 
CSI as currently does not support SELinux.

I filled https://github.com/container-storage-interface/spec/issues/235 to update the specification, however it may take a while before it's fixed in the spec and all CSI drivers.

I will update the docs.
Comment 4 Jan Safranek 2018-05-18 12:33:31 UTC 
Upstream PR: https://github.com/kubernetes/kubernetes/pull/64026
Comment 5 Jan Safranek 2018-05-23 12:52:19 UTC 
Origin PR: https://github.com/openshift/origin/pull/19816
Comment 7 Qin Ping 2018-06-06 02:56:25 UTC 
Verified in openshift:
oc v3.10.0-0.60.0
openshift v3.10.0-0.60.0
kubernetes v1.10.0+b81c8f8

csi-attacher-0.2.0-3.git27299be.el7.x86_64
csi-provisioner-0.2.0-2.el7.x86_64
csi-driver-registrar-0.2.0-1.el7.x86_64

# uname -a
Linux qe-piqin-master-etcd-nfs-1 3.10.0-862.3.2.el7.x86_64 #1 SMP Tue May 15 18:22:15 EDT 2018 x86_64 x86_64 x86_64 GNU/Linux

# cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 7.5 (Maipo)
Comment 9 errata-xmlrpc 2018-07-30 19:15:23 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:1816