Description of problem:
No write permission in a directory mounted as a PVC with Azure File
Version-Release number of selected component (if applicable):
(It doesn't happen in v3.7.23)
Steps to Reproduce:
1. Install or Upgrade OpenShift 3.7.42
2. Run a pod mounting a PVC with Azure File
A pod failed to write a file.
A pod can write a file.
In the Pod (PVC is mounted at /data2):
sh-4.2$ ls -al /data2/
sh-4.2$ touch /data2/ocp37
touch: cannot touch '/data2/ocp37': Permission denied
sh-4.2$ mkdir /data2/ocp37
mkdir: cannot create directory '/data2/ocp37': Permission denied
In the node:
//xxx.file.core.windows.net/ocp-filetest-fileshare01 on /var/lib/origin/openshift.local.volumes/pods/uuid/volumes/kubernetes.io~azure-file/azure-file-pv01 type cifs (rw,relatime,vers=3.0,sec=ntlmssp,cache=strict,username=user,domain=X,uid=0,noforceuid,gid=0,noforcegid,addr=x.x.x.x,file_mode=0755,dir_mode=0755,persistenthandles,nounix,serverino,mapposix,rsize=1048576,wsize=1048576,echo_interval=60,actimeo=1)
The same thing happens also in OpenShift 3.9 but we can work around using mountOptions in PV or StorageClass at OpenShift 3.9.
The dir_mode and file_mode of mount options are hard-coded at OpenShift 3.7.
I have reported similar bug before, pls check my comment in https://bugzilla.redhat.com/show_bug.cgi?id=1543229#c4
Thank you for a comment. However, I have two concerns.
- The issue is fixed only at OpenShift 3.9. Do you have an idea to work around at OpenShift 3.7.42?
I'm afraid we should fix the issue for v3.7.42.
- The issue is fixed by introducing fsGroup. Since it seems the user operation is required when upgrading 3.7 to 3.9, do we need a document?
Also, I'm writing a KCS for permission issue of Azure File.
(In reply to Takayoshi Tanaka from comment #3)
> Thank you for a comment. However, I have two concerns.
> - The issue is fixed only at OpenShift 3.9. Do you have an idea to work
> around at OpenShift 3.7.42?
> I'm afraid we should fix the issue for v3.7.42.
Could you please try my solution in your 3.7 env to check whether it works or not? If not, I think we need to backport some PRs to resolve this issue. Thanks.
"mountOptions" is not available at 3.7 because it's introduced at kubernetes 1.9 (and OpenShift 3.9).
The regression was caused by  and the file/dir mode is (partially) reverted to 0755 , both fixes are back in 3.7.42. However, there is no upstream consensus to go back to 0777.
@Anshul, 3.6 doesn't have this issue. The file and dir modes are 0777
The file/dir mode regression happens if the pod uid/gid are not the same as those in azure file. Customer can either upgrade to 3.9 to use mount option, or stay at 3.7.23 (before the regression issue)
3.7 fix is at https://github.com/openshift/ose/pull/1244
I have manually successful installed OCP with latest 3.7 with below version:
This bug is fixed in 3.7
$ oc get pods
NAME READY STATUS RESTARTS AGE
azfpod 1/1 Running 0 9m
$ oc exec -it azfpod sh
/ $ ls /mnt/azure/
/ $ touch /mnt/azure/wehe
/ $ ls /mnt/azure/
/ $ exit
BTW, because of the image tag issue, make so much effort on installation with update local packages, update image tag, and finally deploy OCP 3.7 latest successfully.
We have updated to Openshift 3.7.46 and are now able to write to azure file. Can someone confirm this was backported to 3.7.46 as the change logs does not have this Bugzilla listed.
I can confirm that https://github.com/openshift/ose/pull/1244 is merged to all versions after v3.7.45-1.