Bug 2080449 - [Azure-file CSI Driver] Read/Write permission denied for non-admin user on azure file csi provisioned volume with fsType=ext4,ext3,ext2,xfs
Summary: [Azure-file CSI Driver] Read/Write permission denied for non-admin user on az...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Storage
Version: 4.11
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: 4.12.0
Assignee: Roman Bednář
QA Contact: Rahul Deore
URL:
Whiteboard:
Depends On:
Blocks: 2105972
TreeView+ depends on / blocked
 
Reported: 2022-04-29 16:25 UTC by Rahul Deore
Modified: 2023-01-17 19:48 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: Release Note
Doc Text:
* Users can now opt into using the experimental VHD feature on Azure File. To opt in, users must specify the `fstype` parameter in a storage class and enable it with `--enable-vhd=true`. If `fstype` is used and the feature is not set to `true`, the volumes will fail to provision. + To opt out of using the VHD feature, remove the `fstype` parameter from your storage class. (link:https://bugzilla.redhat.com/show_bug.cgi?id=2080449[*BZ#2080449*])
Clone Of:
Environment:
Last Closed: 2023-01-17 19:48:18 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift azure-file-csi-driver-operator pull 33 0 None Merged Bug 2080449: disable VHD disk feature 2022-07-07 07:01:18 UTC
Github openshift azure-file-csi-driver pull 15 0 None Merged Bug 2080449: UPSTREAM: 1023: add new option to allow VHD feature opt-out 2022-06-28 08:34:21 UTC
Red Hat Product Errata RHSA-2022:7399 0 None None None 2023-01-17 19:48:38 UTC

Description Rahul Deore 2022-04-29 16:25:48 UTC
Description of problem:

Creating PV/PVC using a storage class with "file.csi.azure.com", with default fsType='cifs', and when mounted on pod, non-admin user can Read/Write into the mounted volume on pod.

But when PV/PVC created using the storage class with "file.csi.azure.com", with  fsType= ext4.ext3,ext2,xfs, mounted on a pod, only admin user have Read/Write permissions. Non-admin user gets permission denied error when creating/reading files on the mounted volume.

Version-Release number of selected component (if applicable):
4.11.0-0.nightly-2022-04-26-181148

How reproducible:

Steps to Reproduce:
1. Set up Azure cluster
2. Install Azure-file CSI Driver with enable the feature gate
3. Create csi storageclass with parameters.fstype: 'ext4'

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: mystorageclass
provisioner: file.csi.azure.com
parameters:
  fstype: ext4
reclaimPolicy: Delete
allowVolumeExpansion: true
volumeBindingMode: WaitForFirstConsumer

4. Create a pvc with the csi storageclass created in step-3
    
5. Create pod with the pvc created in step-4 and wait for the pod to be ready

6. Check the pod gets created successfully and mounted volume (/mnt/storage) fstype is 'ext4'

7. Check the pod volume mounted on /mnt/storage can be read and write by 'non-admin' user

Actual results:
 Can not read/write into mounted volume due to "permission denied Error"
 ERROR: cannot create regular file '/mnt/storage/hello': Permission denied

Expected results:
 Non-admin user can read/write into mounted volume successfully

Master Log:

Node Log (of failed PODs):

PV Dump:

PVC Dump:

StorageClass Dump (if StorageClass used by PV/PVC):

Additional info:

Comment 1 Hemant Kumar 2022-05-06 17:03:41 UTC
Filed an upstream bug - https://github.com/kubernetes-sigs/azurefile-csi-driver/issues/1012 . This happens only for newly introduced VHD disk feature of azure file. VHD disks don't appear to be respecting fsgroup specified in the pods and hence non-admin pods can't write to the volume.

Comment 2 Hemant Kumar 2022-05-10 14:34:45 UTC
We are not going to support VHD disks over azurefile CSI volumes. Support for VHD disks will either be disabled or prevented via webhook validation.

Comment 3 Roman Bednář 2022-05-18 13:14:41 UTC
So it seems there already is a patch upstream that fixes the issue:
https://github.com/kubernetes-sigs/azurefile-csi-driver/pull/1013

Please test again with the latest upstream driver and close/verify this BZ if the bug can't be reproduced anymore.

Comment 4 Rahul Deore 2022-05-24 16:56:21 UTC
Tested on 4.11.0-0.nightly-2022-05-20-213928, but still failing with error as,
ERROR: cannot create regular file '/mnt/storage/hello': Permission denied

Comment 5 Wei Duan 2022-05-31 03:52:10 UTC
Hi @roman, just declare that QE will use the CSI Driver shipped with Openshift but not upstream one, current Azure-file CSI Driver version is 1.14.0 and looks like no plan to update to 1.18.0 in 4.11. 
So assign it back.

Comment 6 Roman Bednář 2022-06-06 10:53:01 UTC
Ack, it also turned out the upstream change did not address the bug as I thought, so I created a new one and trying to get it merged upstream. Then we can agree on a rebase or I'll do a cherry-pick.

https://github.com/kubernetes-sigs/azurefile-csi-driver/pull/1023

Comment 10 Rahul Deore 2022-06-21 15:51:05 UTC
Tested on 4.11.0-0.nightly-2022-06-21-040754, but scenarion is still failing for non-admin user, below are the observations,


1. When create pod as common-user (non-admin), neither admin nor common-user can write into the mounted pod volume
   (Permission denied for both admin and non-admin)

sh-5.1$ id
uid=1000670000(1000670000) gid=0(root) groups=0(root),1000670000
sh-5.1$  echo 'hello' > /mnt/azure/test.txt
sh: /mnt/azure/test.txt: Permission denied
sh-5.1$ 
sh-5.1$ 


2. When create pod as Admin user, both Admin and common-user (non-admin) can write into the pod volume successfully
 
sh-5.1# id
uid=0(root) gid=0(root) groups=0(root)
sh-5.1# 
sh-5.1# echo 'hello' > /mnt/azure/test.txt
sh-5.1# cat /mnt/azure/test.txt
hello
sh-5.1#

Comment 22 errata-xmlrpc 2023-01-17 19:48:18 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.12.0 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:7399


Note You need to log in before you can comment on or make changes to this bug.