Bug 1575933

Since the namespace is not known in advance, it is not possible to create a default rule.

This needs to be documented as below:

When using storage class to provision azure file shares that is to be used in a namespace, a corresponding rbac rule should be created to allow azure file provisioner to allow create secret in that namespace.

assign to Traci.

*** Bug 1578583 has been marked as a duplicate of this bug. ***

(In reply to hchen from comment #2)
> This needs to be documented as below:
> 
> When using storage class to provision azure file shares that is to be used
> in a namespace, a corresponding rbac rule should be created to allow azure
> file provisioner to allow create secret in that namespace.
> 
> assign to Traci.

Where should this be documented? We have information on Azure File volumes here:

https://docs.openshift.com/container-platform/3.9/install_config/persistent_storage/persistent_storage_azure_file.html

Adding vigoyal as the Docs Contact.

```
During provision, a secret is created for mounting credentials. If the cluster has enabled both RBAC and Controller Roles, add the create permission of resource secret for clusterrole system:controller:persistent-volume-binder.
```

I am a newbie to authorization:

It seems that system:controller:persistent-volume-binder is a clusterrole, ie, not associated with any namespace.
Would it be proper to just add `create` permission to `secrets` resource by default?

Thanks.

> Where should this be documented? We have information on Azure File volumes
> here:
> 
> https://docs.openshift.com/container-platform/3.9/install_config/
> persistent_storage/persistent_storage_azure_file.html

Helle Traci,

I believe that hchen is asking for the way to do dynamic provisioning based on sc of azure file.

The document above follows the load of creating a PV and using it in a PVC.

(In reply to hchen from comment #1)
> Since the namespace is not known in advance, it is not possible to create a
> default rule.

@Huamin, The role system:controller:persistent-volume-binder is cluster role which used by azure file sc to create secrets in end user's project. And I agree with Hongkai's comment #6, it does not need to know the name space in advance, the user just need to create a pvc and use the sc, and then the system:controller:persistent-volume-binder will create the secret in the user's project.

# oc get sc sc-hkl5e -o yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  creationTimestamp: 2018-05-21T08:10:09Z
  name: sc-hkl5e
  resourceVersion: "94194"
  selfLink: /apis/storage.k8s.io/v1/storageclasses/sc-hkl5e
  uid: 61295b07-5cce-11e8-9424-000d3a1ae476
provisioner: kubernetes.io/azure-file
reclaimPolicy: Delete
volumeBindingMode: Immediate

# oc get pvc -o yaml
apiVersion: v1
items:
- apiVersion: v1
  kind: PersistentVolumeClaim
  metadata:
    annotations:
      pv.kubernetes.io/bind-completed: "yes"
      pv.kubernetes.io/bound-by-controller: "yes"
      volume.beta.kubernetes.io/storage-class: sc-hkl5e
      volume.beta.kubernetes.io/storage-provisioner: kubernetes.io/azure-file
    creationTimestamp: 2018-05-21T08:10:12Z
    finalizers:
    - kubernetes.io/pvc-protection
    name: azpvc
    namespace: hkl5e
    resourceVersion: "94215"
    selfLink: /api/v1/namespaces/hkl5e/persistentvolumeclaims/azpvc
    uid: 6296f848-5cce-11e8-9424-000d3a1ae476
  spec:
    accessModes:
    - ReadWriteMany
    resources:
      requests:
        storage: 1Gi
    volumeMode: Filesystem
    volumeName: pvc-6296f848-5cce-11e8-9424-000d3a1ae476
  status:
    accessModes:
    - ReadWriteMany
    capacity:
      storage: 1Gi
    phase: Bound
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""

# oc get pv pvc-6296f848-5cce-11e8-9424-000d3a1ae476 -o yaml
apiVersion: v1
kind: PersistentVolume
metadata:
  annotations:
    kubernetes.io/createdby: azure-file-dynamic-provisioner
    pv.kubernetes.io/bound-by-controller: "yes"
    pv.kubernetes.io/provisioned-by: kubernetes.io/azure-file
  creationTimestamp: 2018-05-21T08:10:14Z
  finalizers:
  - kubernetes.io/pv-protection
  name: pvc-6296f848-5cce-11e8-9424-000d3a1ae476
  resourceVersion: "94212"
  selfLink: /api/v1/persistentvolumes/pvc-6296f848-5cce-11e8-9424-000d3a1ae476
  uid: 64135c97-5cce-11e8-9424-000d3a1ae476
spec:
  accessModes:
  - ReadWriteMany
  azureFile:
    secretName: azure-storage-account-dsb24acb944ea411e883af0-secret
    secretNamespace: hkl5e
    shareName: kubernetes-dynamic-pvc-6296f848-5cce-11e8-9424-000d3a1ae476
  capacity:
    storage: 1Gi
  claimRef:
    apiVersion: v1
    kind: PersistentVolumeClaim
    name: azpvc
    namespace: hkl5e
    resourceVersion: "94204"
    uid: 6296f848-5cce-11e8-9424-000d3a1ae476
  persistentVolumeReclaimPolicy: Delete
  storageClassName: sc-hkl5e
  volumeMode: Filesystem
status:
  phase: Bound

# oc get secrets -n hkl5e
NAME                                                   TYPE                                  DATA      AGE
azure-storage-account-dsb24acb944ea411e883af0-secret   Opaque                                2         10m

So I think we need to ask authorization to add the "create" and "delete"( which used by deleting secret after user delete pvc) to the Cluster role system:controller:persistent-volume-binder by default. Just like other dynamic provision, we do not need to do any authorization work before using azure file dynamic provision.

The Storage team will follow up on this bug.

As comment #2, below are the steps to make azure file dynamic provision work.

1. Admin need to create the role in user's project as below:
$ cat azf-role.yaml 
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: system:controller:persistent-volume-binder
  namespace: <user's project name> 
rules:
  - apiGroups: [""]
    resources: ["secrets"]
    verbs: ["create", "get", "delete"]

2. Admin need to create the role binding to the service account "persistent-volume-binder" in "kube-system" project

$ cat azf-rolebind.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: system:controller:persistent-volume-binder
  namespace: <user's project> 
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: system:controller:persistent-volume-binder
subjects:
- kind: ServiceAccount
  name: persistent-volume-binder
namespace: kube-system

3. Admin add service account as admin to user's project
oc policy add-role-to-user admin system:serviceaccount:kube-system:persistent-volume-binder -n <user's project>

4 Admin create storage class of azure file

$ cat azfsc.yaml | oc create -f -
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: azfsc
provisioner: kubernetes.io/azure-file
mountOptions:
  - dir_mode=0777
  - file_mode=0777

5. Then end user create a pvc which use this sc, azure file dynamic provision can work

Thank you Wenqi for the instructions.


Traci, can these steps be documented? Thanks

(In reply to hchen from comment #11)
> Thank you Wenqi for the instructions.
> 
> 
> Traci, can these steps be documented? Thanks

Yes, I just need to know where this should be documented. Is this new content or can this be added to existing content? Thanks.

I think we can add this to below doc? https://docs.openshift.com/container-platform/3.9/install_config/persistent_storage/dynamically_provisioning_pvs.html?

Because this is related to dynamic provision and storage class, to add azure file in it is more appropriate. Thanks.

Work in progress: https://github.com/openshift/openshift-docs/pull/9991

(In reply to Traci Morrison from comment #14)
> Work in progress: https://github.com/openshift/openshift-docs/pull/9991

Made changes as described: Added a section for the Azure file. This can be reviewed here: https://github.com/openshift/openshift-docs/pull/9991. Moving this bug to MODIFIED.

Merged: https://github.com/openshift/openshift-docs/pull/9991

CP to 3.10: https://github.com/openshift/openshift-docs/pull/10167

Published for 3.10: https://docs.openshift.com/container-platform/3.10/install_config/persistent_storage/dynamically_provisioning_pvs.html#azure-file-secret-permission