Hide Forgot
Description of problem: Create a storage class with no parameters, dynamic provision failed Version-Release number of selected component (if applicable): openshift v3.7.0-0.146.0 kubernetes v1.7.6+a08f5eeb62 How reproducible: Always Steps to Reproduce: 1. Create a storage class with no parameters 2. Create a pvc which point this storage class 3. PVC would keep pending Actual results: $ oc get pvc NAME STATUS VOLUME CAPACITY ACCESSMODES STORAGECLASS AGE azpvc Pending sc-kytwu 4m $ oc describe pvc azpvc Name: azpvc Namespace: kytwu StorageClass: sc-kytwu Status: Pending Volume: Labels: <none> Annotations: volume.beta.kubernetes.io/storage-class=sc-kytwu volume.beta.kubernetes.io/storage-provisioner=kubernetes.io/azure-disk Capacity: Access Modes: Events: FirstSeen LastSeen Count From SubObjectPath Type Reason Message --------- -------- ----- ---- ------------- -------- ------ ------- 5m 5s 22 persistentvolume-controller Warning ProvisioningFailed Failed to provision volume with StorageClass "sc-kytwu": Create Storage Account: p443172070, error: storage.AccountsClient#Create: Failure responding to request: StatusCode=400 -- Original Error: autorest/azure: Service returned an error. Status=400 Code="LocationRequired" Message="The location property is required for this definition." Expected results: PVC can be provisioned and bound Master Log: Oct 10 09:10:42 wehe-master-etcd-nfs-1 atomic-openshift-master-controllers: I1010 09:10:42.354472 21378 pv_controller.go:1337] failed to provision volume for claim "default/registry-pvc" with StorageClass "grs": azureDisk - Standard_GRS is not supported sku/storageaccounttype. Supported values are [Premium_LRS Standard_LRS] Oct 10 09:10:44 wehe-master-etcd-nfs-1 atomic-openshift-node: I1010 09:10:44.225558 14737 kubelet_node_status.go:490] Using Node Hostname from cloudprovider: "wehe-master-etcd-nfs-1.eastus.cloudapp.azure.com" Oct 10 09:10:44 wehe-master-etcd-nfs-1 atomic-openshift-master-api: I1010 09:10:44.240381 21352 rest.go:349] Starting watch for /api/v1/services, rv=2299 labels= fields= timeout=7m33s Oct 10 09:10:49 wehe-master-etcd-nfs-1 atomic-openshift-master-controllers: I1010 09:10:49.692346 21378 azure_blobDiskController.go:602] azureDisk - Creating storage account p443172070 type Standard_LRS Oct 10 09:10:49 wehe-master-etcd-nfs-1 atomic-openshift-master-controllers: I1010 09:10:49.733551 21378 pv_controller.go:1337] failed to provision volume for claim "kytwu/azpvc" with StorageClass "sc-kytwu": Create Storage Account: p443172070, error: storage.AccountsClient#Create: Failure responding to request: StatusCode=400 -- Original Error: autorest/azure: Service returned an error. Status=400 Code="LocationRequired" Message="The location property is required for this definition." Oct 10 09:10:50 wehe-master-etcd-nfs-1 atomic-openshift-master-api: I1010 09:10:50.478416 21352 rest.go:349] Starting watch for /apis/network.openshift.io/v1/egressnetworkpolicies, rv=2254 labels= fields= timeout=6m6s Oct 10 09:10:54 wehe-master-etcd-nfs-1 atomic-openshift-node: I1010 09:10:54.409112 14737 kubelet_node_status.go:490] Using Node Hostname from cloudprovider: "wehe-master-etcd-nfs-1.eastus.cloudapp.azure.com" Oct 10 09:10:56 wehe-master-etcd-nfs-1 atomic-openshift-master-api: I1010 09:10:56.268606 21352 rest.go:349] Starting watch for /apis/extensions/v1beta1/networkpolicies, rv=19624 labels= fields= timeout=6m16s Oct 10 09:10:56 wehe-master-etcd-nfs-1 atomic-openshift-master-api: I1010 09:10:56.652807 21352 rest.go:349] Starting watch for /apis/extensions/v1beta1/daemonsets, rv=19695 labels= fields= timeout=6m7s Oct 10 09:10:56 wehe-master-etcd-nfs-1 atomic-openshift-node: I1010 09:10:56.905512 14737 qos_container_manager_linux.go:286] [ContainerManager]: Updated QoS cgroup configuration Node Log (of failed PODs): PV Dump: PVC Dump: $ oc export pvc azpvc apiVersion: v1 kind: PersistentVolumeClaim metadata: annotations: volume.beta.kubernetes.io/storage-class: sc-kytwu volume.beta.kubernetes.io/storage-provisioner: kubernetes.io/azure-disk creationTimestamp: null name: azpvc spec: accessModes: - ReadWriteOnce resources: requests: storage: 1Gi status: {} StorageClass Dump (if StorageClass used by PV/PVC): # oc export storageclass sc-kytwu apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: creationTimestamp: null name: sc-kytwu provisioner: kubernetes.io/azure-disk Additional info: From the master log, we can see another issue, sku name only support Premium_LRS Standard_LRS now, not sure whether this is new design for OCP
Turns out the azure disk refactoring in 1.7 introduced a regression. It requires "location" to be set in azure cloud config.
Upstream issue https://github.com/kubernetes/kubernetes/issues/52396
for workaround, please set storage account name in the storage class.
There is another issue with running azure provisioner in openshift, https://github.com/kubernetes/kubernetes/issues/54570
Due to kubernetes issues #54570 and #52396, we cannot use "Kind" parameter (will hit #54570) or empty parameter (will hit #52396)in openshift.
Thanks, both location and storageAccount are needed for 3.7.