Bug 1567610 - Storage Class parameters ignored after upgrade to 3.7.42
Summary: Storage Class parameters ignored after upgrade to 3.7.42
Keywords:
Status: CLOSED WORKSFORME
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Storage
Version: 3.7.0
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
: ---
Assignee: hchen
QA Contact: Wenqi He
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-04-15 12:34 UTC by hgomes
Modified: 2018-05-08 22:12 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-04-23 14:08:56 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description hgomes 2018-04-15 12:34:43 UTC
Description of problem:

Upgrade cluster to 3.7.42 and dynamically provisioned persistent volumes stopped working with this error message:

atomic-openshift-master-controllers:  pv_controller.go:1380] failed to provision volume for claim "projectname/pvcname" with StorageClass "generic": Create Storage Account: pvc3096215148001, error: storage.AccountsClient#Create: Failure responding to request: StatusCode=400 -- Original Error: autorest/azure: Service returned an error. Status=400 Code="LocationRequired" Message="The location property is required for this definition."

Storage class definition BEFORE the upgrade:

kind: StorageClass
apiVersion: storage.k8s.io/v1beta1
metadata:
  name: "generic"
  annotations:
    storageclass.beta.kubernetes.io/is-default-class: "true"
    volume.beta.kubernetes.io/storage-class: "generic"
    volume.beta.kubernetes.io/storage-provisioner: kubernetes.io/azure-disk
provisioner: kubernetes.io/azure-disk
parameters:
  storageAccount: <account-name>

File /etc/azure/azure.conf BEFORE the upgrade, it had definitions:
resourceGroup, subscriptionID, aadClientID,aadClientSecret,tenantID,useIntanceMedatadata

The storage account exists in Azure and contains dynamically created persistent volumes.

How reproducible:
Before the configuration changes, it was able to reproduce this behavior every time I created a persistent volume claim with storage class "generic".


Additional info:

Comment 1 hgomes 2018-04-15 12:41:31 UTC
As a workaround:

Added two parameters (location and skuName) in the storage class:

kind: StorageClass
apiVersion: storage.k8s.io/v1beta1
metadata:
  name: "generic"
  annotations:
    storageclass.beta.kubernetes.io/is-default-class: "true"
    volume.beta.kubernetes.io/storage-class: "generic"
    volume.beta.kubernetes.io/storage-provisioner: kubernetes.io/azure-disk
provisioner: kubernetes.io/azure-disk
parameters:
  location: westus
  storageAccount: <storage-name> <<<<<
  skuName: Premium_LRS           <<<<<

Added the 'location' in /etc/azure/azure.conf:

{"resourceGroup": "lab-purple-edition", "subscriptionID": "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX", "aadClientID": "AAAAAAAA-AAAA-AAAA-AAAA-AAAAAAAAAAAA", "aadClientSecret": "SECRET", "tenantID": "BBBBBBBB-BBBB-BBBB-BBBB-BBBBBBBBBBBB", "useInstanceMetadata": true, "location": "purple" }

Comment 2 Wenqi He 2018-04-16 03:28:13 UTC
Please refer my upstream issue: https://github.com/kubernetes/kubernetes/issues/61265 

The error message shows your azure conf need the "location" parameter. 

After kube 1.7 azure storage class has been revised, you need to specify the "kind: Dedicated" if you still want to use the azure storage class you created before upgrade, otherwise, add location in you azure conf to use the default "kind: Shared" and it will create few new storage accounts and keep using the new created ones.

Comment 3 Wenqi He 2018-04-16 03:39:13 UTC
Example of the sc:
$ cat desc.yaml
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: addsc 
provisioner: kubernetes.io/azure-disk
parameters:
  kind: Dedicated
  storageaccount: eastusimg

Comment 4 hchen 2018-04-23 14:08:56 UTC
Please refer to Comment 3 for resolution.


Note You need to log in before you can comment on or make changes to this bug.