Bug 1390970

Summary: ProvisioningFailed: Failed to provision volume with StorageClass "slow": failed to get secret from ["storage-project"/"heketi-secret"]
Product: Red Hat Gluster Storage Reporter: Prasanth <pprakash>
Component: CNS-deploymentAssignee: Humble Chirammal <hchiramm>
Status: CLOSED CURRENTRELEASE QA Contact: Prasanth <pprakash>
Severity: high Docs Contact:
Priority: unspecified    
Version: cns-3.4CC: akhakhar, annair, hchiramm, jrivera, madam, mliyazud, mzywusko, nerawat, pprakash, rcyriac, rmekala, rreddy, rtalur
Target Milestone: ---   
Target Release: CNS 3.4   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-01-23 07:11:03 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On:    
Bug Blocks: 1385247    

Description Prasanth 2016-11-02 10:30:13 UTC
Description of problem:

ProvisioningFailed: Failed to provision volume with StorageClass "slow": failed to get secret from ["storage-project"/"heketi-secret"]

Version-Release number of selected component (if applicable):
oc v3.4.0.18+ada983f
kubernetes v1.4.0+776c994
features: Basic-Auth GSSAPI Kerberos SPNEGO

Server https://dhcp46-41.lab.eng.blr.redhat.com:8443
openshift v3.4.0.18+ada983f
kubernetes v1.4.0+776c994


How reproducible:


Steps to Reproduce:
1. Create Secret
2. Create StorageClass
3. Create a claim

Actual results:

# oc describe pvc claim1
Name:           claim1
Namespace:      storage-project
StorageClass:   slow
Status:         Pending
Volume:
Labels:         <none>
Capacity:
Access Modes:
Events:
  FirstSeen     LastSeen        Count   From                            SubobjectPath   Type            Reason                  Message
  ---------     --------        -----   ----                            -------------   --------        ------                  -------
  4m            5s              20      {persistentvolume-controller }                  Warning         ProvisioningFailed      Failed to provision volume with StorageClass
"slow": failed to get secret from ["storage-project"/"heketi-secret"]



Expected results:


Additional info:

Comment 2 Humble Chirammal 2016-11-02 10:40:15 UTC
The PC controller does not have enough permission to get the secret so the fetch fails for all the provisioners ( same story with Ceph and others) . The fix for this issue is in merge queue and should be available https://github.com/openshift/origin/pull/11581 soon

Comment 8 Prasanth 2016-11-02 14:05:23 UTC
(In reply to Humble Chirammal from comment #2)
> The PC controller does not have enough permission to get the secret so the
> fetch fails for all the provisioners ( same story with Ceph and others) .
> The fix for this issue is in merge queue and should be available
> https://github.com/openshift/origin/pull/11581 soon

Thanks for the summary! This helps and I hope that this fix will make it to the next OCP 3.4 build (Friday's).

Comment 10 Humble Chirammal 2016-11-10 09:28:34 UTC
The fix is available in OSE v3.4.0.23 or newer. Flipping the status to "ON_QA".

Comment 11 Prasanth 2016-11-10 10:00:23 UTC
Verified as fixed in the latest OCP build:

oc version
oc v3.4.0.24+52fd77b
kubernetes v1.4.0+776c994
features: Basic-Auth GSSAPI Kerberos SPNEGO

Server https://dhcp47-127.lab.eng.blr.redhat.com:8443
openshift v3.4.0.24+52fd77b
kubernetes v1.4.0+776c994


# oc describe pvc claim1
Name:           claim1
Namespace:      storage-project
StorageClass:   gold
Status:         Bound
Volume:         pvc-f134fbf5-a72b-11e6-b89f-005056b3bd15
Labels:         <none>
Capacity:       12Gi
Access Modes:   RWO
No events.

# oc get pvc
NAME      STATUS    VOLUME                                     CAPACITY   ACCESSMODES   AGE
claim1    Bound     pvc-f134fbf5-a72b-11e6-b89f-005056b3bd15   12Gi       RWO           54s

Comment 12 Humble Chirammal 2016-11-10 11:59:56 UTC
(In reply to Prasanth from comment #11)
> Verified as fixed in the latest OCP build:
> 
> oc version
> oc v3.4.0.24+52fd77b
> kubernetes v1.4.0+776c994
> features: Basic-Auth GSSAPI Kerberos SPNEGO
> 
> Server https://dhcp47-127.lab.eng.blr.redhat.com:8443
> openshift v3.4.0.24+52fd77b
> kubernetes v1.4.0+776c994
> 
> 
> # oc describe pvc claim1
> Name:           claim1
> Namespace:      storage-project
> StorageClass:   gold
> Status:         Bound
> Volume:         pvc-f134fbf5-a72b-11e6-b89f-005056b3bd15
> Labels:         <none>
> Capacity:       12Gi
> Access Modes:   RWO
> No events.
> 
> # oc get pvc
> NAME      STATUS    VOLUME                                     CAPACITY  
> ACCESSMODES   AGE
> claim1    Bound     pvc-f134fbf5-a72b-11e6-b89f-005056b3bd15   12Gi      
> RWO           54s

Thanks Prasanth!