Bug 1388316
| Summary: | Failed to provision GlusterFS PV/volume with StorageClass using secret + namespace | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Jianwei Hou <jhou> |
| Component: | Storage | Assignee: | Humble Chirammal <hchiramm> |
| Status: | CLOSED ERRATA | QA Contact: | Jianwei Hou <jhou> |
| Severity: | urgent | Docs Contact: | |
| Priority: | urgent | ||
| Version: | 3.4.0 | CC: | aos-bugs, bchilds, eparis, hchen, jliggitt, jsafrane, rcyriac, tdawson |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | No Doc Update | |
| Doc Text: |
undefined
|
Story Points: | --- |
| Clone Of: | Environment: | ||
| Last Closed: | 2017-01-18 12:44:38 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
The process changed. The new Looks like both Ceph ( https://bugzilla.redhat.com/show_bug.cgi?id=1388368 ) and GlusterFS fails with an error to fetch secrets in Openshift. iic, this was tested in kubernetes upstream and passed. I am not sure whether some special setup is required for openshift. I am adding Jan and Huamin for their thoughts. Gluster provisioner seems not to have access to secrets. * I filled https://github.com/kubernetes/kubernetes/pull/35615 to get nicer message in oc describe pvc: Failed to provision volume with StorageClass "glusterprovisioner1": failed to get secret default/heketi-secret: User "system:serviceaccount:openshift-infra:pv-binder-controller" cannot get secrets in project "default"] (this not really *required* in Origin in 3.4) * I filled https://github.com/openshift/origin/pull/11581 with system:serviceaccount:openshift-infra:pv-binder-controller permission updates *** Bug 1388368 has been marked as a duplicate of this bug. *** (In reply to Jan Safranek from comment #3) > Gluster provisioner seems not to have access to secrets. > > * I filled https://github.com/kubernetes/kubernetes/pull/35615 to get nicer > message in oc describe pvc: > Failed to provision volume with StorageClass "glusterprovisioner1": failed > to get secret default/heketi-secret: User > "system:serviceaccount:openshift-infra:pv-binder-controller" cannot get > secrets in project "default"] > > (this not really *required* in Origin in 3.4) > > * I filled https://github.com/openshift/origin/pull/11581 with > system:serviceaccount:openshift-infra:pv-binder-controller permission updates Thanks Jan!! This has been merged into ose and is in OSE v3.4.0.23 or newer. This is still reproduced on:
openshift v3.4.0.23+24b1a58
kubernetes v1.4.0+776c994
etcd 3.1.0-rc.0
# oc describe pvc glusterc
Name: glusterc
Namespace: jhou
StorageClass: glusterprovisioner1
Status: Pending
Volume:
Labels: <none>
Capacity:
Access Modes:
Events:
FirstSeen LastSeen Count From SubobjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
30s 10s 3 {persistentvolume-controller } Warning ProvisioningFailed Failed to provision volume with StorageClass "glusterprovisioner1": failed to get secret from ["jhou"/"heketi-secret"]
# oc get secret/heketi-secret -n jhou
NAME TYPE DATA AGE
heketi-secret Opaque 1 14m
The secret type must match the provisioner, not be Opaque As Jordan wrote, the secret type must be either "kubernetes.io/glusterfs" or "kubernetes.io/rbd" respectively. Relevant docs change is here: https://github.com/kubernetes/kubernetes.github.io/pull/1594/files Sorry, I should have let you know earlier. @jliggitt @jsafrane Thank you! With secret type being "kubernetes.io/glusterfs", the issue is gone! This bug can be verified now. Could you please change its status to ON_QA? moving to on_qa per comment. This is fixed according to comment 15. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2017:0066 |
Description of problem: Create a StorageClass and configure its parameters to use secret + namespace, the PV creation fails with error: Failed to provision volume with StorageClass "glusterprovisioner1": failed to get secret from ["jhou"/"heketi-secret"] But the secret is present in the target namespace: Version-Release number of selected component (if applicable): openshift v3.4.0.15+9c963ec kubernetes v1.4.0+776c994 etcd 3.1.0-alpha.1 How reproducible: Always Steps to Reproduce: 1. Create a secret in a target namespace 2. Update the StorageClass's parameters field, use secretName and secretNamespace(not restuserkey) 3. Create the StorageClass 4. Create a PVC that uses this StorageClass as provisioner Actual results: No PV provisioned. # oc describe pvc glusterc Name: glusterc Namespace: jhou StorageClass: glusterprovisioner1 Status: Pending Volume: Labels: <none> Capacity: Access Modes: Events: FirstSeen LastSeen Count From SubobjectPath Type Reason Message --------- -------- ----- ---- ------------- -------- ------ ------- 6m 14s 27 {persistentvolume-controller } Warning ProvisioningFailed Failed to provision volume with StorageClass "glusterprovisioner1": failed to get secret from ["jhou"/"heketi-secret"] # oc get secrets heketi-secret -n jhou NAME TYPE DATA AGE heketi-secret Opaque 1 4m # oc get storageclass -o yaml apiVersion: v1 items: - apiVersion: storage.k8s.io/v1beta1 kind: StorageClass metadata: creationTimestamp: 2016-10-25T05:51:05Z name: glusterprovisioner1 resourceVersion: "8078" selfLink: /apis/storage.k8s.io/v1beta1/storageclasses/glusterprovisioner1 uid: 04dbdd54-9a77-11e6-9420-0e4f3633a564 parameters: resturl: http://<hidden> restuser: admin secretName: heketi-secret secretNamespace: jhou provisioner: kubernetes.io/glusterfs kind: List metadata: {} Expected results: Could provision PV/volume successfully Additional info: This feature was tested upstream, test version: Client Version: version.Info{Major:"1", Minor:"5+", GitVersion:"v1.5.0-alpha.1.444+1d323adade5fc6", GitCommit:"1d323adade5fc6ecd898908d2c6cb391c8748835", GitTreeState:"clean", BuildDate:"2016-10-18T02:54:24Z", GoVersion:"go1.6.2", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"5+", GitVersion:"v1.5.0-alpha.1.444+1d323adade5fc6", GitCommit:"1d323adade5fc6ecd898908d2c6cb391c8748835", GitTreeState:"clean", BuildDate:"2016-10-18T02:54:24Z", GoVersion:"go1.6.2", Compiler:"gc", Platform:"linux/amd64"}