Bug 1491331 - failed to provision volume for claim test/pv0001-claim with StorageClass cinder. No suitable endpoint could be found in the service catalog
Summary: failed to provision volume for claim test/pv0001-claim with StorageClass cind...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Storage
Version: 3.6.1
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: ---
: 3.7.0
Assignee: Pavel Pospisil
QA Contact: Qin Ping
URL:
Whiteboard:
Depends On:
Blocks: 1533360 1563085
TreeView+ depends on / blocked
 
Reported: 2017-09-13 14:10 UTC by Javier Ramirez
Modified: 2018-06-28 13:06 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: Dynamic provisioning of Cinder volumes stops working after certain amount of time. OpenShift restart resolves this problem temporarily. That's because OpenShift is not able to re-authenticate to OpenStack Keystone V3. Consequence: It is not possible to create new Cinder volumes using dynamic provisioning. Fix: The OpenShift re-authentication to OpenStack Keystone V3 was fixed. Result: Dynamic provisioning of new Cinder volumes works as documented.
Clone Of:
: 1533360 1563085 (view as bug list)
Environment:
Last Closed: 2017-11-28 22:10:32 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2017:3188 0 normal SHIPPED_LIVE Moderate: Red Hat OpenShift Container Platform 3.7 security, bug, and enhancement update 2017-11-29 02:34:54 UTC

Description Javier Ramirez 2017-09-13 14:10:47 UTC
Description of problem:
Creating a new pvc using cinder storageclass results in the pvc being stuck in a pending state. restart openshift services on masters ( systemctl restart atomic-openshift-*) resolves the issue temporarily and the volume is created in the cinder backend. After some time the pvc is stuck pending again until we restart services. 


Version-Release number of selected component (if applicable):
contianerised install of OCP 3.6 (v3.6.173.0.5-5)

How reproducible:
Always

Steps to Reproduce:
1. Creating a new pvc using cinder storageclass
2.
3.

Actual results:
pvc being stuck in a pending state. 

Expected results:
Volume to be created in the cinder backend.

Error:
"failed to provision volume for claim test/pv0001-claim with StorageClass cinder. No suitable endpoint could be found in the service catalog."

Additional info:
Seems to be similar to: https://github.com/openshift/origin/issues/15979

Workaround seems to be re-configured the cloudprovider openstack.conf to use keystone v2.0 with a local OpenStack user.

Same workaround was mentioned in another bugzilla:
https://bugzilla.redhat.com/show_bug.cgi?id=1490768 - Bug 1490768 - Failed to provision cinder volume, got http response code 300

Comment 2 Pavel Pospisil 2017-09-13 14:15:39 UTC
Please, will you provide cloud.conf configuration file.

Will you, please, also provide OpenStack service catalog, especially the Cinder entries.

Comment 22 Qin Ping 2017-11-08 03:30:44 UTC
Verified this bug in OCP3.7 on our openstack

openshift v3.7.0-0.189.0
kubernetes v1.7.6+a08f5eeb62

After reauth, PVC can be created successfully and get into "Bound" phase.

So marked this bug as verified.

Comment 26 errata-xmlrpc 2017-11-28 22:10:32 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:3188

Comment 30 Brian Ward 2018-06-28 13:06:50 UTC
Note: Backport to 3.6 provided in 1527973.


Note You need to log in before you can comment on or make changes to this bug.