Bug 1491331 – failed to provision volume for claim test/pv0001-claim with StorageClass cinder. No suitable endpoint could be found in the service catalog

Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.

Bug 1491331 - failed to provision volume for claim test/pv0001-claim with StorageClass cinder. No suitable endpoint could be found in the service catalog

Summary:	failed to provision volume for claim test/pv0001-claim with StorageClass cind...

Status:	CLOSED ERRATA

Aliases:	None

Product:	OpenShift Container Platform
Classification:	Red Hat
Component:	Storage (Show other bugs)
Sub Component:
Version:	3.6.1
Hardware:	Unspecified Unspecified

Priority	urgent Severity urgent
Target Milestone:	---
Target Release:	3.7.0
Assigned To:	Pavel Pospisil
QA Contact:	Qin Ping
Docs Contact:

URL:
Whiteboard:
Keywords:	Reopened

Depends On:
Blocks:	1533360 1563085
	Show dependency tree / graph

Reported:	2017-09-13 10:10 EDT by Javier Ramirez
Modified:	2018-06-28 09:06 EDT (History)
CC List:	11 users (show)

See Also:
Fixed In Version:
Doc Type:	Bug Fix
Doc Text:	Cause: Dynamic provisioning of Cinder volumes stops working after certain amount of time. OpenShift restart resolves this problem temporarily. That's because OpenShift is not able to re-authenticate to OpenStack Keystone V3. Consequence: It is not possible to create new Cinder volumes using dynamic provisioning. Fix: The OpenShift re-authentication to OpenStack Keystone V3 was fixed. Result: Dynamic provisioning of new Cinder volumes works as documented.
Story Points:	---
Clone Of:
Clones:	1533360 1563085 (view as bug list)
Environment:
Last Closed:	2017-11-28 17:10:32 EST
Type:	Bug
Regression:	---
Mount Type:	---
Documentation:	---
CRM:
Verified Versions:
Category:	---
oVirt Team:	---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---

Attachments	(Terms of Use)
Add an attachment (proposed patch, testcase, etc.)

External Trackers
Tracker	ID	Priority	Status	Summary	Last Updated
Red Hat Product Errata	RHSA-2017:3188	normal	SHIPPED_LIVE	Moderate: Red Hat OpenShift Container Platform 3.7 security, bug, and enhancement update	2017-11-28 21:34:54 EST

Groups: None (edit)

Description Javier Ramirez 2017-09-13 10:10:47 EDT

Description of problem:
Creating a new pvc using cinder storageclass results in the pvc being stuck in a pending state. restart openshift services on masters ( systemctl restart atomic-openshift-*) resolves the issue temporarily and the volume is created in the cinder backend. After some time the pvc is stuck pending again until we restart services. 


Version-Release number of selected component (if applicable):
contianerised install of OCP 3.6 (v3.6.173.0.5-5)

How reproducible:
Always

Steps to Reproduce:
1. Creating a new pvc using cinder storageclass
2.
3.

Actual results:
pvc being stuck in a pending state. 

Expected results:
Volume to be created in the cinder backend.

Error:
"failed to provision volume for claim test/pv0001-claim with StorageClass cinder. No suitable endpoint could be found in the service catalog."

Additional info:
Seems to be similar to: https://github.com/openshift/origin/issues/15979

Workaround seems to be re-configured the cloudprovider openstack.conf to use keystone v2.0 with a local OpenStack user.

Same workaround was mentioned in another bugzilla:
https://bugzilla.redhat.com/show_bug.cgi?id=1490768 - Bug 1490768 - Failed to provision cinder volume, got http response code 300

Comment 2 Pavel Pospisil 2017-09-13 10:15:39 EDT

Please, will you provide cloud.conf configuration file.

Will you, please, also provide OpenStack service catalog, especially the Cinder entries.

Comment 22 Qin Ping 2017-11-07 22:30:44 EST

Verified this bug in OCP3.7 on our openstack

openshift v3.7.0-0.189.0
kubernetes v1.7.6+a08f5eeb62

After reauth, PVC can be created successfully and get into "Bound" phase.

So marked this bug as verified.

Comment 26 errata-xmlrpc 2017-11-28 17:10:32 EST

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:3188

Comment 30 Brian Ward 2018-06-28 09:06:50 EDT

Note: Backport to 3.6 provided in 1527973.

Note You need to log in before you can comment on or make changes to this bug.