Bug 857290

Summary: Keystone catalog fails if not all URLs are defined in an endpoint
Product: Red Hat OpenStack Reporter: Kai(Kimi) Zhang <kimi.zhang>
Component: openstack-keystoneAssignee: Alan Pevec <apevec>
Status: CLOSED ERRATA QA Contact: Nir Magnezi <nmagnezi>
Severity: high Docs Contact:
Priority: unspecified    
Version: 2.0 (Folsom)CC: mmagr
Target Milestone: snapshot2Keywords: Reopened, Triaged
Target Release: 2.1   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: openstack-keystone-2012.2.1-2.el6ost Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-02-12 12:50:48 EST Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
Bug Depends On: 888328    
Bug Blocks:    

Description Kai(Kimi) Zhang 2012-09-13 23:18:43 EDT
Description of problem:


Version-Release number of selected component (if applicable):
 rpm -qa | grep openstack
openstack-nova-common-2012.1.1-15.el6.noarch
openstack-nova-volume-2012.1.1-15.el6.noarch
openstack-nova-2012.1.1-15.el6.noarch
openstack-nova-novncproxy-0.3-10.el6.noarch
openstack-swift-1.4.8-4.el6.noarch
openstack-swift-object-1.4.8-4.el6.noarch
openstack-nova-scheduler-2012.1.1-15.el6.noarch
openstack-nova-cert-2012.1.1-15.el6.noarch
openstack-nova-network-2012.1.1-15.el6.noarch
openstack-nova-api-2012.1.1-15.el6.noarch
openstack-nova-compute-2012.1.1-15.el6.noarch
openstack-dashboard-2012.1.1-2.el6.noarch
openstack-swift-container-1.4.8-4.el6.noarch
openstack-swift-proxy-1.4.8-4.el6.noarch
openstack-utils-2012.1-2.el6.noarch
openstack-keystone-2012.1.1-1.el6.noarch
openstack-glance-2012.1.1-1.el6.noarch
openstack-nova-objectstore-2012.1.1-15.el6.noarch
openstack-nova-console-2012.1.1-15.el6.noarch
openstack-swift-account-1.4.8-4.el6.noarch

I did following two commands:

[root@RHOpenStack ~(keystone_admin)]$ keystone endpoint-create --service_id b8a59b4895bc4ca2bc83486d17c4da3e
+-------------+----------------------------------+
|   Property  |              Value               |
+-------------+----------------------------------+
| adminurl    | None                             |
| id          | 64419531f0d84f33b8229b2d1bc5a225 |
| internalurl | None                             |
| publicurl   | None                             |
| region      | regionOne                        |
| service_id  | b8a59b4895bc4ca2bc83486d17c4da3e |
+-------------+----------------------------------+

[root@RHOpenStack ~(keystone_admin)]$ keystone endpoint-delete 64419531f0d84f33b8229b2d1bc5a225
No handlers could be found for logger "keystoneclient.v2_0.client"
Authorization Failed: An unexpected error prevented the server from fulfilling your request. 'NoneType' object has no attribute 'replace' (HTTP 500)

After this, any keystone command can not be executed, all get the same error:

[root@RHOpenStack ~(keystone_admin)]$ keystone endpoint-list
No handlers could be found for logger "keystoneclient.v2_0.client"
Authorization Failed: An unexpected error prevented the server from fulfilling your request. 'NoneType' object has no attribute 'replace' (HTTP 500)


service openstack-keystone restart does not help.

BTW: The service I tried to create endpoint to was created by command below:
keystone service-create --name=ec2 --type=ec2 --description="EC2 Compatibility Layer"
Comment 2 Alan Pevec 2012-09-14 18:18:30 EDT
I could not reproduce this with keystone 2012.1.1 and keystone 2012.2 folsom rc1
Which version is your python-keystoneclient (rpm -q python-keystoneclient)
Comment 3 Kai(Kimi) Zhang 2012-09-16 23:26:09 EDT
[root@RHOpenStack ~(keystone_admin)]$ rpm -q python-keystoneclient
python-keystoneclient-2012.1-2.el6.noarch

I can reproduce this again by following steps:

1. [root@RHOpenStack ~(keystone_admin)]$ keystone service-create --name=ec2 --type=ec2 --description="EC2 Compatibility Layer"
+-------------+----------------------------------+
|   Property  |              Value               |
+-------------+----------------------------------+
| description | EC2 Compatibility Layer          |
| id          | b4a3686d5ef5454b84d490140c0d5a28 |
| name        | ec2                              |
| type        | ec2                              |
+-------------+----------------------------------+

2. [root@RHOpenStack ~(keystone_admin)]$ keystone endpoint-create --service_id b4a3686d5ef5454b84d490140c0d5a28
+-------------+----------------------------------+
|   Property  |              Value               |
+-------------+----------------------------------+
| adminurl    | None                             |
| id          | 1879ac9a104d43f6968d4f6ada717495 |
| internalurl | None                             |
| publicurl   | None                             |
| region      | regionOne                        |
| service_id  | b4a3686d5ef5454b84d490140c0d5a28 |
+-------------+----------------------------------+

3. [root@RHOpenStack ~(keystone_admin)]$ keystone endpoint-delete 1879ac9a104d43f6968d4f6ada717495
No handlers could be found for logger "keystoneclient.v2_0.client"
Authorization Failed: An unexpected error prevented the server from fulfilling your request. 'NoneType' object has no attribute 'replace' (HTTP 500)

After step 3, all keystone command can't be executed...
Comment 4 Martin Magr 2012-12-18 08:58:53 EST
Reopening since this is still happening in Folsom:

[para@virtual-rhel-beta ~(keystone_admin)]$ rpm -qa python-keystoneclient
python-keystoneclient-0.1.3.27-1.el6.noarch

[para@virtual-rhel-beta ~(keystone_admin)]$ keystone endpoint-list
No handlers could be found for logger "keystoneclient.v2_0.client"
Authorization Failed: An unexpected error prevented the server from fulfilling your request. 'NoneType' object has no attribute 'replace' (HTTP 500)

See also bug #888328.
Comment 5 Alan Pevec 2012-12-18 19:24:57 EST
Upstream fix to be backported:
 https://github.com/openstack/keystone/commit/ba1df90656f9bcff4d769b18042f731a6d295faa
Comment 6 Alan Pevec 2012-12-18 19:27:21 EST
stable/folsom review https://review.openstack.org/18361
Comment 10 Nir Magnezi 2013-02-05 07:26:38 EST
Verified NVR: openstack-keystone-2012.2.1-3.el6ost.noarch

$ rpm -qa | grep keystone 
python-keystoneclient-0.1.3.27-1.el6.noarch
openstack-keystone-2012.2.1-3.el6ost.noarch
python-keystone-2012.2.1-3.el6ost.noarch

1. $ keystone service-create --name=ec2 --type=ec2 --description="EC2 Compatibility Layer"
+-------------+----------------------------------+
|   Property  |              Value               |
+-------------+----------------------------------+
| description |     EC2 Compatibility Layer      |
|      id     | 772841acc0a44eebb2aaaf2d72f09f98 |
|     name    |               ec2                |
|     type    |               ec2                |
+-------------+----------------------------------+

2. $ keystone endpoint-create --service_id 772841acc0a44eebb2aaaf2d72f09f98
+-------------+----------------------------------+
|   Property  |              Value               |
+-------------+----------------------------------+
|   adminurl  |                                  |
|      id     | eb36473ae9fc4ee2b92b293e7ab2c501 |
| internalurl |                                  |
|  publicurl  |                                  |
|    region   |            regionOne             |
|  service_id | 772841acc0a44eebb2aaaf2d72f09f98 |
+-------------+----------------------------------+
3. $ keystone endpoint-delete eb36473ae9fc4ee2b92b293e7ab2c501
Endpoint has been deleted.


** Keystone still works with no errors.
4.$ keystone service-list
+----------------------------------+----------+----------+----------------------------+
|                id                |   name   |   type   |        description         |
+----------------------------------+----------+----------+----------------------------+
| 0d36cfbb6d874916a33c0cb63e883a26 | keystone | identity | OpenStack Identity Service |
| 772841acc0a44eebb2aaaf2d72f09f98 |   ec2    |   ec2    |  EC2 Compatibility Layer   |
| b6cfad966291420cae3e1f8b09e03779 |  glance  |  image   |  Openstack Image Service   |
| b71553cb612a44f1afd57a2a0cbe2d84 | nova_ec2 |   ec2    |        EC2 Service         |
| e18246f6bebe4c37aae3643a6127c151 |   nova   | compute  | Openstack Compute Service  |
| efac7ebe8f0c4a7c92e4b600e438d5dd |  cinder  |  volume  |       Cinder Service       |
+----------------------------------+----------+----------+----------------------------+
Comment 12 errata-xmlrpc 2013-02-12 12:50:48 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2013-0253.html