Bug 1233956

Summary:	[rhel-osp-director][unified-cli] : Error when attempting to attach volume to instance using unified-cli (endpoint for volumev2 service named cinderv2 not found).
Product:	Red Hat OpenStack	Reporter:	Omri Hochman <ohochman>
Component:	python-rdomanager-oscplugin	Assignee:	Brad P. Crochet <brad>
Status:	CLOSED ERRATA	QA Contact:	Omri Hochman <ohochman>
Severity:	medium	Docs Contact:
Priority:	medium
Version:	Director	CC:	brad, calfonso, christopher_dearborn, gfidente, jslagle, mburns, rhel-osp-director-maint, sclewis
Target Milestone:	ga	Keywords:	Triaged
Target Release:	Director
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:	os-cloud-config-0.2.8-2.el7ost python-rdomanager-oscplugin-0.0.8-7.el7ost	Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2015-08-05 13:54:52 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1238779

Description Omri Hochman 2015-06-19 20:04:33 UTC

[rhel-osp-director][unified-cli] : Error when attempting to attach volume to instance using unified-cli (endpoint for volumev2 service named cinderv2 not found). 

environment:
------------
python-rdomanager-oscplugin-0.0.8-1.el7ost.noarch
python-tuskarclient-0.1.17-3.el7ost.noarch
openstack-tuskar-0.4.18-2.el7ost.noarch
openstack-tuskar-ui-extras-0.0.4-1.el7ost.noarch
openstack-tuskar-ui-0.3.0-2.el7ost.noarch
openstack-heat-templates-0-0.6.20150605git.el7ost.noarch
openstack-heat-api-2015.1.0-3.el7ost.noarch
heat-cfntools-1.2.8-2.el7.noarch
openstack-heat-common-2015.1.0-3.el7ost.noarch
python-heatclient-0.6.0-1.el7ost.noarch
openstack-heat-api-cloudwatch-2015.1.0-3.el7ost.noarch
openstack-tripleo-heat-templates-0.8.6-9.el7ost.noarch



Steps :
-------
(1) Create instance 
(2) Create volume 
(3) attempt to attach volume to instance (using: openstack server add volume)

results:
---------
[stack@instack ~]$ openstack server list
+--------------------------------------+------------+--------+----------------------------------+
| ID                                   | Name       | Status | Networks                         |
+--------------------------------------+------------+--------+----------------------------------+
| 6bff82b7-68cc-46ec-886d-e71d8fdbf98e | vm1        | ACTIVE | default-net=10.0.0.5             |
| 2f93fafc-afed-4afd-a6d8-f5b1b91262e8 | instance01 | ACTIVE | default-net=10.0.0.4, 192.0.2.46 |
+--------------------------------------+------------+--------+----------------------------------+
[stack@instack ~]$ openstack volume list
+--------------------------------------+--------------+-----------+------+-------------+
| ID                                   | Display Name | Status    | Size | Attached to |
+--------------------------------------+--------------+-----------+------+-------------+
| 52402721-6478-400f-86b5-ee131a05cb41 | vol1         | available |    2 |             |
+--------------------------------------+--------------+-----------+------+-------------+


[stack@instack ~]$ openstack server add volume vm1 vol1 
ERROR: openstack The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-ec225bdd-8269-4550-ba79-d2224287ba3c)



nova-api.log:
----------------
2015-06-19 15:46:18.072 19855 ERROR nova.api.openstack [req-ec225bdd-8269-4550-ba79-d2224287ba3c 3a43e4dcad084365b72320fcd7eb1f3d 41cfe531f2f249eeb124f8de3a3d
096d - - -] Caught error: publicURL endpoint for volumev2 service named cinderv2 not found
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack Traceback (most recent call last):
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/api/openstack/__init__.py", line 125, in __call__
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack     return req.get_response(self.application)
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/request.py", line 1296, in send
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack     application, catch_exc_info=False)
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/request.py", line 1260, in call_application
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack     app_iter = application(self.environ, start_response)
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 144, in __call__
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack     return resp(environ, start_response)
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystonemiddleware/auth_token/__init__.py", line 634, in __cal
l__
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack     return self._call_app(env, start_response)
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystonemiddleware/auth_token/__init__.py", line 554, in _call
_app
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack     return self._app(env, _fake_start_response)
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 144, in __call__
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack     return resp(environ, start_response)
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 144, in __call__
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack     return resp(environ, start_response)
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/routes/middleware.py", line 131, in __call__
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack     response = self.app(environ, start_response)
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 144, in __call__
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack     return resp(environ, start_response)
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 130, in __call__
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack     resp = self.call_func(req, *args, **self.kwargs)
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 195, in call_func
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack     return self.func(req, *args, **kwargs)
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py", line 756, in __call__
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack     content_type, body, accept)
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py", line 821, in _process_stack
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack     action_result = self.dispatch(meth, request, action_args)
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py", line 911, in dispatch
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack     return method(req=request, **action_args)
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/api/openstack/compute/contrib/volumes.py", line 305, in create
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack     volume_id, device)
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 230, in wrapped
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack     return func(self, context, target, *args, **kwargs)
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 219, in inner
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack     return function(self, context, instance, *args, **kwargs)
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 200, in inner
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack     return f(self, context, instance, *args, **kw)
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 3035, in attach_volume
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack     disk_bus, device_type)
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 3016, in _attach_volume
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack     volume_bdm.destroy()
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 85, in __exit__
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack     six.reraise(self.type_, self.value, self.tb)
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 3009, in _attach_volume
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack     volume = self.volume_api.get(context, volume_id)
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/volume/cinder.py", line 214, in wrapper
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack     res = method(self, ctx, volume_id, *args, **kwargs)
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/volume/cinder.py", line 286, in get
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack     item = cinderclient(context).volumes.get(volume_id)
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/volume/cinder.py", line 115, in cinderclient
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack     url = _SESSION.get_endpoint(auth, **service_parameters)
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 635, in get_endpoint
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack     return auth.get_endpoint(self, **kwargs)
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystoneclient/auth/identity/base.py", line 219, in get_endpoint
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack     service_name=service_name)
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystoneclient/utils.py", line 318, in inner
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack     return func(*args, **kwargs)
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystoneclient/service_catalog.py", line 254, in url_for
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack     raise exceptions.EndpointNotFound(msg)
2015-06-19 15:46:18.072 19855 TRACE nova.api.openstack EndpointNotFound: publicURL endpoint for volumev2 service named cinderv2 not found

Comment 3 Brad P. Crochet 2015-06-22 13:26:48 UTC

Proposed fix: https://review.gerrithub.io/237219

Comment 4 Brad P. Crochet 2015-06-23 13:24:25 UTC

Upstream patch also necessary: https://review.openstack.org/#/c/194651/

Comment 7 Omri Hochman 2015-07-06 19:01:31 UTC

Verified with
- python-rdomanager-oscplugin-0.0.8-18.el7ost.noarch
- os-cloud-config-0.2.8-4.el7ost.noarch  


[stack@rhos-compute-node-18 ~]$ openstack server list
+--------------------------------------+------+--------+--------------------+
| ID                                   | Name | Status | Networks           |
+--------------------------------------+------+--------+--------------------+
| d3f260a5-3451-4f19-971d-dcb14b59f2be | pipi | BUILD  | ext-net=192.0.2.46 |
+--------------------------------------+------+--------+--------------------+
[stack@rhos-compute-node-18 ~]$ openstack volume list
+--------------------------------------+--------------+-----------+------+-------------+
| ID                                   | Display Name | Status    | Size | Attached to |
+--------------------------------------+--------------+-----------+------+-------------+
| b4f457d8-00ee-4c9e-b8b3-3cb34488d740 | kaka         | available |    2 |             |
+--------------------------------------+--------------+-----------+------+-------------+
[stack@rhos-compute-node-18 ~]$ openstack server add volume pipi kaka
[stack@rhos-compute-node-18 ~]$ 
[stack@rhos-compute-node-18 ~]$ 
[stack@rhos-compute-node-18 ~]$ 
[stack@rhos-compute-node-18 ~]$ 
[stack@rhos-compute-node-18 ~]$ openstack volume list
+--------------------------------------+--------------+--------+------+-------------------------------+
| ID                                   | Display Name | Status | Size | Attached to                   |
+--------------------------------------+--------------+--------+------+-------------------------------+
| b4f457d8-00ee-4c9e-b8b3-3cb34488d740 | kaka         | in-use |    2 | Attached to pipi on /dev/vdb  |
+--------------------------------------+--------------+--------+------+-------------------------------+

Comment 8 Jon Schlueter 2015-07-07 14:31:25 UTC

*** Bug 1233945 has been marked as a duplicate of this bug. ***

Comment 10 errata-xmlrpc 2015-08-05 13:54:52 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2015:1549