Bug 1340001

Summary: Keystone auth_url set wrongly in metadata_agent.ini
Product: Red Hat OpenStack Reporter: Robin Cernin <rcernin>
Component: rhosp-directorAssignee: Brent Eagles <beagles>
Status: CLOSED INSUFFICIENT_DATA QA Contact: Arik Chernetsky <achernet>
Severity: medium Docs Contact:
Priority: medium    
Version: 11.0 (Ocata)CC: aludwar, amuller, beagles, dbecker, felipe.alfaro, ihrachys, jslagle, mburns, mcornea, morazi, rcernin, rhel-osp-director-maint, shardy
Target Milestone: zstreamKeywords: Triaged
Target Release: 11.0 (Ocata)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-04-01 12:34:40 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1339488    

Description Robin Cernin 2016-05-26 10:33:40 UTC
Description of problem:

# grep auth_url /etc/neutron/metadata_agent.ini
auth_url = http://192.0.2.10:35357/

If there is RPC issue and the fallback to neutron client happens then this creates error messages with 404's as the resource /tokens resides in http://192.0.2.10:35357/v2.0/tokens and not in http://192.0.2.10:35357/tokens:

{"error": {"message": "The resource could not be found.", "code": 404, "title": "Not Found"}}

When the URL is constructed it uses token_url = self.auth_url + "/tokens" however when deployed with OpenStack director we see that this URL is not existent. And generates the error above.

~~~
File "/usr/lib/python2.7/site-packages/neutronclient/client.py", line 218

209         if self.auth_url is None:
210             raise exceptions.NoAuthURLProvided()
211 
212         token_url = self.auth_url + "/tokens"
213         resp, resp_body = self._cs_request(token_url, "POST",
214                                            body=json.dumps(body),
215                                            content_type="application/json",
216                                            allow_redirects=True)
217         if resp.status_code != 200:
218             raise exceptions.Unauthorized(message=resp_body)
219         if resp_body:
220             try:
221                 resp_body = json.loads(resp_body)
222             except ValueError:
223                 pass
224         else:
225             resp_body = None
226         self._extract_service_catalog(resp_body)
~~~

Thank you,
Kind Regards,
Robin Černín

Comment 2 Ihar Hrachyshka 2016-05-26 10:39:07 UTC
Note that means that once agent configuration is fixed, all metadata agents should be restarted, otherwise we leave those agents prone to breakage on any messaging error.

Comment 3 Felipe Alfaro Solana 2016-06-14 09:23:08 UTC
This is how I fixed Director templates to get auth_url for Neutron metadata agent:

diff --git a/templates/my-overcloud/puppet/controller-puppet.yaml b/templates/my-overcloud/puppet/controller-puppet.yaml
index b58f83e..f95187a 100644
--- a/templates/my-overcloud/puppet/controller-puppet.yaml
+++ b/templates/my-overcloud/puppet/controller-puppet.yaml
@@ -1233,7 +1233,7 @@ resources:
                 neutron::agents::metadata::auth_password: {get_input: neutron_password}
                 neutron_dnsmasq_options: {get_input: neutron_dnsmasq_options}
                 neutron_dsn: {get_input: neutron_dsn}
-                neutron::agents::metadata::auth_url: {get_input: keystone_identity_uri}
+                neutron::agents::metadata::auth_url: {get_input: keystone_auth_uri}
                 neutron::server::notifications::nova_url: {get_input: nova_internal_url}
                 neutron::server::notifications::auth_url: {get_input: nova_admin_auth_url}
                 neutron::server::notifications::tenant_name: 'service'

Comment 4 Steven Hardy 2016-12-02 10:16:15 UTC
So the workaround in comment #3 seems reasonable, but it moves the neutron::agents::metadata::auth_url from the keystone admin endpoint to the public one, which I suspect isn't what we want (you just need to add the suffix to the existing admin endpoint, right?)

See here:

https://github.com/openstack/tripleo-heat-templates/blob/stable/liberty/puppet/controller.yaml#L1039

You're moving:

keystone_identity_uri: { get_param: [EndpointMap, KeystoneAdmin, uri_no_suffix] }

to 

keystone_auth_uri: {get_param: [EndpointMap, KeystoneInternal, uri] }

I suspect you actually need this:

https://github.com/openstack/tripleo-heat-templates/blob/stable/liberty/puppet/controller.yaml#L1150

neutron_admin_auth_url: { get_param: [ EndpointMap, KeystoneAdmin, uri ] }

Please try this workaround - if you can confirm it works OK I'll go ahead and propose it as the fix:


diff --git a/puppet/controller.yaml b/puppet/controller.yaml
index 06585a7..e1d28f1 100644
--- a/puppet/controller.yaml
+++ b/puppet/controller.yaml
@@ -1499,7 +1499,7 @@ resources:
                 neutron::agents::metadata::auth_password: {get_input: neutron_password}
                 neutron_dnsmasq_options: {get_input: neutron_dnsmasq_options}
                 neutron_dsn: {get_input: neutron_dsn}
-                neutron::agents::metadata::auth_url: {get_input: keystone_identity_uri}
+                neutron::agents::metadata::auth_url: {get_input: neutron_admin_auth_url}
                 neutron::db::mysql::password: {get_input: neutron_password}
                 neutron::keystone::auth::public_url: {get_input: neutron_public_url }
                 neutron::keystone::auth::internal_url: {get_input: neutron_internal_url }


Adding needinfo as I'd like confirmation the above fixes the problem from Robin.

Comment 5 James Slagle 2017-02-28 21:36:19 UTC
this is still an issue upstream (OSP10, OSP11, etc). See:
https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/neutron-metadata.yaml#L73

reassigning to DFG:Networking for a fix, then we can consider backporting.

Comment 6 Brent Eagles 2018-01-23 21:24:31 UTC
If I understand correctly, this is a keystone version specific detail and is not necessarily an issue in versions other than the one reported. If we can obtain the feedback requested by Steven in #c4, we can create a version specific fix.

Comment 7 Assaf Muller 2018-04-01 12:34:40 UTC
Robin please see comment 4 and re-open if relevant.