Bug 1370018 - When querying OpenStack APIs, the links that are returned by APIs are with an http:// and not https://
Summary: When querying OpenStack APIs, the links that are returned by APIs are with an...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: puppet-tripleo
Version: 8.0 (Liberty)
Hardware: x86_64
OS: Linux
medium
medium
Target Milestone: ga
: 10.0 (Newton)
Assignee: RHOS Maint
QA Contact: Rodrigo Duarte
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-08-25 04:56 UTC by VIKRANT
Modified: 2016-12-14 15:53 UTC (History)
17 users (show)

Fixed In Version: puppet-tripleo-5.4.0-3.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-12-14 15:53:18 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2016:2948 normal SHIPPED_LIVE Red Hat OpenStack Platform 10 enhancement update 2016-12-14 19:55:27 UTC
OpenStack gerrit 395521 None None None 2016-11-14 17:35:26 UTC
OpenStack gerrit 400591 None None None 2016-11-22 10:25:13 UTC

Description VIKRANT 2016-08-25 04:56:26 UTC
Description of problem:

When querying OpenStack APIs, the links that are returned by APIs are with an http:// and not https://

Found some workaround for other services like nova, cinder and glance by specifically mentioning "public_endpoint" in configuration file in that is not present then by setting "secure_proxy_ssl_header" in configuration files. But none of these parameter is present in neutron.conf file. 

Version-Release number of selected component (if applicable):
RHEL OSP 8

How reproducible:
Everytime. 

Steps to Reproduce:

1. Try to do curl on endpoint in overcloud setup which is deployed using SSL, command is returning http instead of https. 

curl -s -H "X-Auth-Token: $OS_TOKEN" https://test.net:13696/v2.0/ | jq

2.
3.

Actual results:
It's returning http. 

Expected results:
It should return https instead of http.

Additional info:

Found some old bugs regarding same query but can't the public_endpoint parameter in neutron.conf file. 

 https://bugzilla.redhat.com/show_bug.cgi?id=1237207

Comment 2 Assaf Muller 2016-09-27 12:40:23 UTC
Reassigning to Keystone. I'm not sure if Neutron is doing something wrong but I hope that Keystone developers will be able to shed more light on this issue.

Comment 3 Adam Young 2016-10-07 14:37:28 UTC
Service catalog setup is done by Tripleo, and is not affected by, nor controlled by Keystone.  Keystone is just a repository.

Making this work requires TLS everywhere in Tripleo setup.

Comment 5 Juan Antonio Osorio 2016-10-10 05:52:30 UTC
So, this is handled by the http_proxy_to_wsgi middleware which was introduced to several services as part of the TLS work for TripleO that we did. This is no longer a problem in nova, glance and cinder because they use that middleware (as verified by Marius Cornea). However, neutron is not using this middleware (yet). So I have to propose a patch for neutron, and then enable the middleware in TripleO for it to work. Fortunately, making calls to neutron seems to use the https endpoint (cause it's coming from the keystone catalog).

Anyway, this is a neutron issue (since it doesn't use such middleware) and we need to enable this functionality in TripleO once that lands.

Comment 6 Juan Antonio Osorio 2016-10-10 06:20:25 UTC
Marius Cornea took the time to check more services and these are the ones that need the middleware:
* neutron
* aodh
* ceilometer
* heat-cfn (heat-api uses it already)
* gnocchi

I'm submitting patches for them.

Comment 7 Nathan Kinder 2016-10-12 21:43:41 UTC
Separate bugs have been filed for addressing this issue (by using the HTTPProxyToWSGI middleware) in some of the components mentioned in comment#6:

  neutron    - needs downstream BZ (https://review.openstack.org/384294)
  aodh       - bug#1383180 (https://review.openstack.org/384305)
  ceilometer - bug#1383183 (https://review.openstack.org/384311)
  heat-cfn   - bug#1383185 (https://review.openstack.org/384314)
  gnocchi    - bug#1383186 (https://review.openstack.org/384301)

Additionally, the puppet modules for these services need to be updated.

  puppet-neutron    - needs downstream BZ (patch not submitted upstream yet?)
  puppet-aodh       - needs downstream BZ (https://review.openstack.org/384371)
  puppet-ceilometer - needs downstream BZ (https://review.openstack.org/384366)
  puppet-heat       - needs downstream BZ (patch not submitted upstream yet?)
  puppet-gnocchi    - needs downstream BZ (https://review.openstack.org/384358)
 
This will still need to be addressed in openstack-tripleo-heat-templates once all of the affected services and puppet modules are updated to use the HTTPProxyToWSGI middleware.  This bug should be used for the updates to openstack-tripleo-heat-templates.

Comment 9 Juan Antonio Osorio 2016-10-13 06:29:44 UTC
Nathan, the heat fix doesn't need a change in puppet-heat. The option is already available, since heat-api was using it already.

Comment 16 Mehdi ABAAKOUK 2016-11-22 10:24:25 UTC
The haproxy configuration is still wrong,

haproxy is configured to do TCP proxy and not mode http proxy:

defaults
  ...
  mode  tcp
  ...

So all 'http-request ....' are ignored by haproxy, since it does parse the http layer. And application continue to not receive the X-Forwarded-Proto header.

I would suggest to change the mode to 'http' at least for listeners that need to change headers.

Comment 17 Juan Antonio Osorio 2016-11-22 10:25:56 UTC
Added the change to the CR. Heat worked cause we did have mode http from before.

Comment 22 Rodrigo Duarte 2016-12-06 11:59:23 UTC
verified for

- First, check keystone's service catalog:

[stack@undercloud-0 ~]$ source overcloudrc 
[stack@undercloud-0 ~]$ openstack catalog list
+------------+----------------+---------------------------------------------------------------------------------+
| Name       | Type           | Endpoints                                                                       |
+------------+----------------+---------------------------------------------------------------------------------+
| nova       | compute        | regionOne                                                                       |
|            |                |   publicURL: https://10.0.0.101:13774/v2.1                                      |
|            |                |   internalURL: http://172.17.1.12:8774/v2.1                                     |
|            |                |   adminURL: http://172.17.1.12:8774/v2.1                                        |
|            |                |                                                                                 |
| neutron    | network        | regionOne                                                                       |
|            |                |   publicURL: https://10.0.0.101:13696                                           |
|            |                |   internalURL: http://172.17.1.12:9696                                          |
|            |                |   adminURL: http://172.17.1.12:9696                                             |
|            |                |                                                                                 |
| cinderv2   | volumev2       | regionOne                                                                       |
|            |                |   publicURL: https://10.0.0.101:13776/v2/99a38f7af81c4ee5b3c92bb837674314       |
|            |                |   internalURL: http://172.17.1.12:8776/v2/99a38f7af81c4ee5b3c92bb837674314      |
|            |                |   adminURL: http://172.17.1.12:8776/v2/99a38f7af81c4ee5b3c92bb837674314         |
|            |                |                                                                                 |
| cinderv3   | volumev3       | regionOne                                                                       |
|            |                |   publicURL: https://10.0.0.101:13776/v3/99a38f7af81c4ee5b3c92bb837674314       |
|            |                |   internalURL: http://172.17.1.12:8776/v3/99a38f7af81c4ee5b3c92bb837674314      |
|            |                |   adminURL: http://172.17.1.12:8776/v3/99a38f7af81c4ee5b3c92bb837674314         |
|            |                |                                                                                 |
| aodh       | alarming       | regionOne                                                                       |
|            |                |   publicURL: https://10.0.0.101:13042                                           |
|            |                |   internalURL: http://172.17.1.12:8042                                          |
|            |                |   adminURL: http://172.17.1.12:8042                                             |
|            |                |                                                                                 |
| glance     | image          | regionOne                                                                       |
|            |                |   publicURL: https://10.0.0.101:13292                                           |
|            |                |   internalURL: http://172.17.3.10:9292                                          |
|            |                |   adminURL: http://172.17.3.10:9292                                             |
|            |                |                                                                                 |
| ceilometer | metering       | regionOne                                                                       |
|            |                |   publicURL: https://10.0.0.101:13777                                           |
|            |                |   internalURL: http://172.17.1.12:8777                                          |
|            |                |   adminURL: http://172.17.1.12:8777                                             |
|            |                |                                                                                 |
| heat-cfn   | cloudformation | regionOne                                                                       |
|            |                |   publicURL: https://10.0.0.101:13005/v1                                        |
|            |                |   internalURL: http://172.17.1.12:8000/v1                                       |
|            |                |   adminURL: http://172.17.1.12:8000/v1                                          |
|            |                |                                                                                 |
| cinder     | volume         | regionOne                                                                       |
|            |                |   publicURL: https://10.0.0.101:13776/v1/99a38f7af81c4ee5b3c92bb837674314       |
|            |                |   internalURL: http://172.17.1.12:8776/v1/99a38f7af81c4ee5b3c92bb837674314      |
|            |                |   adminURL: http://172.17.1.12:8776/v1/99a38f7af81c4ee5b3c92bb837674314         |
|            |                |                                                                                 |
| heat       | orchestration  | regionOne                                                                       |
|            |                |   publicURL: https://10.0.0.101:13004/v1/99a38f7af81c4ee5b3c92bb837674314       |
|            |                |   internalURL: http://172.17.1.12:8004/v1/99a38f7af81c4ee5b3c92bb837674314      |
|            |                |   adminURL: http://172.17.1.12:8004/v1/99a38f7af81c4ee5b3c92bb837674314         |
|            |                |                                                                                 |
| swift      | object-store   | regionOne                                                                       |
|            |                |   publicURL: https://10.0.0.101:13808/v1/AUTH_99a38f7af81c4ee5b3c92bb837674314  |
|            |                |   internalURL: http://172.17.3.10:8080/v1/AUTH_99a38f7af81c4ee5b3c92bb837674314 |
|            |                |   adminURL: http://172.17.3.10:8080                                             |
|            |                |                                                                                 |
| gnocchi    | metric         | regionOne                                                                       |
|            |                |   publicURL: https://10.0.0.101:13041                                           |
|            |                |   internalURL: http://172.17.1.12:8041                                          |
|            |                |   adminURL: http://172.17.1.12:8041                                             |
|            |                |                                                                                 |
| keystone   | identity       | regionOne                                                                       |
|            |                |   publicURL: https://10.0.0.101:13000/v2.0                                      |
|            |                |   internalURL: http://172.17.1.12:5000/v2.0                                     |
|            |                |   adminURL: http://192.0.2.8:35357/v2.0                                         |
|            |                |                                                                                 |
+------------+----------------+---------------------------------------------------------------------------------+

- Now, check individual APIs:

* neutron:

[stack@undercloud-0 ~]$ curl -s -H "X-Auth-Token: $OS_TOKEN" https://10.0.0.101:13696 | python -m json.tool
{
    "versions": [
        {
            "id": "v2.0",
            "links": [
                {
                    "href": "https://10.0.0.101:13696/v2.0",
                    "rel": "self"
                }
            ],
            "status": "CURRENT"
        }
    ]
}

* aodh:

[stack@undercloud-0 ~]$ curl -s -H "X-Auth-Token: $OS_TOKEN" https://10.0.0.101:13042 | python -m json.tool
{
    "versions": {
        "values": [
            {
                "id": "v2",
                "links": [
                    {
                        "href": "https://10.0.0.101:13042/v2",
                        "rel": "self"
                    },
                    {
                        "href": "http://docs.openstack.org/",
                        "rel": "describedby",
                        "type": "text/html"
                    }
                ],
                "media-types": [
                    {
                        "base": "application/json",
                        "type": "application/vnd.openstack.telemetry-v2+json"
                    },
                    {
                        "base": "application/xml",
                        "type": "application/vnd.openstack.telemetry-v2+xml"
                    }
                ],
                "status": "stable",
                "updated": "2013-02-13T00:00:00Z"
            }
        ]
    }
}

* ceilometer

[stack@undercloud-0 ~]$ curl -s -H "X-Auth-Token: $OS_TOKEN" https://10.0.0.101:13777 | python -m json.tool
{
    "versions": {
        "values": [
            {
                "id": "v2",
                "links": [
                    {
                        "href": "https://10.0.0.101:13777/v2",
                        "rel": "self"
                    },
                    {
                        "href": "http://docs.openstack.org/",
                        "rel": "describedby",
                        "type": "text/html"
                    }
                ],
                "media-types": [
                    {
                        "base": "application/json",
                        "type": "application/vnd.openstack.telemetry-v2+json"
                    },
                    {
                        "base": "application/xml",
                        "type": "application/vnd.openstack.telemetry-v2+xml"
                    }
                ],
                "status": "stable",
                "updated": "2013-02-13T00:00:00Z"
            }
        ]
    }
}

* heat-cfn

[stack@undercloud-0 ~]$ curl -s -H "X-Auth-Token: $OS_TOKEN" https://10.0.0.101:13005 | python -m json.tool
{
    "versions": [
        {
            "id": "v1.0",
            "links": [
                {
                    "href": "https://10.0.0.101:13005/v1/",
                    "rel": "self"
                }
            ],
            "status": "CURRENT"
        }
    ]
}

* gnocchi

[stack@undercloud-0 ~]$ curl -s -H "X-Auth-Token: $OS_TOKEN" https://10.0.0.101:13041 | python -m json.tool
{
    "versions": [
        {
            "id": "v1.0",
            "links": [
                {
                    "href": "https://10.0.0.101:13041/v1/",
                    "rel": "self"
                }
            ],
            "status": "CURRENT",
            "updated": "2015-03-19"
        }
    ]
}

Comment 23 Rodrigo Duarte 2016-12-06 13:32:39 UTC
(In reply to Rodrigo Duarte from comment #22)
> verified for
> 
> - First, check keystone's service catalog:
> 
> [stack@undercloud-0 ~]$ source overcloudrc 
> [stack@undercloud-0 ~]$ openstack catalog list
> +------------+----------------+----------------------------------------------
> -----------------------------------+
> | Name       | Type           | Endpoints                                   
> |
> +------------+----------------+----------------------------------------------
> -----------------------------------+
> | nova       | compute        | regionOne                                   
> |
> |            |                |   publicURL: https://10.0.0.101:13774/v2.1  
> |
> |            |                |   internalURL: http://172.17.1.12:8774/v2.1 
> |
> |            |                |   adminURL: http://172.17.1.12:8774/v2.1    
> |
> |            |                |                                             
> |
> | neutron    | network        | regionOne                                   
> |
> |            |                |   publicURL: https://10.0.0.101:13696       
> |
> |            |                |   internalURL: http://172.17.1.12:9696      
> |
> |            |                |   adminURL: http://172.17.1.12:9696         
> |
> |            |                |                                             
> |
> | cinderv2   | volumev2       | regionOne                                   
> |
> |            |                |   publicURL:
> https://10.0.0.101:13776/v2/99a38f7af81c4ee5b3c92bb837674314       |
> |            |                |   internalURL:
> http://172.17.1.12:8776/v2/99a38f7af81c4ee5b3c92bb837674314      |
> |            |                |   adminURL:
> http://172.17.1.12:8776/v2/99a38f7af81c4ee5b3c92bb837674314         |
> |            |                |                                             
> |
> | cinderv3   | volumev3       | regionOne                                   
> |
> |            |                |   publicURL:
> https://10.0.0.101:13776/v3/99a38f7af81c4ee5b3c92bb837674314       |
> |            |                |   internalURL:
> http://172.17.1.12:8776/v3/99a38f7af81c4ee5b3c92bb837674314      |
> |            |                |   adminURL:
> http://172.17.1.12:8776/v3/99a38f7af81c4ee5b3c92bb837674314         |
> |            |                |                                             
> |
> | aodh       | alarming       | regionOne                                   
> |
> |            |                |   publicURL: https://10.0.0.101:13042       
> |
> |            |                |   internalURL: http://172.17.1.12:8042      
> |
> |            |                |   adminURL: http://172.17.1.12:8042         
> |
> |            |                |                                             
> |
> | glance     | image          | regionOne                                   
> |
> |            |                |   publicURL: https://10.0.0.101:13292       
> |
> |            |                |   internalURL: http://172.17.3.10:9292      
> |
> |            |                |   adminURL: http://172.17.3.10:9292         
> |
> |            |                |                                             
> |
> | ceilometer | metering       | regionOne                                   
> |
> |            |                |   publicURL: https://10.0.0.101:13777       
> |
> |            |                |   internalURL: http://172.17.1.12:8777      
> |
> |            |                |   adminURL: http://172.17.1.12:8777         
> |
> |            |                |                                             
> |
> | heat-cfn   | cloudformation | regionOne                                   
> |
> |            |                |   publicURL: https://10.0.0.101:13005/v1    
> |
> |            |                |   internalURL: http://172.17.1.12:8000/v1   
> |
> |            |                |   adminURL: http://172.17.1.12:8000/v1      
> |
> |            |                |                                             
> |
> | cinder     | volume         | regionOne                                   
> |
> |            |                |   publicURL:
> https://10.0.0.101:13776/v1/99a38f7af81c4ee5b3c92bb837674314       |
> |            |                |   internalURL:
> http://172.17.1.12:8776/v1/99a38f7af81c4ee5b3c92bb837674314      |
> |            |                |   adminURL:
> http://172.17.1.12:8776/v1/99a38f7af81c4ee5b3c92bb837674314         |
> |            |                |                                             
> |
> | heat       | orchestration  | regionOne                                   
> |
> |            |                |   publicURL:
> https://10.0.0.101:13004/v1/99a38f7af81c4ee5b3c92bb837674314       |
> |            |                |   internalURL:
> http://172.17.1.12:8004/v1/99a38f7af81c4ee5b3c92bb837674314      |
> |            |                |   adminURL:
> http://172.17.1.12:8004/v1/99a38f7af81c4ee5b3c92bb837674314         |
> |            |                |                                             
> |
> | swift      | object-store   | regionOne                                   
> |
> |            |                |   publicURL:
> https://10.0.0.101:13808/v1/AUTH_99a38f7af81c4ee5b3c92bb837674314  |
> |            |                |   internalURL:
> http://172.17.3.10:8080/v1/AUTH_99a38f7af81c4ee5b3c92bb837674314 |
> |            |                |   adminURL: http://172.17.3.10:8080         
> |
> |            |                |                                             
> |
> | gnocchi    | metric         | regionOne                                   
> |
> |            |                |   publicURL: https://10.0.0.101:13041       
> |
> |            |                |   internalURL: http://172.17.1.12:8041      
> |
> |            |                |   adminURL: http://172.17.1.12:8041         
> |
> |            |                |                                             
> |
> | keystone   | identity       | regionOne                                   
> |
> |            |                |   publicURL: https://10.0.0.101:13000/v2.0  
> |
> |            |                |   internalURL: http://172.17.1.12:5000/v2.0 
> |
> |            |                |   adminURL: http://192.0.2.8:35357/v2.0     
> |
> |            |                |                                             
> |
> +------------+----------------+----------------------------------------------
> -----------------------------------+
> 
> - Now, check individual APIs:
> 
> * neutron:
> 
> [stack@undercloud-0 ~]$ curl -s -H "X-Auth-Token: $OS_TOKEN"
> https://10.0.0.101:13696 | python -m json.tool
> {
>     "versions": [
>         {
>             "id": "v2.0",
>             "links": [
>                 {
>                     "href": "https://10.0.0.101:13696/v2.0",
>                     "rel": "self"
>                 }
>             ],
>             "status": "CURRENT"
>         }
>     ]
> }
> 
> * aodh:
> 
> [stack@undercloud-0 ~]$ curl -s -H "X-Auth-Token: $OS_TOKEN"
> https://10.0.0.101:13042 | python -m json.tool
> {
>     "versions": {
>         "values": [
>             {
>                 "id": "v2",
>                 "links": [
>                     {
>                         "href": "https://10.0.0.101:13042/v2",
>                         "rel": "self"
>                     },
>                     {
>                         "href": "http://docs.openstack.org/",
>                         "rel": "describedby",
>                         "type": "text/html"
>                     }
>                 ],
>                 "media-types": [
>                     {
>                         "base": "application/json",
>                         "type": "application/vnd.openstack.telemetry-v2+json"
>                     },
>                     {
>                         "base": "application/xml",
>                         "type": "application/vnd.openstack.telemetry-v2+xml"
>                     }
>                 ],
>                 "status": "stable",
>                 "updated": "2013-02-13T00:00:00Z"
>             }
>         ]
>     }
> }
> 
> * ceilometer
> 
> [stack@undercloud-0 ~]$ curl -s -H "X-Auth-Token: $OS_TOKEN"
> https://10.0.0.101:13777 | python -m json.tool
> {
>     "versions": {
>         "values": [
>             {
>                 "id": "v2",
>                 "links": [
>                     {
>                         "href": "https://10.0.0.101:13777/v2",
>                         "rel": "self"
>                     },
>                     {
>                         "href": "http://docs.openstack.org/",
>                         "rel": "describedby",
>                         "type": "text/html"
>                     }
>                 ],
>                 "media-types": [
>                     {
>                         "base": "application/json",
>                         "type": "application/vnd.openstack.telemetry-v2+json"
>                     },
>                     {
>                         "base": "application/xml",
>                         "type": "application/vnd.openstack.telemetry-v2+xml"
>                     }
>                 ],
>                 "status": "stable",
>                 "updated": "2013-02-13T00:00:00Z"
>             }
>         ]
>     }
> }
> 
> * heat-cfn
> 
> [stack@undercloud-0 ~]$ curl -s -H "X-Auth-Token: $OS_TOKEN"
> https://10.0.0.101:13005 | python -m json.tool
> {
>     "versions": [
>         {
>             "id": "v1.0",
>             "links": [
>                 {
>                     "href": "https://10.0.0.101:13005/v1/",
>                     "rel": "self"
>                 }
>             ],
>             "status": "CURRENT"
>         }
>     ]
> }
> 
> * gnocchi
> 
> [stack@undercloud-0 ~]$ curl -s -H "X-Auth-Token: $OS_TOKEN"
> https://10.0.0.101:13041 | python -m json.tool
> {
>     "versions": [
>         {
>             "id": "v1.0",
>             "links": [
>                 {
>                     "href": "https://10.0.0.101:13041/v1/",
>                     "rel": "self"
>                 }
>             ],
>             "status": "CURRENT",
>             "updated": "2015-03-19"
>         }
>     ]
> }

verified for puppet-tripleo-5.4.0-3.el7ost.noarch

Comment 25 errata-xmlrpc 2016-12-14 15:53:18 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-2948.html


Note You need to log in before you can comment on or make changes to this bug.