Bug 1451508 - backport request neutron: Add http_proxy_to_wsgi to api-paste
Summary: backport request neutron: Add http_proxy_to_wsgi to api-paste
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-neutron
Version: 9.0 (Mitaka)
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: zstream
: 9.0 (Mitaka)
Assignee: Bernard Cafarelli
QA Contact: Alexander Stafeyev
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-05-16 21:09 UTC by Assaf Muller
Modified: 2019-02-17 03:58 UTC (History)
13 users (show)

Fixed In Version: openstack-neutron-8.3.0-10.el7ost
Doc Type: Enhancement
Doc Text:
This enhancement adds http_proxy_to_wsgi to api-paste. As a result, this places the HTTPProxyToWSGI middleware in front of the Neutron-API. The purpose of this middleware is to setup the request URL correctly in case there is a proxy (for example, a loadbalancer such as HAProxy) in front of neutron. For example, when TLS connections are being terminated in the proxy, and you attempt to get the versions from the `/` neutron resource, the protocol is incorrect and reports as 'http' instead of 'https'. The HTTPProxyToWSGI middleware handles such cases and helps keystone discovery work correctly. HTTPProxyToWSGI is off by default and needs to be enabled with a configuration value.
Clone Of: 1451496
Environment:
Last Closed: 2017-07-12 13:18:57 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2017:1736 0 normal SHIPPED_LIVE openstack-neutron bug fix advisory 2017-07-12 17:13:37 UTC

Description Assaf Muller 2017-05-16 21:09:29 UTC
+++ This bug was initially created as a clone of Bug #1451496 +++

Description of problem:
Backport https://review.openstack.org/#/c/384294/5

Add http_proxy_to_wsgi to api-paste This sets up the HTTPProxyToWSGI middleware in front of Neutron-API. The purpose of this middleware is to set up the request URL correctly in case there is a proxy (For instance, a loadbalancer such as HAProxy) in front of Neutron. So, for instance, when TLS connections are being terminated in the proxy, and one tries to get the versions from the / resource of Neutron, one will notice that the protocol is incorrect; It will show 'http' instead of 'https'. So this middleware handles such cases. Thus helping Keystone discovery work correctly. The HTTPProxyToWSGI is off by default and needs to be enabled via a configuration value. 

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

--- Additional comment from Andreas Karis on 2017-05-16 15:23:00 EDT ---

here is a manual workaround that I used:


https://review.openstack.org/#/c/384294/5/etc/api-paste.ini

So we can fix this about the same way as cinder - slightly more complex:

First, fix haproxy
~~~
listen neutron
  bind 10.0.0.4:13696 transparent ssl crt /etc/pki/tls/private/overcloud_endpoint.pem
  bind 172.16.2.4:9696 transparent
  mode http
  http-request set-header X-Forwarded-Proto https if { ssl_fc }
  http-request set-header X-Forwarded-Proto http if !{ ssl_fc }
  server overcloud-controller-0 172.16.2.6:9696 check fall 5 inter 2000 rise 2
~~~

Important are:
~~~
  mode http  
  http-request set-header X-Forwarded-Proto https if { ssl_fc }
  http-request set-header X-Forwarded-Proto http if !{ ssl_fc }
~~~

Change neutron api-paste.ini configuration:
~~~
[root@overcloud-controller-0 neutron]# crudini --set /etc/neutron/neutron.conf DEFAULT api_paste_config /etc/neutron/api-paste.https.ini
[root@overcloud-controller-0 neutron]# crudini --get /etc/neutron/neutron.conf DEFAULT api_paste_config
/etc/neutron/api-paste.https.ini
~~~

Create new /etc/neutron/api-paste.https.ini file:
~~~
[root@overcloud-controller-0 neutron]# cat /etc/neutron/api-paste.https.ini 
[composite:neutron]
use = egg:Paste#urlmap
/: neutronversions_composite
/v2.0: neutronapi_v2_0

[composite:neutronapi_v2_0]
use = call:neutron.auth:pipeline_factory
noauth = http_proxy_to_wsgi request_id catch_errors extensions neutronapiapp_v2_0
keystone = http_proxy_to_wsgi request_id catch_errors authtoken keystonecontext extensions neutronapiapp_v2_0

[composite:neutronversions_composite]
use = call:neutron.auth:pipeline_factory
noauth = http_proxy_to_wsgi neutronversions
keystone = http_proxy_to_wsgi neutronversions

[filter:request_id]
paste.filter_factory = oslo_middleware:RequestId.factory

[filter:catch_errors]
paste.filter_factory = oslo_middleware:CatchErrors.factory

[filter:http_proxy_to_wsgi]
paste.filter_factory = oslo_middleware.http_proxy_to_wsgi:HTTPProxyToWSGI.factory

[filter:keystonecontext]
paste.filter_factory = neutron.auth:NeutronKeystoneContext.factory

[filter:authtoken]
paste.filter_factory = keystonemiddleware.auth_token:filter_factory

[filter:extensions]
paste.filter_factory = neutron.api.extensions:plugin_aware_extension_middleware_factory

[app:neutronversions]
paste.app_factory = neutron.api.versions:Versions.factory

[app:neutronapiapp_v2_0]
paste.app_factory = neutron.api.v2.router:APIRouter.factory
~~~


Restart all neutron services:
~~~
systemctl list-units | grep neutron | awk '{print $1}' | xargs -I {} systemctl restart {}
~~~

Verify:
~~~
[stack@undercloud-2 ~]$ curl https://osp.example.net:13696/
{"versions": [{"status": "CURRENT", "id": "v2.0", "links": [{"href": "https://osp.example.net:13696/v2.0", "rel": "self"}]}]}[stack@undercloud-2 ~]$
~~~

--- Additional comment from Andreas Karis on 2017-05-16 16:08:23 EDT ---

https://access.redhat.com/solutions/3034681

Comment 2 Bernard Cafarelli 2017-05-30 13:24:49 UTC
It may require an additional backport, waiting for update in bug #1451496

Comment 4 Alexander Stafeyev 2017-06-12 11:50:41 UTC
Hi, 

Can you assist with reproduction steps pls ? 

tnx

Comment 5 Bernard Cafarelli 2017-06-12 12:57:08 UTC
Andreas Karis in original bug may have some more info, but here is a base outline:
* have haproxy in front of neutron-server, with TLS enabled
* enable in server.conf enable_proxy_headers_parsing in oslo_middleware section
* run a request on /, the answer should have pointers to "https://" URLs, not "http://" as would appear with the parameter enabled

Code-wise, the change should show the mentioned parameter in neutron.conf, andd http_proxy_to_wsgi step in composites sections in api-paste.ini

Comment 7 Andreas Karis 2017-06-14 21:03:05 UTC
For more info about how to set up haproxy, have a look here: https://access.redhat.com/solutions/3034681   (this was for OSP 8)

Comment 20 errata-xmlrpc 2017-07-12 13:18:57 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:1736


Note You need to log in before you can comment on or make changes to this bug.