Bug 1351255 - nova-api not properly configure secure_proxy_ssl_header option in nova.conf when using HAProxy and SSL
Summary: nova-api not properly configure secure_proxy_ssl_header option in nova.conf w...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates
Version: 9.0 (Mitaka)
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ga
: 9.0 (Mitaka)
Assignee: Marios Andreou
QA Contact: Daniel Mellado
URL:
Whiteboard:
Depends On:
Blocks: 1568469 1631473
TreeView+ depends on / blocked
 
Reported: 2016-06-29 14:23 UTC by Arx Cruz
Modified: 2018-09-20 16:21 UTC (History)
12 users (show)

Fixed In Version: openstack-tripleo-heat-templates-2.0.0-26.el7ost
Doc Type: Bug Fix
Doc Text:
Prior to this update, the secure_proxy_ssl_header option for Compute was not being set in nova.conf by Red Hat OpenStack Platform director (as discussed in the upstream bug https://bugs.launchpad.net/tripleo/+bug/1606863). Consequently, when haproxy and SSL were enabled for the director deployment, nova-api could not handle service requests since it was not configured to handle the "X-Forwarded-Proto" header in HTTP requests. In particular, the tempest.api.compute.test_versions.TestVersions.test_get_version_details tests failed with the error: 'Connection aborted.', BadStatusLine("''",) With this update, the secure_proxy_ssl_header is now set to the appropriate value (X-Forwarded-Proto) for director deployments, see https://review.openstack.org/#/c/347806/ for more details. As a result, the nova-api service should now be able to handle service requests correctly when haproxy and SSL are enabled for the director deployment.
Clone Of:
: 1568469 (view as bug list)
Environment:
Last Closed: 2016-08-24 13:01:32 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1606863 0 None None None 2016-07-27 13:00:29 UTC
OpenStack gerrit 347806 0 'None' MERGED Set secure_proxy_ssl_header 'HTTP_X_FORWARDED_PROTO' for nova-api 2020-07-14 05:30:51 UTC
Red Hat Product Errata RHEA-2016:1762 0 normal SHIPPED_LIVE Red Hat OpenStack Platform 9 director Advisory 2016-08-24 16:59:57 UTC

Description Arx Cruz 2016-06-29 14:23:42 UTC
Description of problem:
tempest.api.compute.test_versions.TestVersions.test_get_version_details fails with BadStatusLine because it doesn't set the option secure_proxy_ssl_header when OSPD is installed with HAProxy and SSL enabled

Version-Release number of selected component (if applicable):
openstack-nova-scheduler-13.1.0-1.el7ost.noarch
python-novaclient-3.3.0-1.el7ost.noarch
openstack-nova-common-13.1.0-1.el7ost.noarch
openstack-nova-compute-13.1.0-1.el7ost.noarch
openstack-nova-api-13.1.0-1.el7ost.noarch
openstack-nova-cert-13.1.0-1.el7ost.noarch
openstack-nova-conductor-13.1.0-1.el7ost.noarch
openstack-nova-console-13.1.0-1.el7ost.noarch
python-nova-13.1.0-1.el7ost.noarch
openstack-nova-novncproxy-13.1.0-1.el7ost.noarch


How reproducible:
Whenever you deploy OSPD 9 with HAProxy and SSL enabled

Steps to Reproduce:
1. Install OSPD with HA and SSL enabled
2. Setup tempest
3. Run tempest test tempest.api.compute.test_versions.TestVersions.test_get_version_details

Actual results:
Fails with the following:

ythonlogging:'': {{{
2016-06-29 04:18:25,940 9368 WARNING  [urllib3.connectionpool] Retrying (Retry(total=9, connect=None, read=None, redirect=5)) after connection broken by 'ProtocolError('Connection aborted.', BadStatusLine("''",))': /v2/
2016-06-29 04:18:25,941 9368 WARNING  [urllib3.connectionpool] Retrying (Retry(total=8, connect=None, read=None, redirect=5)) after connection broken by 'ProtocolError('Connection aborted.', BadStatusLine("''",))': /v2/
2016-06-29 04:18:25,942 9368 WARNING  [urllib3.connectionpool] Retrying (Retry(total=7, connect=None, read=None, redirect=5)) after connection broken by 'ProtocolError('Connection aborted.', BadStatusLine("''",))': /v2/
2016-06-29 04:18:25,943 9368 WARNING  [urllib3.connectionpool] Retrying (Retry(total=6, connect=None, read=None, redirect=5)) after connection broken by 'ProtocolError('Connection aborted.', BadStatusLine("''",))': /v2/
2016-06-29 04:18:25,944 9368 WARNING  [urllib3.connectionpool] Retrying (Retry(total=5, connect=None, read=None, redirect=5)) after connection broken by 'ProtocolError('Connection aborted.', BadStatusLine("''",))': /v2/
2016-06-29 04:18:25,944 9368 WARNING  [urllib3.connectionpool] Retrying (Retry(total=4, connect=None, read=None, redirect=5)) after connection broken by 'ProtocolError('Connection aborted.', BadStatusLine("''",))': /v2/
2016-06-29 04:18:25,945 9368 WARNING  [urllib3.connectionpool] Retrying (Retry(total=3, connect=None, read=None, redirect=5)) after connection broken by 'ProtocolError('Connection aborted.', BadStatusLine("''",))': /v2/
2016-06-29 04:18:25,946 9368 WARNING  [urllib3.connectionpool] Retrying (Retry(total=2, connect=None, read=None, redirect=5)) after connection broken by 'ProtocolError('Connection aborted.', BadStatusLine("''",))': /v2/
2016-06-29 04:18:25,947 9368 WARNING  [urllib3.connectionpool] Retrying (Retry(total=1, connect=None, read=None, redirect=5)) after connection broken by 'ProtocolError('Connection aborted.', BadStatusLine("''",))': /v2/
2016-06-29 04:18:25,948 9368 WARNING  [urllib3.connectionpool] Retrying (Retry(total=0, connect=None, read=None, redirect=5)) after connection broken by 'ProtocolError('Connection aborted.', BadStatusLine("''",))': /v2/
}}}

Traceback (most recent call last):
  File "tempest/api/compute/test_versions.py", line 69, in test_get_version_details
    result = self.versions_client.get_version_by_url(link['href'])
  File "tempest/lib/services/compute/versions_client.py", line 58, in get_version_by_url
    {'X-Auth-Token': self.token})
  File "tempest/lib/common/rest_client.py", line 578, in raw_request
    body=body, chunked=chunked)
  File "tempest/lib/common/http.py", line 54, in request
    *args, **new_kwargs)
  File "/home/stack/tempest/.tox/py27/lib/python2.7/site-packages/urllib3/request.py", line 69, in request
    **urlopen_kw)
  File "/home/stack/tempest/.tox/py27/lib/python2.7/site-packages/urllib3/request.py", line 90, in request_encode_url
    return self.urlopen(method, url, **extra_kw)
  File "/home/stack/tempest/.tox/py27/lib/python2.7/site-packages/urllib3/poolmanager.py", line 248, in urlopen
    response = conn.urlopen(method, u.request_uri, **kw)
  File "/home/stack/tempest/.tox/py27/lib/python2.7/site-packages/urllib3/connectionpool.py", line 668, in urlopen
    release_conn=release_conn, **response_kw)
  File "/home/stack/tempest/.tox/py27/lib/python2.7/site-packages/urllib3/connectionpool.py", line 668, in urlopen
    release_conn=release_conn, **response_kw)
  File "/home/stack/tempest/.tox/py27/lib/python2.7/site-packages/urllib3/connectionpool.py", line 668, in urlopen
    release_conn=release_conn, **response_kw)
  File "/home/stack/tempest/.tox/py27/lib/python2.7/site-packages/urllib3/connectionpool.py", line 668, in urlopen
    release_conn=release_conn, **response_kw)
  File "/home/stack/tempest/.tox/py27/lib/python2.7/site-packages/urllib3/connectionpool.py", line 668, in urlopen
    release_conn=release_conn, **response_kw)
  File "/home/stack/tempest/.tox/py27/lib/python2.7/site-packages/urllib3/connectionpool.py", line 668, in urlopen
    release_conn=release_conn, **response_kw)
  File "/home/stack/tempest/.tox/py27/lib/python2.7/site-packages/urllib3/connectionpool.py", line 668, in urlopen
    release_conn=release_conn, **response_kw)
  File "/home/stack/tempest/.tox/py27/lib/python2.7/site-packages/urllib3/connectionpool.py", line 668, in urlopen
    release_conn=release_conn, **response_kw)
  File "/home/stack/tempest/.tox/py27/lib/python2.7/site-packages/urllib3/connectionpool.py", line 668, in urlopen
    release_conn=release_conn, **response_kw)
  File "/home/stack/tempest/.tox/py27/lib/python2.7/site-packages/urllib3/connectionpool.py", line 668, in urlopen
    release_conn=release_conn, **response_kw)
  File "/home/stack/tempest/.tox/py27/lib/python2.7/site-packages/urllib3/connectionpool.py", line 640, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/home/stack/tempest/.tox/py27/lib/python2.7/site-packages/urllib3/util/retry.py", line 287, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host=u'10.0.0.101', port=13774): Max retries exceeded with url: /v2/ (Caused by ProtocolError('Connection aborted.', BadStatusLine("''",)))


Expected results:
Test pass

Additional info:
Editing /etc/nova/nova.conf on the controller, and adds the option secure_proxy_ssl_header=HTTP_X_FORWARDED_PROTO and restarting openstack-nova-api fix the problem.

Comment 2 Jason Guiditta 2016-07-06 13:43:50 UTC
Looking at puppet-nova, I see this is being set as so:
nova/manifests/api.pp:  $secure_proxy_ssl_header   = $::os_service_default,

So my guess is tht just needs to configure this, in the same manner as is done elsewhere

Comment 3 Marios Andreou 2016-07-27 13:05:56 UTC
moved to ON_DEV -  I filed an upstream bug for this (required for stable/mitaka, which is where this is needed here) https://bugs.launchpad.net/tripleo/+bug/1606863 - reviews to master and mitaka linked.

Comment 14 Daniel Mellado 2016-08-24 11:11:07 UTC
Working with latest puddle

# rhos-release 9-director   -p 2016-08-19.3

[stack@undercloud-0 tempest]$ python -m testtools.run tempest.api.compute.test_versions.TestVersions.test_get_version_details
Tests running...

Ran 1 test in 2.563s
OK

openstack-nova-compute-13.1.1-2.el7ost.noarch
openstack-nova-console-13.1.1-2.el7ost.noarch
python-novaclient-3.3.1-1.el7ost.noarch
python-nova-13.1.1-2.el7ost.noarch
openstack-nova-novncproxy-13.1.1-2.el7ost.noarch
openstack-nova-common-13.1.1-2.el7ost.noarch
openstack-nova-api-13.1.1-2.el7ost.noarch
openstack-nova-conductor-13.1.1-2.el7ost.noarch
openstack-nova-cert-13.1.1-2.el7ost.noarch
openstack-nova-scheduler-13.1.1-2.el7ost.noarch

Comment 18 errata-xmlrpc 2016-08-24 13:01:32 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-1762.html


Note You need to log in before you can comment on or make changes to this bug.