Bug 1244358 - rhel-osp-director: unable to register the nodes with SSL undercloud
Summary: rhel-osp-director: unable to register the nodes with SSL undercloud
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-puppet-modules
Version: unspecified
Hardware: All
OS: Linux
high
urgent
Target Milestone: ga
: 7.0 (Kilo)
Assignee: Ivan Chavero
QA Contact: Alexander Chuzhoy
URL:
Whiteboard:
: 1244806 1244995 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-07-17 22:01 UTC by Alexander Chuzhoy
Modified: 2023-02-22 23:02 UTC (History)
9 users (show)

Fixed In Version: openstack-puppet-modules-2015.1.8-8.el7ost
Doc Type: Known Issue
Doc Text:
The Director uses misconfigured HAProxy settings when deploying the Bare Metal and Telemetry services with SSL enabled in the undercloud. This prevents some nodes from registering. To work around this, comment out 'option ssl-hello-chk' under the Bare Metal and Telemetry sections in /etc/haproxy/haproxy.cfg after installing the undercloud.
Clone Of:
Environment:
Last Closed: 2015-08-05 13:29:44 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
/var/log/messages (435.36 KB, application/x-gzip)
2015-07-17 22:09 UTC, Alexander Chuzhoy
no flags Details


Links
System ID Private Priority Status Summary Last Updated
OpenStack gerrit 203298 0 None None None Never
Red Hat Product Errata RHEA-2015:1548 0 normal SHIPPED_LIVE Red Hat Enterprise Linux OpenStack Platform Enhancement Advisory 2015-08-05 17:07:06 UTC

Description Alexander Chuzhoy 2015-07-17 22:01:55 UTC
rhel-osp-director: unable to register the nodes with "openstack baremetal import --json instackenv.json"

Environment:
instack-undercloud-2.1.2-21.el7ost.noarch

Steps to reproduce:
1. Install the undercloud with SSL.
2. Attempt to register the nodes using the (created in advance) instackenv.json file.

Result:
[stack@rhos-compute-node-13 ~]$ openstack baremetal import --json instackenv.json
/usr/lib/python2.7/site-packages/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
/usr/lib/python2.7/site-packages/requests/packages/urllib3/connection.py:251: SecurityWarning: Certificate has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.)
  SecurityWarning
ERROR: openstack ''



Expected result:
The hosts should be registered.

Comment 3 Alexander Chuzhoy 2015-07-17 22:09:30 UTC
Created attachment 1053251 [details]
/var/log/messages

Comment 4 Alexander Chuzhoy 2015-07-17 22:11:02 UTC
There are repeating glance-registry errors:
Jul 17 18:10:06 rhos-compute-node-13 glance-registry: 192.0.2.1 - - [17/Jul/2015 18:10:06] code 400, message Bad request syntax ('\x16\x03\x00\x00y\x01\x00\x00u\x03\x00U\xa9}>HAPROXYSSLCHK')                       
Jul 17 18:10:06 rhos-compute-node-13 glance-registry: 192.0.2.1 - - [17/Jul/2015 18:10:06] "                                                                                                                         
Jul 17 18:10:06 rhos-compute-node-13 glance-registry: Traceback (most recent call last):                                                                                                                             
Jul 17 18:10:06 rhos-compute-node-13 glance-registry: File "/usr/lib/python2.7/site-packages/eventlet/greenpool.py", line 82, in _spawn_n_impl                                                                       
Jul 17 18:10:06 rhos-compute-node-13 glance-registry: func(*args, **kwargs)                                                                                                                                          
Jul 17 18:10:06 rhos-compute-node-13 glance-registry: File "/usr/lib/python2.7/site-packages/eventlet/wsgi.py", line 686, in process_request                                                                         
Jul 17 18:10:06 rhos-compute-node-13 glance-registry: proto.__init__(sock, address, self)                                                                                                                            
Jul 17 18:10:06 rhos-compute-node-13 glance-registry: File "/usr/lib64/python2.7/SocketServer.py", line 649, in __init__                                                                                             
Jul 17 18:10:06 rhos-compute-node-13 glance-registry: self.handle()                                                                                                                                                  
Jul 17 18:10:06 rhos-compute-node-13 glance-registry: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 340, in handle                                                                                             
Jul 17 18:10:06 rhos-compute-node-13 glance-registry: self.handle_one_request()                                                                                                                                      
Jul 17 18:10:06 rhos-compute-node-13 glance-registry: File "/usr/lib/python2.7/site-packages/eventlet/wsgi.py", line 325, in handle_one_request
Jul 17 18:10:06 rhos-compute-node-13 glance-registry: if not self.parse_request():
Jul 17 18:10:06 rhos-compute-node-13 glance-registry: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 286, in parse_request
Jul 17 18:10:06 rhos-compute-node-13 glance-registry: self.send_error(400, "Bad request syntax (%r)" % requestline)
Jul 17 18:10:06 rhos-compute-node-13 glance-registry: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 368, in send_error
Jul 17 18:10:06 rhos-compute-node-13 glance-registry: self.send_response(code, message)
Jul 17 18:10:06 rhos-compute-node-13 glance-registry: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 396, in send_response
Jul 17 18:10:06 rhos-compute-node-13 glance-registry: self.send_header('Date', self.date_time_string())
Jul 17 18:10:06 rhos-compute-node-13 glance-registry: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 401, in send_header
Jul 17 18:10:06 rhos-compute-node-13 glance-registry: self.wfile.write("%s: %s\r\n" % (keyword, value))
Jul 17 18:10:06 rhos-compute-node-13 glance-registry: File "/usr/lib64/python2.7/socket.py", line 324, in write
Jul 17 18:10:06 rhos-compute-node-13 glance-registry: self.flush()
Jul 17 18:10:06 rhos-compute-node-13 glance-registry: File "/usr/lib64/python2.7/socket.py", line 303, in flush
Jul 17 18:10:06 rhos-compute-node-13 glance-registry: self._sock.sendall(view[write_offset:write_offset+buffer_size])
Jul 17 18:10:06 rhos-compute-node-13 glance-registry: File "/usr/lib/python2.7/site-packages/eventlet/greenio/base.py", line 376, in sendall
Jul 17 18:10:06 rhos-compute-node-13 glance-registry: tail = self.send(data, flags)
Jul 17 18:10:06 rhos-compute-node-13 glance-registry: File "/usr/lib/python2.7/site-packages/eventlet/greenio/base.py", line 359, in send
Jul 17 18:10:06 rhos-compute-node-13 glance-registry: total_sent += fd.send(data[total_sent:], flags)
Jul 17 18:10:06 rhos-compute-node-13 glance-registry: error: [Errno 104] Connection reset by peer
Jul 17 18:10:06 rhos-compute-node-13 ironic-api: 192.0.2.1 - - [17/Jul/2015 18:10:06] code 400, message Bad request syntax ('\x16\x03\x00\x00y\x01\x00\x00u\x03\x00U\xa9}>HAPROXYSSLCHK')
Jul 17 18:10:06 rhos-compute-node-13 ironic-api: 192.0.2.1 - - [17/Jul/2015 18:10:06] "

Comment 5 James Slagle 2015-07-17 23:38:05 UTC
This is a regression in the undercloud ssl support caused by:
https://review.openstack.org/#/c/199507/

You can either:
- not use ssl in the undercloud ssl (the default)
- or, after the installation is done, edit /etc/haproxy/haproxy.cfg and comment out the "option ssl-hello-chk" under the "listen ironic section". so after the edit the listen ironic section looks like:

listen ironic
  bind 192.0.2.2:13385 ssl crt /etc/haproxy/test.pem
  bind 192.0.2.3:6385
  balance roundrobin
  option tcplog
  # option ssl-hello-chk
  server 192.0.2.1 192.0.2.1:6385 check fall 5 inter 2000 rise 2

Note that ceilometer api services are also probably not working in the undercloud due to this issue as well.

Comment 6 James Slagle 2015-07-18 01:53:39 UTC
proposed upstream fix https://review.openstack.org/#/c/203298

Comment 7 Alexander Chuzhoy 2015-07-18 17:27:01 UTC
After applying the suggestion in comment #5 and restarting the haproxy - I was able to register the hosts.

Comment 8 Mike Burns 2015-07-20 14:10:10 UTC
*** Bug 1244806 has been marked as a duplicate of this bug. ***

Comment 9 Mike Burns 2015-07-21 11:49:01 UTC
*** Bug 1244995 has been marked as a duplicate of this bug. ***

Comment 10 Mike Burns 2015-07-21 13:28:46 UTC
The patch was backported as part of bug 1236057 so this is resolved.

Comment 11 Omri Hochman 2015-07-21 14:13:37 UTC
Just important note ": 

This Errors "HAPROXYSSLCHK"  flood in /var/log/messages is not happen only with SSL  - Actually it happened to me on *Non-SSL* environment : 

('\x16\x03\x00\x00y\x01\x00\x00u\x03\x00U\xa9}>HAPROXYSSLCHK')    

this Bz should not be handle as it happens only with SSL=true.

Comment 12 Jon Schlueter 2015-07-22 12:35:14 UTC
could use pm and qe ack for this bug.  The fix for this bug made it into errata 20511 OSP7 ga puddle already would like to be able to verify this bug otherwise the bug will push out to a1

Comment 14 Alexander Chuzhoy 2015-07-23 20:38:19 UTC
Verified:

Environment:
instack-undercloud-2.1.2-22.el7ost.noarch


The reported issue is resolved.

Comment 15 Alexander Chuzhoy 2015-07-23 20:38:56 UTC
Environment:
openstack-puppet-modules-2015.1.8-8.el7ost.noarch
instack-undercloud-2.1.2-22.el7ost.noarch

Comment 17 errata-xmlrpc 2015-08-05 13:29:44 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2015:1548


Note You need to log in before you can comment on or make changes to this bug.