Instack messages are flooded with: ('\x16\x03\x00\x00y\x01\x00\x00u\x03\x00U\xadTSHAPROXYSSLCHK') Environment: ------------ instack-undercloud-2.1.2-21.el7ost.noarch instack-0.0.7-1.el7ost.noarch haproxy-1.5.4-4.el7_1.x86_64 openstack-tripleo-puppet-elements-0.0.1-4.el7ost.noarch openstack-puppet-modules-2015.1.8-7.el7ost.noarch puppet-3.6.2-2.el7.noarch Description: ------------- After successful HA deployment the /var/log/messages is flooding with the following Error, probably something with the haproxy configuration [root@rhos-compute-node-18 ~]# cat /etc/haproxy/haproxy.cfg # This file managed by Puppet global daemon group haproxy log /dev/log local0 maxconn 10000 pidfile /var/run/haproxy.pid user haproxy defaults log global mode tcp option tcpka option tcplog retries 3 timeout http-request 10s timeout queue 1m timeout connect 10s timeout client 1m timeout server 1m timeout check 10s listen ceilometer bind 192.168.0.2:8777 bind 192.168.0.3:8777 balance roundrobin option tcplog option ssl-hello-chk server 192.168.0.1 192.168.0.1:8777 check fall 5 inter 2000 rise 2 listen glance_api bind 192.168.0.2:9292 bind 192.168.0.3:9292 option httpchk GET / server 192.168.0.1 192.168.0.1:9292 check fall 5 inter 2000 rise 2 listen glance_registry bind 192.168.0.3:9191 balance roundrobin option tcplog option ssl-hello-chk server 192.168.0.1 192.168.0.1:9191 check fall 5 inter 2000 rise 2 listen haproxy.stats bind 192.168.0.3:1993 mode http stats enable stats uri / listen heat_api bind 192.168.0.2:8004 bind 192.168.0.3:8004 mode http option httpchk GET / server 192.168.0.1 192.168.0.1:8004 check fall 5 inter 2000 rise 2 listen ironic bind 192.168.0.2:6385 bind 192.168.0.3:6385 balance roundrobin option tcplog option ssl-hello-chk server 192.168.0.1 192.168.0.1:6385 check fall 5 inter 2000 rise 2 listen keystone_admin bind 192.168.0.2:35357 bind 192.168.0.3:35357 option httpchk GET / server 192.168.0.1 192.168.0.1:35357 check fall 5 inter 2000 rise 2 listen keystone_public bind 192.168.0.2:5000 bind 192.168.0.3:5000 option httpchk GET / server 192.168.0.1 192.168.0.1:5000 check fall 5 inter 2000 rise 2 listen neutron bind 192.168.0.2:9696 bind 192.168.0.3:9696 option httpchk GET / server 192.168.0.1 192.168.0.1:9696 check fall 5 inter 2000 rise 2 listen nova_metadata bind 192.168.0.3:8775 option httpchk GET / server 192.168.0.1 192.168.0.1:8775 check fall 5 inter 2000 rise 2 listen nova_osapi bind 192.168.0.2:8774 bind 192.168.0.3:8774 option httpchk GET / server 192.168.0.1 192.168.0.1:8774 check fall 5 inter 2000 rise 2 listen rabbitmq bind 192.168.0.3:5672 timeout client 0 timeout server 0 server 192.168.0.1 192.168.0.1:5672 check fall 5 inter 2000 rise 2 listen swift_proxy_server bind 192.168.0.2:8080 bind 192.168.0.3:8080 option httpchk GET /info server 192.168.0.1 192.168.0.1:8080 check fall 5 inter 2000 rise 2 /var/log/messages (instack machine) : -------------------------------------- Jul 20 16:04:35 rhos-compute-node-18 glance-registry: 192.168.0.1 - - [20/Jul/2015 16:04:35] " Jul 20 16:04:35 rhos-compute-node-18 glance-registry: Traceback (most recent call last): Jul 20 16:04:35 rhos-compute-node-18 glance-registry: File "/usr/lib/python2.7/site-packages/eventlet/greenpool.py", line 82, in _spawn_n_impl Jul 20 16:04:35 rhos-compute-node-18 glance-registry: func(*args, **kwargs) Jul 20 16:04:35 rhos-compute-node-18 glance-registry: File "/usr/lib/python2.7/site-packages/eventlet/wsgi.py", line 686, in process_request Jul 20 16:04:35 rhos-compute-node-18 glance-registry: proto.__init__(sock, address, self) Jul 20 16:04:35 rhos-compute-node-18 glance-registry: File "/usr/lib64/python2.7/SocketServer.py", line 649, in __init__ Jul 20 16:04:35 rhos-compute-node-18 glance-registry: self.handle() Jul 20 16:04:35 rhos-compute-node-18 glance-registry: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 340, in handle Jul 20 16:04:35 rhos-compute-node-18 glance-registry: self.handle_one_request() Jul 20 16:04:35 rhos-compute-node-18 glance-registry: File "/usr/lib/python2.7/site-packages/eventlet/wsgi.py", line 325, in handle_one_request Jul 20 16:04:35 rhos-compute-node-18 glance-registry: if not self.parse_request(): Jul 20 16:04:35 rhos-compute-node-18 glance-registry: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 286, in parse_request Jul 20 16:04:35 rhos-compute-node-18 glance-registry: self.send_error(400, "Bad request syntax (%r)" % requestline) Jul 20 16:04:35 rhos-compute-node-18 glance-registry: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 368, in send_error Jul 20 16:04:35 rhos-compute-node-18 glance-registry: self.send_response(code, message) Jul 20 16:04:35 rhos-compute-node-18 glance-registry: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 395, in send_response Jul 20 16:04:35 rhos-compute-node-18 glance-registry: self.send_header('Server', self.version_string()) Jul 20 16:04:35 rhos-compute-node-18 glance-registry: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 401, in send_header Jul 20 16:04:35 rhos-compute-node-18 glance-registry: self.wfile.write("%s: %s\r\n" % (keyword, value)) Jul 20 16:04:35 rhos-compute-node-18 glance-registry: File "/usr/lib64/python2.7/socket.py", line 324, in write Jul 20 16:04:35 rhos-compute-node-18 glance-registry: self.flush() Jul 20 16:04:35 rhos-compute-node-18 glance-registry: File "/usr/lib64/python2.7/socket.py", line 303, in flush Jul 20 16:04:35 rhos-compute-node-18 glance-registry: self._sock.sendall(view[write_offset:write_offset+buffer_size]) Jul 20 16:04:35 rhos-compute-node-18 glance-registry: File "/usr/lib/python2.7/site-packages/eventlet/greenio/base.py", line 376, in sendall Jul 20 16:04:35 rhos-compute-node-18 glance-registry: tail = self.send(data, flags) Jul 20 16:04:35 rhos-compute-node-18 glance-registry: File "/usr/lib/python2.7/site-packages/eventlet/greenio/base.py", line 359, in send Jul 20 16:04:35 rhos-compute-node-18 glance-registry: total_sent += fd.send(data[total_sent:], flags) Jul 20 16:04:35 rhos-compute-node-18 glance-registry: error: [Errno 104] Connection reset by peer Jul 20 16:04:35 rhos-compute-node-18 ironic-api: 192.168.0.1 - - [20/Jul/2015 16:04:35] code 400, message Bad request syntax ('\x16\x03\x00\x00y\x01\x00\x00u\x03\x00U\xadTSHAPROXYSSLCHK') Jul 20 16:04:35 rhos-compute-node-18 ironic-api: 192.168.0.1 - - [20/Jul/2015 16:04:35] " Jul 20 16:04:36 rhos-compute-node-18 proxy-server: Invalid user token - deferring reject downstream Jul 20 16:04:36 rhos-compute-node-18 proxy-server: 192.168.0.1 192.168.0.1 20/Jul/2015/20/04/36 GET /info HTTP/1.0 200 - - - - 738 - tx8f2bb6fcdcd14cd0b44dc-0055ad5454 - 0.0010 - - 1437422676.484005928 1437422676.484970093 - Jul 20 16:04:37 rhos-compute-node-18 glance-registry: 192.168.0.1 - - [20/Jul/2015 16:04:37] code 400, message Bad request syntax ('\x16\x03\x00\x00y\x01\x00\x00u\x03\x00U\xadTUHAPROXYSSLCHK') Jul 20 16:04:37 rhos-compute-node-18 glance-registry: 192.168.0.1 - - [20/Jul/2015 16:04:37] " Jul 20 16:04:37 rhos-compute-node-18 glance-registry: Traceback (most recent call last): Jul 20 16:04:37 rhos-compute-node-18 glance-registry: File "/usr/lib/python2.7/site-packages/eventlet/greenpool.py", line 82, in _spawn_n_impl Jul 20 16:04:37 rhos-compute-node-18 glance-registry: func(*args, **kwargs) Jul 20 16:04:37 rhos-compute-node-18 glance-registry: File "/usr/lib/python2.7/site-packages/eventlet/wsgi.py", line 686, in process_request Jul 20 16:04:37 rhos-compute-node-18 glance-registry: proto.__init__(sock, address, self) Jul 20 16:04:37 rhos-compute-node-18 glance-registry: File "/usr/lib64/python2.7/SocketServer.py", line 649, in __init__ Jul 20 16:04:37 rhos-compute-node-18 glance-registry: self.handle() Jul 20 16:04:37 rhos-compute-node-18 glance-registry: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 340, in handle Jul 20 16:04:37 rhos-compute-node-18 glance-registry: self.handle_one_request() Jul 20 16:04:37 rhos-compute-node-18 glance-registry: File "/usr/lib/python2.7/site-packages/eventlet/wsgi.py", line 325, in handle_one_request Jul 20 16:04:37 rhos-compute-node-18 glance-registry: if not self.parse_request(): Jul 20 16:04:37 rhos-compute-node-18 glance-registry: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 286, in parse_request Jul 20 16:04:37 rhos-compute-node-18 glance-registry: self.send_error(400, "Bad request syntax (%r)" % requestline) Jul 20 16:04:37 rhos-compute-node-18 glance-registry: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 368, in send_error Jul 20 16:04:37 rhos-compute-node-18 glance-registry: self.send_response(code, message) Jul 20 16:04:37 rhos-compute-node-18 glance-registry: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 395, in send_response Jul 20 16:04:37 rhos-compute-node-18 glance-registry: self.send_header('Server', self.version_string()) Jul 20 16:04:37 rhos-compute-node-18 glance-registry: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 401, in send_header Jul 20 16:04:37 rhos-compute-node-18 glance-registry: self.wfile.write("%s: %s\r\n" % (keyword, value)) Jul 20 16:04:37 rhos-compute-node-18 glance-registry: File "/usr/lib64/python2.7/socket.py", line 324, in write Jul 20 16:04:37 rhos-compute-node-18 glance-registry: self.flush() Jul 20 16:04:37 rhos-compute-node-18 glance-registry: File "/usr/lib64/python2.7/socket.py", line 303, in flush Jul 20 16:04:37 rhos-compute-node-18 glance-registry: self._sock.sendall(view[write_offset:write_offset+buffer_size]) Jul 20 16:04:37 rhos-compute-node-18 glance-registry: File "/usr/lib/python2.7/site-packages/eventlet/greenio/base.py", line 376, in sendall Jul 20 16:04:37 rhos-compute-node-18 glance-registry: tail = self.send(data, flags) Jul 20 16:04:37 rhos-compute-node-18 glance-registry: File "/usr/lib/python2.7/site-packages/eventlet/greenio/base.py", line 359, in send Jul 20 16:04:37 rhos-compute-node-18 glance-registry: total_sent += fd.send(data[total_sent:], flags) Jul 20 16:04:37 rhos-compute-node-18 glance-registry: error: [Errno 104] Connection reset by peer Jul 20 16:04:37 rhos-compute-node-18 account-server: Beginning replication run Jul 20 16:04:37 rhos-compute-node-18 account-server: Replication run OVER Jul 20 16:04:37 rhos-compute-node-18 account-server: Attempted to replicate 0 dbs in 0.00425 seconds (0.00000/s) Jul 20 16:04:37 rhos-compute-node-18 account-server: Removed 0 dbs Jul 20 16:04:37 rhos-compute-node-18 account-server: 0 successes, 0 failures Jul 20 16:04:37 rhos-compute-node-18 account-server: no_change:0 ts_repl:0 diff:0 rsync:0 diff_capped:0 hashmatch:0 empty:0 Jul 20 16:04:37 rhos-compute-node-18 ironic-api: 192.168.0.1 - - [20/Jul/2015 16:04:37] code 400, message Bad request syntax ('\x16\x03\x00\x00y\x01\x00\x00u\x03\x00U\xadTUHAPROXYSSLCHK') Jul 20 16:04:37 rhos-compute-node-18 ironic-api: 192.168.0.1 - - [20/Jul/2015 16:04:37] " Jul 20 16:04:38 rhos-compute-node-18 proxy-server: Invalid user token - deferring reject downstream Jul 20 16:04:38 rhos-compute-node-18 proxy-server: 192.168.0.1 192.168.0.1 20/Jul/2015/20/04/38 GET /info HTTP/1.0 200 - - - - 738 - tx343d7d33826c43aa929d5-0055ad5456 - 0.0011 - - 1437422678.489691973 1437422678.490814924 - Jul 20 16:04:39 rhos-compute-node-18 glance-registry: 192.168.0.1 - - [20/Jul/2015 16:04:39] code 400, message Bad request syntax ('\x16\x03\x00\x00y\x01\x00\x00u\x03\x00U\xadTWHAPROXYSSLCHK') Jul 20 16:04:39 rhos-compute-node-18 glance-registry: 192.168.0.1 - - [20/Jul/2015 16:04:39] " Jul 20 16:04:39 rhos-compute-node-18 glance-registry: Traceback (most recent call last): Jul 20 16:04:39 rhos-compute-node-18 glance-registry: File "/usr/lib/python2.7/site-packages/eventlet/greenpool.py", line 82, in _spawn_n_impl Jul 20 16:04:39 rhos-compute-node-18 glance-registry: func(*args, **kwargs) Jul 20 16:04:39 rhos-compute-node-18 glance-registry: File "/usr/lib/python2.7/site-packages/eventlet/wsgi.py", line 686, in process_request Jul 20 16:04:39 rhos-compute-node-18 glance-registry: proto.__init__(sock, address, self) Jul 20 16:04:39 rhos-compute-node-18 glance-registry: File "/usr/lib64/python2.7/SocketServer.py", line 649, in __init__ Jul 20 16:04:39 rhos-compute-node-18 glance-registry: self.handle() Jul 20 16:04:39 rhos-compute-node-18 glance-registry: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 340, in handle Jul 20 16:04:39 rhos-compute-node-18 glance-registry: self.handle_one_request() Jul 20 16:04:39 rhos-compute-node-18 glance-registry: File "/usr/lib/python2.7/site-packages/eventlet/wsgi.py", line 325, in handle_one_request Jul 20 16:04:39 rhos-compute-node-18 glance-registry: if not self.parse_request(): Jul 20 16:04:39 rhos-compute-node-18 glance-registry: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 286, in parse_request Jul 20 16:04:39 rhos-compute-node-18 glance-registry: self.send_error(400, "Bad request syntax (%r)" % requestline) Jul 20 16:04:39 rhos-compute-node-18 glance-registry: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 368, in send_error Jul 20 16:04:39 rhos-compute-node-18 glance-registry: self.send_response(code, message) Jul 20 16:04:39 rhos-compute-node-18 glance-registry: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 396, in send_response Jul 20 16:04:39 rhos-compute-node-18 glance-registry: self.send_header('Date', self.date_time_string()) Jul 20 16:04:39 rhos-compute-node-18 glance-registry: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 401, in send_header Jul 20 16:04:39 rhos-compute-node-18 glance-registry: self.wfile.write("%s: %s\r\n" % (keyword, value)) Jul 20 16:04:39 rhos-compute-node-18 glance-registry: File "/usr/lib64/python2.7/socket.py", line 324, in write Jul 20 16:04:39 rhos-compute-node-18 glance-registry: self.flush() Jul 20 16:04:39 rhos-compute-node-18 glance-registry: File "/usr/lib64/python2.7/socket.py", line 303, in flush Jul 20 16:04:39 rhos-compute-node-18 glance-registry: self._sock.sendall(view[write_offset:write_offset+buffer_size]) Jul 20 16:04:39 rhos-compute-node-18 glance-registry: File "/usr/lib/python2.7/site-packages/eventlet/greenio/base.py", line 376, in sendall Jul 20 16:04:39 rhos-compute-node-18 glance-registry: tail = self.send(data, flags) Jul 20 16:04:39 rhos-compute-node-18 glance-registry: File "/usr/lib/python2.7/site-packages/eventlet/greenio/base.py", line 359, in send Jul 20 16:04:39 rhos-compute-node-18 glance-registry: total_sent += fd.send(data[total_sent:], flags) Jul 20 16:04:39 rhos-compute-node-18 glance-registry: error: [Errno 104] Connection reset by peer Jul 20 16:04:39 rhos-compute-node-18 ironic-api: 192.168.0.1 - - [20/Jul/2015 16:04:39] code 400, message Bad request syntax ('\x16\x03\x00\x00y\x01\x00\x00u\x03\x00U\xadTWHAPROXYSSLCHK') Jul 20 16:04:39 rhos-compute-node-18 ironic-api: 192.168.0.1 - - [20/Jul/2015 16:04:39] " Jul 20 16:04:40 rhos-compute-node-18 proxy-server: Invalid user token - deferring reject downstream Jul 20 16:04:40 rhos-compute-node-18 proxy-server: 192.168.0.1 192.168.0.1 20/Jul/2015/20/04/40 GET /info HTTP/1.0 200 - - - - 738 - txa97e96ef81994f9c89673-0055ad5458 - 0.0011 - - 1437422680.494801998 1437422680.495930910 - Jul 20 16:04:41 rhos-compute-node-18 glance-registry: 192.168.0.1 - - [20/Jul/2015 16:04:41] code 400, message Bad request syntax ('\x16\x03\x00\x00y\x01\x00\x00u\x03\x00U\xadTYHAPROXYSSLCHK') Jul 20 16:04:41 rhos-compute-node-18 glance-registry: 192.168.0.1 - - [20/Jul/2015 16:04:41] " Jul 20 16:04:41 rhos-compute-node-18 glance-registry: Traceback (most recent call last): Jul 20 16:04:41 rhos-compute-node-18 glance-registry: File "/usr/lib/python2.7/site-packages/eventlet/greenpool.py", line 82, in _spawn_n_impl Jul 20 16:04:41 rhos-compute-node-18 glance-registry: func(*args, **kwargs) Jul 20 16:04:41 rhos-compute-node-18 glance-registry: File "/usr/lib/python2.7/site-packages/eventlet/wsgi.py", line 686, in process_request Jul 20 16:04:41 rhos-compute-node-18 glance-registry: proto.__init__(sock, address, self) Jul 20 16:04:41 rhos-compute-node-18 glance-registry: File "/usr/lib64/python2.7/SocketServer.py", line 649, in __init__ Jul 20 16:04:41 rhos-compute-node-18 glance-registry: self.handle() Jul 20 16:04:41 rhos-compute-node-18 glance-registry: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 340, in handle Jul 20 16:04:41 rhos-compute-node-18 glance-registry: self.handle_one_request() Jul 20 16:04:41 rhos-compute-node-18 glance-registry: File "/usr/lib/python2.7/site-packages/eventlet/wsgi.py", line 325, in handle_one_request Jul 20 16:04:41 rhos-compute-node-18 glance-registry: if not self.parse_request(): Jul 20 16:04:41 rhos-compute-node-18 glance-registry: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 286, in parse_request Jul 20 16:04:41 rhos-compute-node-18 glance-registry: self.send_error(400, "Bad request syntax (%r)" % requestline) Jul 20 16:04:41 rhos-compute-node-18 glance-registry: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 368, in send_error Jul 20 16:04:41 rhos-compute-node-18 glance-registry: self.send_response(code, message) Jul 20 16:04:41 rhos-compute-node-18 glance-registry: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 395, in send_response Jul 20 16:04:41 rhos-compute-node-18 glance-registry: self.send_header('Server', self.version_string()) Jul 20 16:04:41 rhos-compute-node-18 glance-registry: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 401, in send_header Jul 20 16:04:41 rhos-compute-node-18 glance-registry: self.wfile.write("%s: %s\r\n" % (keyword, value)) Jul 20 16:04:41 rhos-compute-node-18 glance-registry: File "/usr/lib64/python2.7/socket.py", line 324, in write Jul 20 16:04:41 rhos-compute-node-18 glance-registry: self.flush() Jul 20 16:04:41 rhos-compute-node-18 glance-registry: File "/usr/lib64/python2.7/socket.py", line 303, in flush Jul 20 16:04:41 rhos-compute-node-18 glance-registry: self._sock.sendall(view[write_offset:write_offset+buffer_size]) Jul 20 16:04:41 rhos-compute-node-18 glance-registry: File "/usr/lib/python2.7/site-packages/eventlet/greenio/base.py", line 376, in sendall Jul 20 16:04:41 rhos-compute-node-18 glance-registry: tail = self.send(data, flags) Jul 20 16:04:41 rhos-compute-node-18 glance-registry: File "/usr/lib/python2.7/site-packages/eventlet/greenio/base.py", line 359, in send Jul 20 16:04:41 rhos-compute-node-18 glance-registry: total_sent += fd.send(data[total_sent:], flags) Jul 20 16:04:41 rhos-compute-node-18 glance-registry: error: [Errno 104] Connection reset by peer Jul 20 16:04:41 rhos-compute-node-18 ironic-api: 192.168.0.1 - - [20/Jul/2015 16:04:41] code 400, message Bad request syntax ('\x16\x03\x00\x00y\x01\x00\x00u\x03\x00U\xadTYHAPROXYSSLCHK')
*** This bug has been marked as a duplicate of bug 1244358 ***