Bug 1264200

Summary: seeing multiple invalid user token in message log on undercloud
Product: Red Hat OpenStack Reporter: bigswitch <rhosp-bugs-internal>
Component: rhosp-directorAssignee: chris alfonso <calfonso>
Status: CLOSED DUPLICATE QA Contact: yeylon <yeylon>
Severity: low Docs Contact:
Priority: unspecified    
Version: 7.0 (Kilo)CC: hbrock, jslagle, mburns, rhel-osp-director-maint, srevivo
Target Milestone: ---   
Target Release: 7.0 (Kilo)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-09-22 16:26:02 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description bigswitch 2015-09-17 20:50:15 UTC
Description of problem:
I am seeing multiple invalid user token in message log. not sure if this is indicating something is broken

Sep 17 13:37:52 dell-undercloud ironic-api: error: [Errno 32] Broken pipe
Sep 17 13:37:53 dell-undercloud proxy-server: Invalid user token - deferring reject downstream
Sep 17 13:37:53 dell-undercloud proxy-server: 192.0.2.1 192.0.2.1 17/Sep/2015/20/37/53 GET /info HTTP/1.0 200 - - - - 738 - txe13d02b5f046434088dcb-0055fb24a1 - 0.0005 - - 1442522273.167885065 1442522273.168349028 -
Sep 17 13:37:53 dell-undercloud account-server: Beginning replication run
Sep 17 13:37:53 dell-undercloud account-server: Replication run OVER
Sep 17 13:37:53 dell-undercloud account-server: Attempted to replicate 1 dbs in 0.00306 seconds (326.47054/s)
Sep 17 13:37:53 dell-undercloud account-server: Removed 0 dbs
Sep 17 13:37:53 dell-undercloud account-server: 0 successes, 0 failures
Sep 17 13:37:53 dell-undercloud account-server: no_change:0 ts_repl:0 diff:0 rsync:0 diff_capped:0 hashmatch:0 empty:0
Sep 17 13:37:55 dell-undercloud ironic-api: 192.0.2.1 - - [17/Sep/2015 13:37:55] "GET / HTTP/1.0" 200 328
Sep 17 13:37:55 dell-undercloud ironic-api: Traceback (most recent call last):
Sep 17 13:37:55 dell-undercloud ironic-api: File "/usr/lib64/python2.7/wsgiref/handlers.py", line 86, in run
Sep 17 13:37:55 dell-undercloud ironic-api: self.finish_response()
Sep 17 13:37:55 dell-undercloud ironic-api: File "/usr/lib64/python2.7/wsgiref/handlers.py", line 128, in finish_response
Sep 17 13:37:55 dell-undercloud ironic-api: self.write(data)
Sep 17 13:37:55 dell-undercloud ironic-api: File "/usr/lib64/python2.7/wsgiref/handlers.py", line 212, in write
Sep 17 13:37:55 dell-undercloud ironic-api: self.send_headers()
Sep 17 13:37:55 dell-undercloud ironic-api: File "/usr/lib64/python2.7/wsgiref/handlers.py", line 270, in send_headers
Sep 17 13:37:55 dell-undercloud ironic-api: self.send_preamble()
Sep 17 13:37:55 dell-undercloud ironic-api: File "/usr/lib64/python2.7/wsgiref/handlers.py", line 197, in send_preamble
Sep 17 13:37:55 dell-undercloud ironic-api: self._write('Server: %s\r\n' % self.server_software)
Sep 17 13:37:55 dell-undercloud ironic-api: File "/usr/lib64/python2.7/socket.py", line 324, in write
Sep 17 13:37:55 dell-undercloud ironic-api: self.flush()
Sep 17 13:37:55 dell-undercloud ironic-api: File "/usr/lib64/python2.7/socket.py", line 303, in flush
Sep 17 13:37:55 dell-undercloud ironic-api: self._sock.sendall(view[write_offset:write_offset+buffer_size])
Sep 17 13:37:55 dell-undercloud ironic-api: File "/usr/lib/python2.7/site-packages/eventlet/greenio/base.py", line 376, in sendall
Sep 17 13:37:55 dell-undercloud ironic-api: tail = self.send(data, flags)
Sep 17 13:37:55 dell-undercloud ironic-api: File "/usr/lib/python2.7/site-packages/eventlet/greenio/base.py", line 359, in send
Sep 17 13:37:55 dell-undercloud ironic-api: total_sent += fd.send(data[total_sent:], flags)
Sep 17 13:37:55 dell-undercloud ironic-api: error: [Errno 104] Connection reset by peer
Sep 17 13:37:55 dell-undercloud ironic-api: 192.0.2.1 - - [17/Sep/2015 13:37:55] "GET / HTTP/1.0" 500 59
Sep 17 13:37:55 dell-undercloud ironic-api: Traceback (most recent call last):
Sep 17 13:37:55 dell-undercloud ironic-api: File "/usr/lib64/python2.7/SocketServer.py", line 593, in process_request_thread
Sep 17 13:37:55 dell-undercloud ironic-api: self.finish_request(request, client_address)
Sep 17 13:37:55 dell-undercloud ironic-api: File "/usr/lib64/python2.7/SocketServer.py", line 334, in finish_request
Sep 17 13:37:55 dell-undercloud ironic-api: self.RequestHandlerClass(request, client_address, self)
Sep 17 13:37:55 dell-undercloud ironic-api: File "/usr/lib64/python2.7/SocketServer.py", line 651, in __init__
Sep 17 13:37:55 dell-undercloud ironic-api: self.finish()
Sep 17 13:37:55 dell-undercloud ironic-api: File "/usr/lib64/python2.7/SocketServer.py", line 710, in finish
Sep 17 13:37:55 dell-undercloud ironic-api: self.wfile.close()
Sep 17 13:37:55 dell-undercloud ironic-api: File "/usr/lib64/python2.7/socket.py", line 279, in close
Sep 17 13:37:55 dell-undercloud ironic-api: self.flush()
Sep 17 13:37:55 dell-undercloud ironic-api: File "/usr/lib64/python2.7/socket.py", line 303, in flush
Sep 17 13:37:55 dell-undercloud ironic-api: self._sock.sendall(view[write_offset:write_offset+buffer_size])
Sep 17 13:37:55 dell-undercloud ironic-api: File "/usr/lib/python2.7/site-packages/eventlet/greenio/base.py", line 376, in sendall
Sep 17 13:37:55 dell-undercloud ironic-api: tail = self.send(data, flags)
Sep 17 13:37:55 dell-undercloud ironic-api: File "/usr/lib/python2.7/site-packages/eventlet/greenio/base.py", line 359, in send
Sep 17 13:37:55 dell-undercloud ironic-api: total_sent += fd.send(data[total_sent:], flags)
Sep 17 13:37:55 dell-undercloud ironic-api: error: [Errno 32] Broken pipe
Sep 17 13:37:55 dell-undercloud proxy-server: Invalid user token - deferring reject downstream
Sep 17 13:37:55 dell-undercloud proxy-server: 192.0.2.1 192.0.2.1 17/Sep/2015/20/37/55 GET /info HTTP/1.0 200 - - - - 738 - txb4f5c9c76369408dad618-0055fb24a3 - 0.0003 - - 1442522275.170485020 1442522275.170789003 -
Sep 17 13:37:57 dell-undercloud ironic-api: 192.0.2.1 - - [17/Sep/2015 13:37:57] "GET / HTTP/1.0" 200 328
Sep 17 13:37:57 dell-undercloud ironic-api: Traceback (most recent call last):
Sep 17 13:37:57 dell-undercloud ironic-api: File "/usr/lib64/python2.7/wsgiref/handlers.py", line 86, in run
Sep 17 13:37:57 dell-undercloud ironic-api: self.finish_response()
Sep 17 13:37:57 dell-undercloud ironic-api: File "/usr/lib64/python2.7/wsgiref/handlers.py", line 128, in finish_response
Sep 17 13:37:57 dell-undercloud ironic-api: self.write(data)
Sep 17 13:37:57 dell-undercloud ironic-api: File "/usr/lib64/python2.7/wsgiref/handlers.py", line 212, in write
Sep 17 13:37:57 dell-undercloud ironic-api: self.send_headers()
Sep 17 13:37:57 dell-undercloud ironic-api: File "/usr/lib64/python2.7/wsgiref/handlers.py", line 270, in send_headers
Sep 17 13:37:57 dell-undercloud ironic-api: self.send_preamble()
Sep 17 13:37:57 dell-undercloud ironic-api: File "/usr/lib64/python2.7/wsgiref/handlers.py", line 197, in send_preamble
Sep 17 13:37:57 dell-undercloud ironic-api: self._write('Server: %s\r\n' % self.server_software)
Sep 17 13:37:57 dell-undercloud ironic-api: File "/usr/lib64/python2.7/socket.py", line 324, in write
Sep 17 13:37:57 dell-undercloud ironic-api: self.flush()
Sep 17 13:37:57 dell-undercloud ironic-api: File "/usr/lib64/python2.7/socket.py", line 303, in flush
Sep 17 13:37:57 dell-undercloud ironic-api: self._sock.sendall(view[write_offset:write_offset+buffer_size])
Sep 17 13:37:57 dell-undercloud ironic-api: File "/usr/lib/python2.7/site-packages/eventlet/greenio/base.py", line 376, in sendall
Sep 17 13:37:57 dell-undercloud ironic-api: tail = self.send(data, flags)
Sep 17 13:37:57 dell-undercloud ironic-api: File "/usr/lib/python2.7/site-packages/eventlet/greenio/base.py", line 359, in send
Sep 17 13:37:57 dell-undercloud ironic-api: total_sent += fd.send(data[total_sent:], flags)
Sep 17 13:37:57 dell-undercloud ironic-api: error: [Errno 104] Connection reset by peer
Sep 17 13:37:57 dell-undercloud ironic-api: 192.0.2.1 - - [17/Sep/2015 13:37:57] "GET / HTTP/1.0" 500 59
Sep 17 13:37:57 dell-undercloud ironic-api: Traceback (most recent call last):
Sep 17 13:37:57 dell-undercloud ironic-api: File "/usr/lib64/python2.7/SocketServer.py", line 593, in process_request_thread
Sep 17 13:37:57 dell-undercloud ironic-api: self.finish_request(request, client_address)
Sep 17 13:37:57 dell-undercloud ironic-api: File "/usr/lib64/python2.7/SocketServer.py", line 334, in finish_request
Sep 17 13:37:57 dell-undercloud ironic-api: self.RequestHandlerClass(request, client_address, self)
Sep 17 13:37:57 dell-undercloud ironic-api: File "/usr/lib64/python2.7/SocketServer.py", line 651, in __init__
Sep 17 13:37:57 dell-undercloud ironic-api: self.finish()
Sep 17 13:37:57 dell-undercloud ironic-api: File "/usr/lib64/python2.7/SocketServer.py", line 710, in finish
Sep 17 13:37:57 dell-undercloud ironic-api: self.wfile.close()
Sep 17 13:37:57 dell-undercloud ironic-api: File "/usr/lib64/python2.7/socket.py", line 279, in close
Sep 17 13:37:57 dell-undercloud ironic-api: self.flush()
Sep 17 13:37:57 dell-undercloud ironic-api: File "/usr/lib64/python2.7/socket.py", line 303, in flush
Sep 17 13:37:57 dell-undercloud ironic-api: self._sock.sendall(view[write_offset:write_offset+buffer_size])
Sep 17 13:37:57 dell-undercloud ironic-api: File "/usr/lib/python2.7/site-packages/eventlet/greenio/base.py", line 376, in sendall
Sep 17 13:37:57 dell-undercloud ironic-api: tail = self.send(data, flags)
Sep 17 13:37:57 dell-undercloud ironic-api: File "/usr/lib/python2.7/site-packages/eventlet/greenio/base.py", line 359, in send
Sep 17 13:37:57 dell-undercloud ironic-api: total_sent += fd.send(data[total_sent:], flags)
Sep 17 13:37:57 dell-undercloud ironic-api: error: [Errno 32] Broken pipe
Sep 17 13:37:57 dell-undercloud proxy-server: Invalid user token - deferring reject downstream

Seeing this on two undercloud installation

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 3 bigswitch 2015-09-17 20:52:17 UTC
the message file is uploaded to 

https://bigswitch.box.com/s/e4lq3acakey8xq2za09irplgtg9noxs8

Comment 4 James Slagle 2015-09-22 16:26:02 UTC

*** This bug has been marked as a duplicate of bug 1246525 ***