Bug 535604 (RHQ-2282)
| Summary: | Server log errors: Worker Thread Initialization Failure error, broken pipe, connection reset | ||
|---|---|---|---|
| Product: | [Other] RHQ Project | Reporter: | Jay Shaughnessy <jshaughn> |
| Component: | No Component | Assignee: | RHQ Project Maintainer <rhq-maint> |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | |
| Severity: | medium | Docs Contact: | |
| Priority: | medium | ||
| Version: | 1.2 | CC: | cwelton, rsoares |
| Target Milestone: | --- | Keywords: | SubBug |
| Target Release: | --- | ||
| Hardware: | All | ||
| OS: | All | ||
| URL: | http://jira.rhq-project.org/browse/RHQ-2282 | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: |
The same JDK version server/agent : java version "1.6.0_13" Oracle I think these errors are OS independent
|
|
| Last Closed: | 2014-05-09 15:28:36 UTC | Type: | --- |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | |||
| Bug Blocks: | 565628 | ||
There is a tangentially related forum thread: http://www.jboss.org/index.html?module=bb&op=viewtopic&p=4087180#4087180 Some similarity to messages seen here Try to turn off multicast detection. I saw something yesterday where other jboss remoting servers were getting detected and the jboss/remoting stuff under the covers seemed to try to communicate with those endpoints. To turn off multicast detection, these preferences in the agent must be set to false: rhq.agent.server-auto-detection rhq.communications.multicast-detector.enabled (BTW: The default for these is now false - I made that change yesterday so future builds will have these disabled out of box) Server-Side Multicast detection has also been set off by default in 2.3. It may be useful to turn off server side multicast in 2.2 to help avoid these errors. To do this: 1) shut down RHQ server 2) edit bin/rhq-server.properties set: rhq.communications.multicast-detector.enabled=false 3) restart server This bug was previously known as http://jira.rhq-project.org/browse/RHQ-2282 This bug is related to RHQ-1801 Temporarily adding the keyword "SubBug" so we can be sure we have accounted for all the bugs. keyword: new = Tracking + FutureFeature + SubBug making sure we're not missing any bugs in rhq_triage jay - could we turn detector off and close bug? I believe multicast is now off by default in more recent versions. So, I think this can be closed. |
Originating case: 313963 (has attachments displaying the symptomatic errors) Some environments suffer from the a series of potentially related Server log ERROR messages. The belief is that it may be tied to network instability. Although end user impact is minimal, the errors are disconcerting to the system manager, and perhaps there is some performance impact. Minimally, the errors should be better understood, and perhaps suppressed. There is also some indication that load and/or db locking may contribute. Large group manipulation may result in this situation. Here are examples: 2009-07-27 10:20:24,506 ERROR [org.jboss.remoting.transport.socket.ServerThread] Worker thread initialization failure java.io.EOFException at org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:526) at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:369) at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:165) 2009-07-27 10:20:24,507 DEBUG [org.jboss.remoting.transport.socket.ServerThread] WorkerThread#26[hidden:hidden] closing socketWrapper: ServerSocketWrapper[Socket[addr=/hidden,port=hidden,localport=hidden].28b5982d] 2009-07-27 10:20:24,507 DEBUG [org.jboss.remoting.transport.socket.ClientSocketWrapper] reset timeout: 60000 2009-07-27 10:20:24,507 ERROR [org.jboss.remoting.transport.socket.ServerThread] Worker thread initialization failure java.io.EOFException at org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:526) at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:369) at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:165) 2009-07-27 10:20:24,508 DEBUG [org.jboss.remoting.transport.socket.ServerThread] WorkerThread#182[hidden:hidden] closing socketWrapper: ServerSocketWrapper[Socket[addr=/hidden,port=hidden,localport=hidden].264cba93] 2009-07-27 10:20:24,508 DEBUG [org.jboss.remoting.transport.socket.ServerSocketWrapper] wrote CLOSING 2009-07-27 10:20:24,508 DEBUG [org.jboss.remoting.transport.socket.SocketWrapper] ServerSocketWrapper[Socket[addr=/hidden,port=hidden,localport=hidden].264cba93] closing 2009-07-27 10:20:24,508 DEBUG [org.jboss.remoting.transport.socket.ServerSocketWrapper] unable to writing CLOSING byte java.net.SocketException: Broken pipe at java.net.SocketOutputStream.socketWrite0(Native Method) at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92) at java.net.SocketOutputStream.write(SocketOutputStream.java:136) at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65) at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123) at java.io.ObjectOutputStream$BlockDataOutputStream.flush(ObjectOutputStream.java:1784) at java.io.ObjectOutputStream.flush(ObjectOutputStream.java:691) at org.jboss.remoting.transport.socket.ServerSocketWrapper.close(ServerSocketWrapper.java:57) at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:481) at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:165) 2009-07-27 10:20:24,509 DEBUG [org.jboss.remoting.transport.socket.SocketWrapper] unable to close output stream 2009-07-27 10:20:24,509 DEBUG [org.jboss.remoting.transport.socket.SocketWrapper] ServerSocketWrapper[Socket[addr=/hidden,port=hidden,localport=hidden].28b5982d] closing 2009-07-27 10:16:31,715 DEBUG [org.jboss.remoting.transport.socket.ClientSocketWrapper] reset timeout: 60000 2009-07-02 11:03:22,286 ERROR [org.jboss.remoting.transport.socket.ServerThread] Worker thread initialization failure java.net.SocketException: Connection reset at java.net.SocketInputStream.read(SocketInputStream.java:168) at java.io.BufferedInputStream.fill(BufferedInputStream.java:218) at java.io.BufferedInputStream.read(BufferedInputStream.java:237) at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2249) at java.io.ObjectInputStream$BlockDataInputStream.readBlockHeader(ObjectInputStream.java:2429) at java.io.ObjectInputStream$BlockDataInputStream.refill(ObjectInputStream.java:2499) at java.io.ObjectInputStream$BlockDataInputStream.read(ObjectInputStream.java:2571) at java.io.ObjectInputStream.read(ObjectInputStream.java:820) at org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:810) at org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:506) at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:369) at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:165) 2009-07-02 11:04:17,136 ERROR [org.jboss.remoting.transport.socket.ServerThread] Worker thread initialization failure java.net.SocketTimeoutException: Read timed out at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at java.io.BufferedInputStream.fill(BufferedInputStream.java:218) at java.io.BufferedInputStream.read(BufferedInputStream.java:237) at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2249) at java.io.ObjectInputStream$BlockDataInputStream.readBlockHeader(ObjectInputStream.java:2429) at java.io.ObjectInputStream$BlockDataInputStream.refill(ObjectInputStream.java:2499) at java.io.ObjectInputStream$BlockDataInputStream.read(ObjectInputStream.java:2571) at java.io.ObjectInputStream.read(ObjectInputStream.java:820) at org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:810) at org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:506) at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:369) at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:165)