Bug 1296203 - one-off (BZ1290841) - Critical IO Error, shutting down the server
one-off (BZ1290841) - Critical IO Error, shutting down the server
Status: CLOSED DUPLICATE of bug 1292768
Product: JBoss Enterprise Application Platform 6
Classification: JBoss
Component: HornetQ (Show other bugs)
6.4.5
Unspecified Unspecified
unspecified Severity urgent
: ---
: ---
Assigned To: Clebert Suconic
Miroslav Novak
:
Depends On: 1290841
Blocks:
  Show dependency treegraph
 
Reported: 2016-01-06 10:23 EST by Miroslav Novak
Modified: 2016-01-06 10:29 EST (History)
4 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-01-06 10:29:27 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Miroslav Novak 2016-01-06 10:23:38 EST
There is unexpected server shutdown during in test scenario - "OOM in remote JCA topology in cluster with restart" 
1. Start EAP servers 1 and 3 in HornetQ cluster (jms cluster) with deployed queues InQueue and OutQueue
2. Start other 2 EAP servers 2 and 4 (mdb servers) which has configured resource adapter to connect to jms cluster
3. Send 50 000 small to InQueue to jms cluster
4. Deploy MDB to mdb servers
   - MDB consumes messages from InQueue and for each message send new message to OutQueue. MDB makes JNDI lookup for OutQueue for each message.
5. During processing of messages by MDB cause out of memory on heap on server 2
6. Wait 5 minutes and restart server 2
7. Wait some time until the rest (if there are some) of messages are consumed from InQueue
8. Receive messages from OutQueue

Test failed in step 6. when "Critical IO Error, shutting down the server." was logged on server-3 and server shuted down. There are set lower values for max-size-bytes (1MB) and max-size-bytes (100KB) to stress paging.

Part of the log with critical IO exception from server 3:
19:34:11,005 INFO  [org.hornetq.core.server] (hornetq-failure-check-thread) HQ221021: failed to remove connection
19:34:11,005 WARN  [org.hornetq.core.client] (hornetq-failure-check-thread) HQ212037: Connection failure has been detected: HQ119014: Did not receive data from /127.0.0.1:56802. It is likely the client has exited or crashed without closing its connection, or the network between the server and client has failed. You also might have configured connection-ttl and client-failure-check-period incorrectly. Please check user manual for more information. The connection will now be closed. [code=CONNECTION_TIMEDOUT]
19:34:11,036 WARN  [org.hornetq.core.server] (Old I/O server worker (parentId: -939609074, [id: 0xc7feb40e, /127.0.0.1:7445]), runningThread=Thread-123 (HornetQ-remoting-threads-HornetQServerImpl::serverUUID=8def949c-b406-11e5-a15b-7716caf3a087-252802219-248194785)) HQ222010: Critical IO Error, shutting down the server. file=NIOSequentialFile /qa/hudson_workspace/workspace/eap-64x-patched-hornetq-lodh-oom/server3/jboss-eap/standalone/data/messagingpaging/f1373aef-b406-11e5-b5e3-cbc9dcd21952/000000015.page, message=null: HornetQException[errorType=IO_ERROR message=null]
        at org.hornetq.core.journal.impl.NIOSequentialFile.open(NIOSequentialFile.java:111) [hornetq-journal-2.3.25.SP6-redhat-1-bz-1290841.jar:]
        at org.hornetq.core.journal.impl.NIOSequentialFile.open(NIOSequentialFile.java:96) [hornetq-journal-2.3.25.SP6-redhat-1-bz-1290841.jar:]
Comment 2 Miroslav Novak 2016-01-06 10:29:27 EST
Closing as this will be solved on bz#1292768 as this is regression of the fix provided there.

*** This bug has been marked as a duplicate of bug 1292768 ***

Note You need to log in before you can comment on or make changes to this bug.