Red Hat Bugzilla – Bug 1296203
one-off (BZ1290841) - Critical IO Error, shutting down the server
Last modified: 2016-01-06 10:29:27 EST
There is unexpected server shutdown during in test scenario - "OOM in remote JCA topology in cluster with restart"
1. Start EAP servers 1 and 3 in HornetQ cluster (jms cluster) with deployed queues InQueue and OutQueue
2. Start other 2 EAP servers 2 and 4 (mdb servers) which has configured resource adapter to connect to jms cluster
3. Send 50 000 small to InQueue to jms cluster
4. Deploy MDB to mdb servers
- MDB consumes messages from InQueue and for each message send new message to OutQueue. MDB makes JNDI lookup for OutQueue for each message.
5. During processing of messages by MDB cause out of memory on heap on server 2
6. Wait 5 minutes and restart server 2
7. Wait some time until the rest (if there are some) of messages are consumed from InQueue
8. Receive messages from OutQueue
Test failed in step 6. when "Critical IO Error, shutting down the server." was logged on server-3 and server shuted down. There are set lower values for max-size-bytes (1MB) and max-size-bytes (100KB) to stress paging.
Part of the log with critical IO exception from server 3:
19:34:11,005 INFO [org.hornetq.core.server] (hornetq-failure-check-thread) HQ221021: failed to remove connection
19:34:11,005 WARN [org.hornetq.core.client] (hornetq-failure-check-thread) HQ212037: Connection failure has been detected: HQ119014: Did not receive data from /127.0.0.1:56802. It is likely the client has exited or crashed without closing its connection, or the network between the server and client has failed. You also might have configured connection-ttl and client-failure-check-period incorrectly. Please check user manual for more information. The connection will now be closed. [code=CONNECTION_TIMEDOUT]
19:34:11,036 WARN [org.hornetq.core.server] (Old I/O server worker (parentId: -939609074, [id: 0xc7feb40e, /127.0.0.1:7445]), runningThread=Thread-123 (HornetQ-remoting-threads-HornetQServerImpl::serverUUID=8def949c-b406-11e5-a15b-7716caf3a087-252802219-248194785)) HQ222010: Critical IO Error, shutting down the server. file=NIOSequentialFile /qa/hudson_workspace/workspace/eap-64x-patched-hornetq-lodh-oom/server3/jboss-eap/standalone/data/messagingpaging/f1373aef-b406-11e5-b5e3-cbc9dcd21952/000000015.page, message=null: HornetQException[errorType=IO_ERROR message=null]
at org.hornetq.core.journal.impl.NIOSequentialFile.open(NIOSequentialFile.java:111) [hornetq-journal-2.3.25.SP6-redhat-1-bz-1290841.jar:]
at org.hornetq.core.journal.impl.NIOSequentialFile.open(NIOSequentialFile.java:96) [hornetq-journal-2.3.25.SP6-redhat-1-bz-1290841.jar:]
Closing as this will be solved on bz#1292768 as this is regression of the fix provided there.
*** This bug has been marked as a duplicate of bug 1292768 ***