Bug 1009409

Summary: [QA] (6.4.0) Clean shutdown of server with deployed JMS bridge throws org.hornetq.api.core.HornetQInterruptedException: java.lang.InterruptedException
Product: [JBoss] JBoss Enterprise Application Platform 6 Reporter: Miroslav Novak <mnovak>
Component: HornetQAssignee: Clebert Suconic <csuconic>
Status: CLOSED CURRENTRELEASE QA Contact: Miroslav Novak <mnovak>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 6.2.0CC: ataylor, csuconic, hgao, jawilson, jbertram, jdoyle, jmesnil, kkhan, msvehla
Target Milestone: DR1   
Target Release: EAP 6.4.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1132210 (view as bug list) Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1016141    
Bug Blocks: 1132168, 1132210    
Attachments:
Description Flags
configuration and trace logs from server with deployed bridge
none
threadump.txt (hanging source server) none

Description Miroslav Novak 2013-09-18 11:36:33 UTC
Description of problem:
When EAP 6.2.0.ER1 server with deployed JMS bridge is shutdowned (ctrl-c)  then following error is logged:

13:28:44,049 ERROR [stderr] (pool-4-thread-1) Exception in thread "pool-4-thread-1" org.hornetq.api.core.HornetQInterruptedException: java.lang.InterruptedException
13:28:44,054 ERROR [stderr] (pool-4-thread-1) Exception in thread "pool-4-thread-2" 	at org.hornetq.jms.bridge.impl.JMSBridgeImpl$BatchTimeChecker.run(JMSBridgeImpl.java:2162)
13:28:44,055 ERROR [stderr] (pool-4-thread-1) 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
13:28:44,055 ERROR [stderr] (pool-4-thread-1) 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
13:28:44,055 ERROR [stderr] (pool-4-thread-1) 	at java.lang.Thread.run(Thread.java:662)
13:28:44,055 ERROR [stderr] (pool-4-thread-1) Caused by: java.lang.InterruptedException
13:28:44,055 ERROR [stderr] (pool-4-thread-1) 	at java.lang.Object.wait(Native Method)
13:28:44,055 ERROR [stderr] (pool-4-thread-1) 	at org.hornetq.jms.bridge.impl.JMSBridgeImpl$BatchTimeChecker.run(JMSBridgeImpl.java:2149)
13:28:44,055 ERROR [stderr] (pool-4-thread-1) 	... 3 more
13:28:44,056 ERROR [stderr] (pool-4-thread-2) org.hornetq.api.core.HornetQInterruptedException: java.lang.InterruptedException
13:28:44,056 ERROR [stderr] (pool-4-thread-2) 	at org.hornetq.jms.bridge.impl.JMSBridgeImpl$SourceReceiver.run(JMSBridgeImpl.java:1946)
13:28:44,056 ERROR [stderr] (pool-4-thread-2) 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
13:28:44,056 ERROR [stderr] (pool-4-thread-2) 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
13:28:44,056 ERROR [stderr] (pool-4-thread-2) 	at java.lang.Thread.run(Thread.java:662)
13:28:44,056 ERROR [stderr] (pool-4-thread-2) Caused by: java.lang.InterruptedException
13:28:44,056 ERROR [stderr] (pool-4-thread-2) 	at java.lang.Object.wait(Native Method)
13:28:44,057 ERROR [stderr] (pool-4-thread-2) 	at org.hornetq.jms.bridge.impl.JMSBridgeImpl$SourceReceiver.run(JMSBridgeImpl.java:1938)
13:28:44,057 ERROR [stderr] (pool-4-thread-2) 	... 3 more


Attached configuration of servers + server.log (with TRACE logs from server with deployed bridge)

Comment 1 Miroslav Novak 2013-09-18 11:37:27 UTC
Created attachment 799341 [details]
configuration and trace logs from server with deployed bridge

Comment 2 Clebert Suconic 2013-09-18 15:27:54 UTC
https://github.com/hornetq/hornetq/pull/1291

This is ignorable.. I'm not going to cut a release just because of this, so this will be on EAP 6.next (either 6.3, 6.2.1.. etc)


If something else more critical appears we will include it on the cut.

Comment 3 John Doyle 2013-09-18 15:38:27 UTC
Clebert, this is flagged for 6.2.0. If you're going to push it to a later release you should put a devel_ack-, not a +.

Comment 4 Clebert Suconic 2013-09-18 15:45:16 UTC
I marked it for 6.3.0...

If I ever do another cut off for 6.2 (say another issue is found, then it will be sent on 6.2)... I have no plans to make any more cuts on 6.2 unless an at least Urgent issue is found.


Anyway, this seems related to https://bugzilla.redhat.com/show_bug.cgi?id=1003913, which would probably fix this anyways. I'm adding a further minor change on the component just in case.

Comment 5 Clebert Suconic 2013-09-18 15:53:13 UTC
I will mark this with NACK then.. which I may change it if we do another component cut.

Comment 6 Miroslav Novak 2013-10-22 13:27:23 UTC
I can see that PR is in HQ 2.3.9 (in EAP 6.2.0.ER6) and there is still problem the same exception:

15:06:57,089 ERROR [stderr] (pool-4-thread-2) Exception in thread "pool-4-thread-2" Exception in thread "pool-4-thread-1" org.hornetq.api.core.HornetQInterruptedException: java.lang.InterruptedException
15:06:57,089 ERROR [stderr] (pool-4-thread-2) 	at org.hornetq.jms.bridge.impl.JMSBridgeImpl$SourceReceiver.run(JMSBridgeImpl.java:1946)
15:06:57,090 ERROR [stderr] (pool-4-thread-2) 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
15:06:57,090 ERROR [stderr] (pool-4-thread-2) 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
15:06:57,090 ERROR [stderr] (pool-4-thread-2) 	at java.lang.Thread.run(Thread.java:662)
15:06:57,091 ERROR [stderr] (pool-4-thread-2) Caused by: java.lang.InterruptedException
15:06:57,091 WARN  [org.hornetq.jms.server] (pool-4-thread-3) HQ122004: JMS Bridge Will retry after a pause of 1,000 ms
15:06:57,091 ERROR [stderr] (pool-4-thread-2) 	at java.lang.Object.wait(Native Method)
15:06:57,091 ERROR [stderr] (pool-4-thread-2) 	at org.hornetq.jms.bridge.impl.JMSBridgeImpl$SourceReceiver.run(JMSBridgeImpl.java:1938)
15:06:57,091 ERROR [stderr] (pool-4-thread-2) 	... 3 more
15:06:57,091 ERROR [stderr] (pool-4-thread-1) org.hornetq.api.core.HornetQInterruptedException: java.lang.InterruptedException
15:06:57,092 ERROR [stderr] (pool-4-thread-1) 	at org.hornetq.jms.bridge.impl.JMSBridgeImpl$BatchTimeChecker.run(JMSBridgeImpl.java:2162)
15:06:57,092 ERROR [stderr] (pool-4-thread-1) 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
15:06:57,092 ERROR [stderr] (pool-4-thread-1) 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
15:06:57,092 ERROR [stderr] (pool-4-thread-1) 	at java.lang.Thread.run(Thread.java:662)
15:06:57,092 ERROR [stderr] (pool-4-thread-1) Caused by: java.lang.InterruptedException
15:06:57,092 ERROR [stderr] (pool-4-thread-1) 	at java.lang.Object.wait(Native Method)
15:06:57,093 ERROR [stderr] (pool-4-thread-1) 	at org.hornetq.jms.bridge.impl.JMSBridgeImpl$BatchTimeChecker.run(JMSBridgeImpl.java:2149)
15:06:57,093 ERROR [stderr] (pool-4-thread-1) 	... 3 more

Comment 7 Miroslav Novak 2014-07-07 08:02:52 UTC
This issue is valid for EAP 6.3.0.ER9. Setting jboss-eap-6.4 ? flag and target version as this should be fixed to EAP 6.4 release.

Comment 8 Miroslav Novak 2014-07-07 08:18:26 UTC
Created attachment 915995 [details]
threadump.txt (hanging source server)

Comment 9 Miroslav Novak 2014-07-07 08:19:32 UTC
Just noticed that "source" server for JMS bridge cannot be cleanly shutdowned when source server was shutdowned first. Attaching thread dump from "source server".

Comment 10 Andy Taylor 2014-07-21 13:59:50 UTC
im looking at this too

Comment 11 Andy Taylor 2014-07-22 12:48:10 UTC
fixed in https://github.com/hornetq/hornetq/pull/1720

Comment 12 Justin Bertram 2014-07-22 20:18:24 UTC
To be clear this was fixed on the 2.3.x branch of HornetQ via commit e93149fb6e797ea6d300aaf9603d51499452c269.

Comment 13 Kabir Khan 2014-08-24 10:59:57 UTC
Appears to be fixed by HQ upgrade to 2.3.21 https://bugzilla.redhat.com/show_bug.cgi?id=1132168. Setting to MODIFIED.

Comment 14 Miroslav Novak 2014-09-17 15:20:39 UTC
Hi Andy, Justin,

no exceptions are thrown when the bridge is idle and server with deployed JMS bridge is shutdowned. 

If the bridge is not idle then there are exceptions from arjuna and so on. We should allow all transactions to finish gracefully before clean shutdown. Do you want me to create new bz for it or just reopen this?

Thanks,
Mirek

Comment 15 Andy Taylor 2014-09-17 16:08:50 UTC
thats a different issue and should be raised against the transactions project

Comment 16 Miroslav Novak 2014-09-18 05:09:58 UTC
Setting as verified in EAP 6.4.0.DR1. Thanks for the fix!