Bug 959641 - NPE when node without destinations joins cluster which is used by remote HornetQ Resource Adapter
Summary: NPE when node without destinations joins cluster which is used by remote Horn...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: JBoss Enterprise Application Platform 6
Classification: JBoss
Component: HornetQ
Version: 6.1.0
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ER6
: EAP 6.2.0
Assignee: Andy Taylor
QA Contact:
URL:
Whiteboard:
: 914758 (view as bug list)
Depends On: 1016141
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-05-04 15:13 UTC by Miroslav Novak
Modified: 2013-12-15 16:15 UTC (History)
8 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2013-12-15 16:15:32 UTC
Type: Enhancement
Embargoed:


Attachments (Terms of Use)
reproducer.zip (45.40 KB, application/zip)
2013-05-31 09:58 UTC, Miroslav Novak
no flags Details
patch to stop partial init of MDB (2.32 KB, patch)
2013-09-02 12:57 UTC, Andy Taylor
no flags Details | Diff


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 912688 0 unspecified CLOSED Sometimes MDB throws NPE during receiving 2021-02-22 00:41:40 UTC

Internal Links: 912688

Description Miroslav Novak 2013-05-04 15:13:18 UTC
This enhancement is based on bz#912688.

If EAP 6.1.0 server has MDB which has configured HornetQ resource adapter connected to another server in hq cluster then if any node in the cluster doesn't have one queue/topic which MDB is using deployed, the MDB starts throw NPEs.

Comment 1 Clebert Suconic 2013-05-31 02:25:53 UTC
I don't understand how this could be. BZ912688 was happening because of the same issues you found on SpecJMS before the release where the destination was not being cleared.

At this point I have no clue how to work on this issue as I have no clue how to replicate it...


I will assign it back to you since I need more information.

Comment 2 Miroslav Novak 2013-05-31 09:56:49 UTC
Steps to Reproduce:
0. Download and unzip reproducer.zip from attachemnt. Next steps excexute in unzipped "reproducer" directory
1. First create an IP alias by "ifconfig eth1:0 192.168.40.1 up" under root user (change command according your environment)
2. run - "sh prepare.sh"
  - this downloads EAP 6.1.0.GA
  - creates 3 directories server1,2,3
  - copies directory jboss-eap-6.1 to server1,2,3
  - copies configuration standalone-full-ha-jms.xml to server1
  - copies configuration standalone-full-ha-mdb.xml to server2
  - copies mdb1.jar to server2's deployments directory
3. start first server by "sh start-server1.sh 192.168.40.1"
4. start thrird server by "sh start-server3.sh <third_ip>"
5. start second server by "sh start-server2.sh <second_ip>"
6. start jms producer by "sh start-producer.sh 192.168.40.1 1000"
7. see exceptions in server2 log

It's important to let server1 and server3 create a cluster before server3 is started otherwise exceptions won't occur.

Comment 3 Miroslav Novak 2013-05-31 09:58:11 UTC
Created attachment 755223 [details]
reproducer.zip

Comment 5 Miroslav Novak 2013-08-03 10:34:38 UTC
From feedback from dev (bz#912688) this is not an issue. I've set jboss-eap-6.1.1 flag not to loose focus from this bz.

Comment 6 Miroslav Novak 2013-08-26 11:04:53 UTC
My previous comment may look confusing without knowing context.

By default HornetQ RA load-balances connections to all nodes in cluster. When one node does not have specific destinations deployed then HQ RA starts to throw NPEs and breaks the delivery to MDB which is using those destinations. MDB does not receive any message.

Purpose of this BZ is to make HornetQ RA more robust when it connects to node in cluster which does not have deployed destinations needed by MDB so it can

Comment 7 Miroslav Novak 2013-08-26 11:25:42 UTC
... deliver messages from "well configured" nodes.

Comment 8 Andy Taylor 2013-08-27 10:05:00 UTC
nodes arent aware of the configuration of backup nodes and since the backup may not be available when the live startes there is not much we can do on the live. We could however a check for the queue on failover and if its not present stop the MDB and log a warning.

Comment 9 Miroslav Novak 2013-08-29 06:51:35 UTC
After irc chat with Andy, dev will discuss possible solutions for this.

Comment 10 Andy Taylor 2013-09-02 12:57:27 UTC
Created attachment 792818 [details]
patch to stop partial init of MDB

The issue is that if in HA and sessions are round robinned if the 1st succeeds and is started  the second may fail. At this point we stop the MSB but the first session is akready handling messages. This patch postpones the starting of the session until all sessions are created and then starts them all together. If there is a single failure we stop the whole MDB rather than partially init it.

Comment 11 Miroslav Novak 2013-09-16 11:52:30 UTC
*** Bug 914758 has been marked as a duplicate of this bug. ***

Comment 12 Clebert Suconic 2013-09-17 21:16:26 UTC
Andy: it seems that the patch makes sense. Can you do it?

Comment 13 Andy Taylor 2013-09-18 07:45:10 UTC
pr sent - https://github.com/hornetq/hornetq/pull/1287

Comment 17 Martin Svehla 2013-10-29 07:58:56 UTC
Tested with EAP 6.2.0.ER6 / HornetQ 2.3.9


Note You need to log in before you can comment on or make changes to this bug.