Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 900764 (JBPAPP6-1393)

Summary: Core bridge doesn't failover in HornetQ live-backup setup
Product: [JBoss] JBoss Enterprise Application Platform 6 Reporter: Takayoshi Kimura <tkimura>
Component: HornetQAssignee: Francisco Borges <francisco.borges>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: high Docs Contact:
Priority: high    
Version: 6.0.0CC: anmiller, csuconic, francisco.borges, hgao, jbertram, mnovak, tkimura
Target Milestone: ER6   
Target Release: EAP 6.1.0   
Hardware: Unspecified   
OS: Unspecified   
URL: http://jira.jboss.org/jira/browse/JBPAPP6-1393
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
failover-core-bridge-eap6.zip none

Description Takayoshi Kimura 2012-08-03 06:52:40 UTC
Help Desk Ticket Reference: https://na7.salesforce.com/500A0000009elou
project_key: JBPAPP6

This is basically same issue reported in EAP 5.1.2: https://issues.jboss.org/browse/JBPAPP-8529

Core bridge doesn't failover in HornetQ live-backup setup.

When killed the live server, backup HornetQ comes up live but never deploy a bridge.

Configuration and DEBUG logs are attached.

Comment 1 Takayoshi Kimura 2012-08-03 06:53:45 UTC
Attachment: Added: failover-core-bridge-eap6.zip


Comment 2 Takayoshi Kimura 2012-08-03 06:55:00 UTC
Link: Added: This issue is related to JBPAPP-8529


Comment 3 Anne-Louise Tangring 2012-11-13 20:57:53 UTC
Docs QE Status: Removed: NEW 


Comment 4 Justin Bertram 2013-03-19 22:15:30 UTC
The problem looks like org.hornetq.core.server.cluster.impl.ClusterManagerImpl.start() is invoked before org.hornetq.core.server.impl.HornetQServerImpl.initialisePart2() is invoked.  When this happens the server tries to deploy the bridge before the queue is available and gives this kind of warning:

  WARN  [org.hornetq.core.server.cluster.impl.ClusterManagerImpl] (Activation for server HornetQServerImpl::serverUUID=6a5f0557-90c7-11e2-bfe4-e56356058f3d) No queue found with name jms.queue.source bridge will not be deployed.

See org.hornetq.core.server.impl.HornetQServerImpl.SharedStoreBackupActivation.run() for the problematic code.  

One thing to note here is that initialisePart2() actually invokes ClusterManagerImpl.start() so I doubt that SharedStoreBackupActivation.run() really needs to call ClusterManagerImpl.start().

What do you think, Clebert?

Comment 5 Yong Hao Gao 2013-03-27 05:09:15 UTC
I think Justin is right. However the ClusterManagerImpl.start() is needed because the backup need to start the cluster connections in order to announce its connectors to the cluster. It seems the bridges need not be deployed at this stage.

Comment 6 Francisco Borges 2013-03-27 13:50:48 UTC
Case is fixed, I just have to finish backporting its test case to 2.2 before sending PRs around.

Comment 8 Francisco Borges 2013-03-28 09:53:29 UTC
This issue was confirmed and fixed in all our 3 maintained branches, see commits

master:
https://github.com/hornetq/hornetq/commit/6acabc8cb361e27270abcd0a4659da65bf8e2166

AS7/EAP
https://github.com/hornetq/hornetq/commit/dd512ad646499a91ef0b37c15f27ba46a42baf28

Comment 9 Miroslav Novak 2013-05-05 18:03:21 UTC
I'm trying to verify this bugzilla but I'm not able to figure out the test scenario. Could you provide me more info, please?

Thanks,

Mirek

Comment 10 Yong Hao Gao 2013-05-06 03:01:58 UTC
Hi Mirek,

You can try deploy a core bridge to a hornetq node (let's called it nodeA) that has a backup (nodeB) node up and running. Make the core bridge to forward messages from nodeA to some other node (called nodeC).

Then kill nodeA and let the nodeB to be activated. Verify if the core-bridge can still work normally.

Without the fix the core-bridge won't be able to deployed and started correctly.

Comment 11 Miroslav Novak 2013-05-06 06:45:24 UTC
Thanks for the quick update. This scenario I successfully tried yesterday. I just need to confirm that configuration of HQ bridge must be also on backup's (not only live's) configuration.

Comment 12 Yong Hao Gao 2013-05-06 07:35:03 UTC
Yes the core-bridge configure needs to be in both live and backup configs.

Comment 13 Miroslav Novak 2013-05-06 11:27:14 UTC
For the purpose of verification of this bz I wrote new set of tests for testing HA with HQ core bridges and found a new issue - bz#959789 - but it's not this test scenario.

Use case described in this bz is ok. Verified in EAP 6.1.0.ER6. Thanks to all involved!