Description of problem: Application EAR deployment fails on a collocated live-backup configuration during server fallback. Version-Release number of selected component (if applicable): JBoss-EAP-6.4.6 Steps to Reproduce: 1. Configure collocated-live backup pair using two physical server instances, NodeA and NodeB 2. Configure a destination queue/A at each node. 3. Create an EAR containing an MDB, which listens to the queue/A 4. Deploy the EAR at both Nodes (NodeA and NodeB) 5. Start NodeA then NodeB 6. Shutdown NodeA 7. Restart NodeA 8. The EAR deployment failed to deploy in NodeA Actual results: NodeA failed deploy the EAR Expected results: NodeA should be able to deploy the EAR
Created attachment 1156188 [details] Configuration from Tryonne
Created attachment 1156189 [details] Configuration from Tryonne
Created attachment 1161461 [details] ear the ear
Note to make things a bit easier: 1) to reproduce at nodeA, so you actually only need to deploy the ear to nodeA. 2) normally you need 2 machines. 3) When restart, make sure do it after nodeB failover done. Howard
*The cause of this issue: when live-server service starts (by jms service), it turns into backup mode first because its backup now is in live mode. So when this service is started, the live-server.activation service doesn't get installed and started. As a result, the pooled-connection-factory.hornetq-ra service who depends on live-server.activation won't get started. The pooled-connection-factory service is responsible for creating the jboss.raactivator service. Therefore jboss.raactivator service won't get installed. The jboss.raactivator service is the one that creates the jboss.ra.hornetq-ra resource adapter service. So the jboss.ra.hornetq-ra service won't get installed. When MDB starts to deploy, it will wait on a StabilityMonitor's counter to make sure all of its dependencies are in ready (fully started) states before it goes on to deploy. Unfortunately when all other dependencies are ready, one of its dependency (jboss.ra.hornetq-ra) didn't get installed and started and therefore it has no way to prevent the MDB to go on deploying. MDB then finds that it has a missing dependency (jboss.ra.horentq-ra) and fails the deployment. *How to fix: The key to fix this issue is to let the MDB deployment wait until its dependency jboss.ra.hornetq-ra is ready (installed and started). Based on current hierarchy, we can make live-server.activation start as part of jms service start process, and we make it adds to the StabilityMonitor's counter so that until the live-server.activation startup is completed the MDB keeps waiting. When live-server failback and becomes live again, it calls the registered callback where the live-server.activation service gets notified and finish start. By the time live-server.activation is fully started, the container makes sure its dependents are already kicked off, and that continues keeping the MDB waiting, until the jboss.ra.hornetq-ra is installed and started. That way when MDB stops waiting, everything is ready and the MDB gets deployed successfully.
Here is the PR link https://github.com/jbossas/jboss-eap/pull/2778
Verified with EAP 6.4.15.CP.CR3
Released on May 18 as part of EAP 6.4.15.