Bug 1806709

Summary: [GSS](6.4.z) Make HornetQ UDP Clustering Client discovery resilient
Product: [JBoss] JBoss Enterprise Application Platform 6 Reporter: Brad Maxwell <bmaxwell>
Component: HornetQAssignee: Romain Pelisse <rpelisse>
Status: CLOSED UPSTREAM QA Contact: Peter Mackay <pmackay>
Severity: high Docs Contact:
Priority: high    
Version: unspecifiedCC: csuconic, msvehla, rpelisse
Target Milestone: ---   
Target Release: EAP 6.4.23   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2025-02-10 04:00:06 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1699388, 1700855, 1806052    

Description Brad Maxwell 2020-02-24 19:36:44 UTC

Comment 4 Brad Maxwell 2020-02-29 02:10:30 UTC
An EAP server setup as a HornetQ client using Discovery to locate the HornetQ JMS Brokers via UDP such as in the configuration below.  Can get into a state when there is an unknown issue occur such as a network blip, JVM Bug, OS bug where in it times out with the error below but is unable to recover and even when the network messages are received and will continue to fail with the error below.  The HornetQ UDP Clustering Client Discovery needs to be more resilient so that it can recover and reconnect.

            <subsystem xmlns="urn:jboss:domain:messaging:1.4">
                <hornetq-server>
                    <persistence-enabled>true</persistence-enabled>
                    <message-expiry-scan-period>300000</message-expiry-scan-period>
                    <journal-type>NIO</journal-type>
                    <journal-min-files>2</journal-min-files>

                    <discovery-groups>
                        <discovery-group name="dg-group1">
                            <socket-binding>messaging-group</socket-binding>
                            <refresh-timeout>10000</refresh-timeout>
                            <initial-wait-timeout>20000</initial-wait-timeout>
                        </discovery-group>
                    </discovery-groups>

                    <jms-connection-factories>
                        <pooled-connection-factory name="hornetq-ra">
                            <user>jmsuser</user>
                            <password>redhat1!</password>
                            <min-pool-size>10</min-pool-size>
                            <max-pool-size>30</max-pool-size>
                            <discovery-group-ref discovery-group-name="dg-group1"/>
                            <entries>
                                <entry name="java:/JmsXA"/>
                            </entries>
                            <retry-interval>4000</retry-interval>
                            <max-retry-interval>12000</max-retry-interval>
                            <reconnect-attempts>15</reconnect-attempts>
                            <connection-load-balancing-policy-class-name>org.hornetq.api.core.client.loadbalance.RoundRobinConnectionLoadBalancingPolicy</connection-load-balancing-policy-class-name>
                        </pooled-connection-factory>
                    </jms-connection-factories>
                </hornetq-server>
            </subsystem>

        <socket-binding-group name="full-sockets" default-interface="public">
            ...
            <socket-binding name="messaging" port="5445"/>
            <socket-binding name="messaging-group" port="0" multicast-address="${jboss.messaging.group.address:231.7.7.7}" multicast-port="${jboss.messaging.group.port:9876}"/>
            <socket-binding name="messaging-throughput" port="5455"/>
        </socket-binding-group>


[Server:client] 19:50:08,293 ERROR [org.hornetq.ra] (JMSTest-Runner-0) HQ154002: Could not create session: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/JmsXA
[Server:client] 	at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:410)
[Server:client] 	at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:367)
[Server:client] 	at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:499)
[Server:client] 	at org.hornetq.ra.HornetQRASessionFactoryImpl.allocateConnection(HornetQRASessionFactoryImpl.java:832)
[Server:client] 	at org.hornetq.ra.HornetQRASessionFactoryImpl.createSession(HornetQRASessionFactoryImpl.java:465)
[Server:client] 	at test.JmsConnectionFactoryRestService$Runner.run(JmsConnectionFactoryRestService.java:164) [classes:]
[Server:client] 	at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_202]
[Server:client] Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null
[Server:client] 	at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:457)
[Server:client] 	at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:479)
[Server:client] 	at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:451)
[Server:client] 	at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:344)
[Server:client] 	... 6 more
[Server:client] Caused by: javax.resource.ResourceException: Error during setup
[Server:client] 	at org.hornetq.ra.HornetQRAManagedConnection.<init>(HornetQRAManagedConnection.java:171)
[Server:client] 	at org.hornetq.ra.HornetQRAManagedConnectionFactory.createManagedConnection(HornetQRAManagedConnectionFactory.java:157)
[Server:client] 	at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:874)
[Server:client] 	at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:416)
[Server:client] 	... 9 more
[Server:client] Caused by: javax.resource.ResourceException: Failed to create session factory
[Server:client] 	at org.hornetq.ra.HornetQRAManagedConnection.setup(HornetQRAManagedConnection.java:887)
[Server:client] 	at org.hornetq.ra.HornetQRAManagedConnection.<init>(HornetQRAManagedConnection.java:160)
[Server:client] 	... 12 more
[Server:client] Caused by: javax.jms.JMSException: Failed to create session factory
[Server:client] 	at org.hornetq.jms.client.HornetQConnectionFactory.createConnectionInternal(HornetQConnectionFactory.java:587)
[Server:client] 	at org.hornetq.jms.client.HornetQConnectionFactory.createXAConnection(HornetQConnectionFactory.java:143)
[Server:client] 	at org.hornetq.ra.HornetQRAManagedConnection.setup(HornetQRAManagedConnection.java:871)
[Server:client] 	... 13 more
[Server:client] Caused by: HornetQException[errorType=CONNECTION_TIMEDOUT message=HQ119012: Timed out waiting to receive initial broadcast from cluster]
[Server:client] 	at org.hornetq.core.client.impl.ServerLocatorImpl.createSessionFactory(ServerLocatorImpl.java:859)
[Server:client] 	at org.hornetq.jms.client.HornetQConnectionFactory.createConnectionInternal(HornetQConnectionFactory.java:583)
[Server:client] 	... 15 more

Comment 8 Peter Mackay 2020-06-30 11:57:21 UTC
Verified with EAP 6.4.23.CP.CR2

Comment 9 Red Hat Bugzilla 2025-02-10 04:00:06 UTC
This product has been discontinued or is no longer tracked in Red Hat Bugzilla.