Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 899766 (JBPAPP6-35)

Summary: CLONE - Classloading issue with HornetQLoggerFactory - problem with failback
Product: [JBoss] JBoss Enterprise Application Platform 6 Reporter: Miroslav Novak <mnovak>
Component: HornetQAssignee: Andy Taylor <ataylor>
Status: CLOSED NEXTRELEASE QA Contact:
Severity: urgent Docs Contact:
Priority: urgent    
Version: 6.0.0CC: atangrin
Target Milestone: ---   
Target Release: EAP 6.0.0   
Hardware: Unspecified   
OS: Unspecified   
URL: http://jira.jboss.org/jira/browse/JBPAPP6-35
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-11-04 23:43:12 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
console-log-backup-server.txt
none
console-log-live-server.txt
none
reproducer.zip none

Description Miroslav Novak 2012-01-23 11:42:57 UTC
project_key: JBPAPP6

Test scenario:
1. Start two servers AS7/EAP6 servers - live and its backup in dedicated topology - each on different machine
2. Kill live server using "kill -9 ..."
3. Start live server again

In step 3. there are unexpected messages in console log of live and backup server.

From backup server:
{code}
12:07:26,165 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (Thread-78) HornetQ Server version 2.2.7.Final (HQ_2_2_7_FINAL_AS7, 121) [17700d86-45b2-11e1-a575-d48564b8e1e7] stopped
12:07:26,165 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (Thread-78) unable to restart server, please kill and restart manually: java.lang.IllegalArgumentException: Could not find class org.jboss.as.messaging.HornetQLoggerFactory
	at org.hornetq.utils.ClassloadingUtil$1.run(ClassloadingUtil.java:42) [hornetq-core-2.2.7.Final.jar:]
	at java.security.AccessController.doPrivileged(Native Method) [:1.6.0_22]
	at org.hornetq.utils.ClassloadingUtil.safeInitNewInstance(ClassloadingUtil.java:16) [hornetq-core-2.2.7.Final.jar:]
	at org.hornetq.core.server.impl.HornetQServerImpl.instantiateInstance(HornetQServerImpl.java:1868) [hornetq-core-2.2.7.Final.jar:]
	at org.hornetq.core.server.impl.HornetQServerImpl.initialiseLogging(HornetQServerImpl.java:1301) [hornetq-core-2.2.7.Final.jar:]
	at org.hornetq.core.server.impl.HornetQServerImpl.start(HornetQServerImpl.java:541) [hornetq-core-2.2.7.Final.jar:]
	at org.hornetq.core.server.impl.HornetQServerImpl$SharedStoreBackupActivation$1FailbackChecker$1.run(HornetQServerImpl.java:430) [hornetq-core-2.2.7.Final.jar:]
	at java.lang.Thread.run(Thread.java:679) [:1.6.0_22]
{code}

From live server:
{code}
12:07:51,993 INFO  [org.jboss.as.messaging] (MSC service thread 1-3) JBAS011601: Bound messaging object to jndi name java:/topic/test
12:07:52,007 INFO  [org.jboss.as] (Controller Boot Thread) JBoss EAP 6.0.0.Alpha2 (AS 7.1.0.CR1-redhat-1) started in 31092ms - Started 155 of 263 services (103 services are passive or on-demand)
12:07:53,292 DEBUG [org.hornetq.core.client.impl.ClientSessionFactoryImpl] (Thread-1 (group:HornetQ-client-global-threads-1954829789)) Trying reconnection attempt 1
12:07:53,292 DEBUG [org.hornetq.core.remoting.impl.netty.NettyConnector] (Thread-1 (group:HornetQ-client-global-threads-1954829789)) Started Netty Connector version 3.2.3.Final-r${buildNumber}
12:07:53,292 DEBUG [org.hornetq.core.client.impl.ClientSessionFactoryImpl] (Thread-1 (group:HornetQ-client-global-threads-1954829789)) Trying to connect at the main server using connector :org-hornetq-core-remoting-impl-netty-NettyConnectorFactory?port=5446&host=192-168-10-4
12:07:53,293 DEBUG [org.hornetq.core.client.impl.ClientSessionFactoryImpl] (Thread-1 (group:HornetQ-client-global-threads-1954829789)) Main server is not up. Hopefully there's a backup configured now!
12:07:55,293 DEBUG [org.hornetq.core.client.impl.ClientSessionFactoryImpl] (Thread-1 (group:HornetQ-client-global-threads-1954829789)) Trying reconnection attempt 2
12:07:55,293 DEBUG [org.hornetq.core.remoting.impl.netty.NettyConnector] (Thread-1 (group:HornetQ-client-global-threads-1954829789)) Started Netty Connector version 3.2.3.Final-r${buildNumber}
12:07:55,293 DEBUG [org.hornetq.core.client.impl.ClientSessionFactoryImpl] (Thread-1 (group:HornetQ-client-global-threads-1954829789)) Trying to connect at the main server using connector :org-hornetq-core-remoting-impl-netty-NettyConnectorFactory?port=5446&host=192-168-10-4
12:07:55,294 DEBUG [org.hornetq.core.client.impl.ClientSessionFactoryImpl] (Thread-1 (group:HornetQ-client-global-threads-1954829789)) Main server is not up. Hopefully there's a backup
{code}

I'm not fully sure but it looks like that backup server did not manage to get to "waiting for live to fail" state and live server is not able to detect backup.

There is set "DEBUG" level logging for "org.hornetq" in attached logs.

I've attached reproducer.zip - steps to use:
1. Download and unzip "reproducer.zip"
2. Prepare live and backup server - "sh prepare.sh"
3. Start live - "sh start-server1.sh server1_hostname"
4. Start backup - "sh start-server2.sh server2_hostname"
5. Kill live server using "kill -9 server1_process_id"
6. Start live server again - "sh start-server1.sh server1_hostname"

Note:
In reproducer.zip are configuration files standalone-ha-A.xml, standalone-ha-B.xml (A for live, B for backup).

Comment 1 Miroslav Novak 2012-01-23 11:42:58 UTC
Link: Added: This issue Cloned from AS7-3408


Comment 2 Miroslav Novak 2012-01-23 11:43:49 UTC
Workflow: Removed: GIT Pull Request workflow  Added: jira
Release Notes Docs Status: Added: Not Required
Security: Added: Public


Comment 3 Miroslav Novak 2012-01-25 10:07:55 UTC
Setting as resolved since AS7 jira is resolved.

Comment 4 Miroslav Novak 2012-03-02 08:57:58 UTC
Issue fixed. Closing.

Comment 5 Anne-Louise Tangring 2012-11-04 14:32:43 UTC
Release Notes Docs Status: Removed: Not Required 


Comment 6 Anne-Louise Tangring 2012-11-04 23:42:35 UTC
Fixing Affects and Fix in Version