Bug 1007997

Summary: Intermittent failure in DomainControllerMigrationTestCase
Product: [JBoss] JBoss Enterprise Application Platform 6 Reporter: Brian Stansberry <brian.stansberry>
Component: Domain ManagementAssignee: Emanuel Muckenhuber <emuckenh>
Status: CLOSED CURRENTRELEASE QA Contact: Pavel Jelinek <pjelinek>
Severity: medium Docs Contact: Russell Dickenson <rdickens>
Priority: unspecified    
Version: 6.2.0CC: dandread, emuckenh, lthon, pkremens
Target Milestone: ER7   
Target Release: EAP 6.2.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-12-15 16:21:51 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Brian Stansberry 2013-09-13 17:49:36 UTC
Test fails against 6.x on occasion due to connection closed issues:

java.lang.RuntimeException: "JBAS010839: Operation failed or was rolled back on all servers."
    at org.jboss.as.test.integration.domain.management.util.DomainLifecycleUtil.executeForResult(DomainLifecycleUtil.java:582)
    at org.jboss.as.test.integration.domain.DomainControllerMigrationTestCase.testDCFailover(DomainControllerMigrationTestCase.java:206)

------- Stdout: -------
C:\BuildAgent\work\ab7853673ba2ff34\testsuite\domain\target\test-classes\domain-configs\domain-standard.xml
C:\BuildAgent\work\ab7853673ba2ff34\testsuite\domain\target\test-classes\host-configs\host-failover1.xml
13:04:16,430 INFO  [org.jboss.modules] (main) JBoss Modules version 1.3.0.Beta4
13:04:16,687 INFO  [org.jboss.as.process.Host Controller.status] (main) JBAS012017: Starting process 'Host Controller'
[Host Controller] 13:04:17,520 INFO  [org.jboss.modules] (main) JBoss Modules version 1.3.0.Beta4
[Host Controller] 13:04:17,695 INFO  [org.jboss.msc] (main) JBoss MSC version 1.0.4.GA-redhat-1
[Host Controller] 13:04:17,748 INFO  [org.jboss.as] (MSC service thread 1-1) JBAS015899: JBoss EAP 6.2.0.Alpha1 (AS 7.3.0.Final-redhat-SNAPSHOT) starting
[Host Controller] 13:04:18,663 INFO  [org.xnio] (MSC service thread 1-3) XNIO Version 3.0.7.GA-redhat-1
[Host Controller] 13:04:18,663 INFO  [org.xnio.nio] (MSC service thread 1-3) XNIO NIO Implementation Version 3.0.7.GA-redhat-1
[Host Controller] 13:04:18,663 INFO  [org.jboss.as] (Controller Boot Thread) JBAS010902: Creating http management service using network interface (management) port (9990) securePort (-1)
[Host Controller] 13:04:18,679 INFO  [org.jboss.remoting] (MSC service thread 1-3) JBoss Remoting version 3.2.16.GA-redhat-1
[Host Controller] 13:04:19,030 INFO  [org.jboss.as.remoting] (MSC service thread 1-4) JBAS017100: Listening on 127.0.0.1:9999
[Host Controller] 13:04:19,801 WARN  [org.jboss.as.threads] (Controller Boot Thread) JBAS012400: The 'per-cpu' attribute is no longer supported. The value [10.000000] of the 'count' attribute is being combined with the value [20.000000] of the 'per-cpu' attribute and the current processor count [2] to derive a new value of [50] for 'count'.
[Host Controller] 13:04:20,554 WARN  [org.jboss.as.jmx] (Controller Boot Thread) JBAS011305: <jmx-connector/> is no longer supporting. <remoting-connector/> should be used instead to allow remote connections via JBoss Remoting.
[Host Controller] 13:04:20,624 WARN  [org.jboss.as.threads] (Controller Boot Thread) JBAS012400: The 'per-cpu' attribute is no longer supported. The value [10.000000] of the 'count' attribute is being combined with the value [20.000000] of the 'per-cpu' attribute and the current processor count [2] to derive a new value of [50] for 'count'.
[Host Controller] 13:04:20,633 WARN  [org.jboss.as.jmx] (Controller Boot Thread) JBAS011305: <jmx-connector/> is no longer supporting. <remoting-connector/> should be used instead to allow remote connections via JBoss Remoting.
[Host Controller] 13:04:20,634 WARN  [org.jboss.as.threads] (Controller Boot Thread) JBAS012400: The 'per-cpu' attribute is no longer supported. The value [10.000000] of the 'count' attribute is being combined with the value [20.000000] of the 'per-cpu' attribute and the current processor count [2] to derive a new value of [50] for 'count'.
[Host Controller] 13:04:20,635 WARN  [org.jboss.as.threads] (Controller Boot Thread) JBAS012400: The 'per-cpu' attribute is no longer supported. The value [10.000000] of the 'count' attribute is being combined with the value [20.000000] of the 'per-cpu' attribute and the current processor count [2] to derive a new value of [50] for 'count'.
[Host Controller] 13:04:20,636 WARN  [org.jboss.as.threads] (Controller Boot Thread) JBAS012400: The 'per-cpu' attribute is no longer supported. The value [10.000000] of the 'count' attribute is being combined with the value [20.000000] of the 'per-cpu' attribute and the current processor count [2] to derive a new value of [50] for 'count'.
[Host Controller] 13:04:20,637 WARN  [org.jboss.as.threads] (Controller Boot Thread) JBAS012400: The 'per-cpu' attribute is no longer supported. The value [10.000000] of the 'count' attribute is being combined with the value [20.000000] of the 'per-cpu' attribute and the current processor count [2] to derive a new value of [50] for 'count'.
[Host Controller] 13:04:20,637 WARN  [org.jboss.as.threads] (Controller Boot Thread) JBAS012400: The 'per-cpu' attribute is no longer supported. The value [10.000000] of the 'count' attribute is being combined with the value [20.000000] of the 'per-cpu' attribute and the current processor count [2] to derive a new value of [50] for 'count'.
[Host Controller] 13:04:20,638 WARN  [org.jboss.as.threads] (Controller Boot Thread) JBAS012400: The 'per-cpu' attribute is no longer supported. The value [10.000000] of the 'count' attribute is being combined with the value [20.000000] of the 'per-cpu' attribute and the current processor count [2] to derive a new value of [50] for 'count'.
[Host Controller] 13:04:21,496 INFO  [org.jboss.as.host.controller] (Controller Boot Thread) JBAS010922: Starting server failover-one
13:04:21,512 INFO  [org.jboss.as.process.Server:failover-one.status] (ProcessController-threads - 3) JBAS012017: Starting process 'Server:failover-one'
[Server:failover-one] 13:04:22,364 INFO  [org.jboss.modules] (main) JBoss Modules version 1.3.0.Beta4
[Server:failover-one] 13:04:22,856 INFO  [org.jboss.msc] (main) JBoss MSC version 1.0.4.GA-redhat-1
[Server:failover-one] 13:04:23,142 INFO  [org.jboss.as] (MSC service thread 1-1) JBAS015899: JBoss EAP 6.2.0.Alpha1 (AS 7.3.0.Final-redhat-SNAPSHOT) starting
[Server:failover-one] 13:04:23,236 INFO  [org.xnio] (MSC service thread 1-1) XNIO Version 3.0.7.GA-redhat-1
[Server:failover-one] 13:04:23,242 INFO  [org.xnio.nio] (MSC service thread 1-1) XNIO NIO Implementation Version 3.0.7.GA-redhat-1
[Server:failover-one] 13:04:23,252 INFO  [org.jboss.remoting] (MSC service thread 1-1) JBoss Remoting version 3.2.16.GA-redhat-1
[Host Controller] 13:04:24,286 INFO  [org.jboss.as.domain.controller.mgmt] (Remoting "build02-windows:MANAGEMENT" task-3) JBAS010920: Server [Server:failover-one] registered using connection [Channel ID 690d9f0a (inbound) of Remoting connection 01ad4609 to /127.0.0.1:64143]
[Host Controller] 13:04:24,377 INFO  [org.jboss.as] (Controller Boot Thread) JBAS015961: Http management interface listening on http://127.0.0.1:9990/management
[Host Controller] 13:04:24,379 INFO  [org.jboss.as] (Controller Boot Thread) JBAS015951: Admin console listening on http://127.0.0.1:9990
[Host Controller] 13:04:24,381 INFO  [org.jboss.as] (Controller Boot Thread) JBAS015874: JBoss EAP 6.2.0.Alpha1 (AS 7.3.0.Final-redhat-SNAPSHOT) (Host Controller) started in 7584ms - Started 12 of 12 services (0 services are passive or on-demand)
[Host Controller] 13:04:24,421 INFO  [org.jboss.as.host.controller] (server-registration-threads - 1) JBAS010919: Registering server failover-one
INFO  [org.jboss.as.test.integration.domain.management.util.DomainLifecycleUtil] ServerIdentity{name=failover-one, host=failover-h1, server-group=main-server-group} status is starting
INFO  [org.jboss.as.test.integration.domain.management.util.DomainLifecycleUtil] ServerIdentity{name=failover-one, host=failover-h1, server-group=main-server-group} status is starting
INFO  [org.jboss.as.test.integration.domain.management.util.DomainLifecycleUtil] ServerIdentity{name=failover-one, host=failover-h1, server-group=main-server-group} status is starting
[Server:failover-one] 13:04:26,226 INFO  [org.jboss.as.modcluster] (ServerService Thread Pool -- 28) JBAS011704: Mod_cluster uses default load balancer provider
[Server:failover-one] 13:04:26,229 INFO  [org.jboss.as.connector.subsystems.datasources] (ServerService Thread Pool -- 33) JBAS010403: Deploying JDBC-compliant driver class org.h2.Driver (version 1.3)
[Server:failover-one] 13:04:26,233 INFO  [org.jboss.as.naming] (ServerService Thread Pool -- 44) JBAS011800: Activating Naming Subsystem
[Server:failover-one] 13:04:26,234 INFO  [org.jboss.as.security] (ServerService Thread Pool -- 43) JBAS013171: Activating Security Subsystem
[Server:failover-one] 13:04:26,266 INFO  [org.jboss.as.webservices] (ServerService Thread Pool -- 29) JBAS015537: Activating WebServices Extension
[Server:failover-one] 13:04:26,296 INFO  [org.jboss.as.connector.logging] (MSC service thread 1-2) JBAS010408: Starting JCA Subsystem (IronJacamar 1.0.20.Final-redhat-1)
[Server:failover-one] 13:04:26,308 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 47) JBAS010280: Activating Infinispan subsystem.
[Server:failover-one] 13:04:26,546 INFO  [org.jboss.as.jsf] (ServerService Thread Pool -- 52) JBAS012605: Activated the following JSF Implementations: [main, 1.2]
INFO  [org.jboss.as.test.integration.domain.management.util.DomainLifecycleUtil] ServerIdentity{name=failover-one, host=failover-h1, server-group=main-server-group} status is starting
[Server:failover-one] 13:04:26,850 INFO  [org.jboss.as.security] (MSC service thread 1-4) JBAS013170: Current PicketBox version=4.0.17.SP2-redhat-2
[Server:failover-one] 13:04:26,869 INFO  [org.jboss.as.clustering.jgroups] (ServerService Thread Pool -- 46) JBAS010260: Activating JGroups subsystem.
[Server:failover-one] 13:04:27,043 INFO  [org.jboss.as.naming] (MSC service thread 1-3) JBAS011802: Starting Naming Service
[Server:failover-one] 13:04:27,090 INFO  [org.jboss.jaxr] (MSC service thread 1-1) JBAS014000: Started JAXR subsystem, binding JAXR connection factory into JNDI as: java:jboss/jaxr/ConnectionFactory
[Server:failover-one] 13:04:27,116 INFO  [org.apache.catalina.core] (MSC service thread 1-4) JBWEB001065: The native library which allows optimal performance in production environments was not found on the java.library.path: C:\opt\Java\jdk1.7.0_25\bin;C:\Windows\Sun\Java\bin;C:\Windows\system32;C:\Windows;C:\BuildAgent\tools\maven3\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;;.
INFO  [org.jboss.as.test.integration.domain.management.util.DomainLifecycleUtil] ServerIdentity{name=failover-one, host=failover-h1, server-group=main-server-group} status is starting
[Server:failover-one] 13:04:27,234 INFO  [org.jboss.as.mail.extension] (MSC service thread 1-4) JBAS015400: Bound mail session [java:jboss/mail/Default]
[Server:failover-one] 13:04:27,246 INFO  [org.jboss.ws.common.management] (MSC service thread 1-3) JBWS022052: Starting JBoss Web Services - Stack CXF Server 4.2.0.Final-redhat-1
INFO  [org.jboss.as.test.integration.domain.management.util.DomainLifecycleUtil] ServerIdentity{name=failover-one, host=failover-h1, server-group=main-server-group} status is starting
[Server:failover-one] 13:04:28,067 INFO  [com.arjuna.ats.jbossatx] (MSC service thread 1-3) ARJUNA032010: JBossTS Recovery Service (tag: 449fb1f97e8bce23557820800037cb460091898b) - JBoss Inc.
[Server:failover-one] 13:04:28,070 INFO  [org.apache.coyote.ajp] (MSC service thread 1-4) JBWEB003046: Starting Coyote AJP/1.3 on ajp-127.0.0.1/127.0.0.1:8009
[Server:failover-one] 13:04:28,071 INFO  [org.apache.coyote.http11] (MSC service thread 1-1) JBWEB003001: Coyote HTTP/1.1 initializing on : http-127.0.0.1/127.0.0.1:8080
[Server:failover-one] 13:04:28,093 INFO  [org.apache.coyote.http11] (MSC service thread 1-1) JBWEB003000: Coyote HTTP/1.1 starting on: http-127.0.0.1/127.0.0.1:8080
[Server:failover-one] 13:04:28,113 INFO  [com.arjuna.ats.arjuna] (MSC service thread 1-3) ARJUNA012324: Start RecoveryActivators
[Server:failover-one] 13:04:28,144 INFO  [com.arjuna.ats.arjuna] (Transaction Expired Entry Monitor) ARJUNA012296: ExpiredEntryMonitor running at Fri, 13 Sep 2013 13:04:28
[Server:failover-one] 13:04:28,175 INFO  [org.hornetq.core.server] (MSC service thread 1-2) HQ221000: live server is starting with configuration HornetQ Configuration (clustered=false,backup=false,sharedStore=true,journalDirectory=C:\BuildAgent\work\ab7853673ba2ff34\testsuite\domain\target\domains\DomainControllerMigrationTestCase\failover1\servers\failover-one\data\messagingjournal,bindingsDirectory=C:\BuildAgent\work\ab7853673ba2ff34\testsuite\domain\target\domains\DomainControllerMigrationTestCase\failover1\servers\failover-one\data\messagingbindings,largeMessagesDirectory=C:\BuildAgent\work\ab7853673ba2ff34\testsuite\domain\target\domains\DomainControllerMigrationTestCase\failover1\servers\failover-one\data\messaginglargemessages,pagingDirectory=C:\BuildAgent\work\ab7853673ba2ff34\testsuite\domain\target\domains\DomainControllerMigrationTestCase\failover1\servers\failover-one\data\messagingpaging)
[Server:failover-one] 13:04:28,179 INFO  [org.jboss.modcluster] (ServerService Thread Pool -- 54) MODCLUSTER000001: Initializing mod_cluster ${project.version}
[Server:failover-one] 13:04:28,185 INFO  [com.arjuna.ats.arjuna] (MSC service thread 1-3) ARJUNA012343: RecoveryManagerImple is ready. Socket listener is turned off.
[Server:failover-one] 13:04:28,186 INFO  [com.arjuna.ats.jbossatx] (MSC service thread 1-3) ARJUNA032013: Starting transaction recovery manager
[Server:failover-one] 13:04:28,192 INFO  [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (ServerService Thread Pool -- 47) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
[Server:failover-one] 13:04:28,216 INFO  [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (ServerService Thread Pool -- 47) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
[Server:failover-one] 13:04:28,247 INFO  [org.jboss.modcluster] (ServerService Thread Pool -- 54) MODCLUSTER000032: Listening to proxy advertisements on /224.0.1.105:23364
[Server:failover-one] 13:04:28,461 INFO  [com.arjuna.ats.jbossatx] (MSC service thread 1-1) ARJUNA032017: JBossTS Transaction Service (JTA version - tag: 449fb1f97e8bce23557820800037cb460091898b) - JBoss Inc.
[Server:failover-one] 13:04:28,527 INFO  [org.jboss.as.remoting] (MSC service thread 1-3) JBAS017100: Listening on 127.0.0.1:4447
INFO  [org.jboss.as.test.integration.domain.management.util.DomainLifecycleUtil] ServerIdentity{name=failover-one, host=failover-h1, server-group=main-server-group} status is starting
[Server:failover-one] 13:04:28,671 INFO  [org.hornetq.core.server] (MSC service thread 1-2) HQ221020: Started Netty Acceptor version 3.6.6.Final-redhat-1-fd3c6b7 127.0.0.1:5455 for CORE protocol
[Server:failover-one] 13:04:28,674 INFO  [org.hornetq.core.server] (MSC service thread 1-2) HQ221020: Started Netty Acceptor version 3.6.6.Final-redhat-1-fd3c6b7 127.0.0.1:5445 for CORE protocol
[Server:failover-one] 13:04:28,678 INFO  [org.hornetq.core.server] (MSC service thread 1-2) HQ221007: Server is now live
[Server:failover-one] 13:04:28,679 INFO  [org.hornetq.core.server] (MSC service thread 1-2) HQ221001: HornetQ Server version 2.3.5.Final-redhat-2 (Monster Bee, 123) [8ca76772-1c96-11e3-9390-2f44b96a45d1] 
[Server:failover-one] 13:04:28,793 INFO  [org.hornetq.core.server] (ServerService Thread Pool -- 54) HQ221003: trying to deploy queue jms.queue.testQueue
[Server:failover-one] 13:04:28,852 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-1) JBAS010400: Bound data source [java:jboss/datasources/ExampleDS]
[Server:failover-one] 13:04:28,902 INFO  [org.jboss.as.messaging] (ServerService Thread Pool -- 54) JBAS011601: Bound messaging object to jndi name java:/queue/test
INFO  [org.jboss.as.test.in[Server:failover-one] 13:04:28,927 INFO  [org.jboss.as.connector.deployment] (MSC service thread 1-2) JBAS010406: Registered connection factory java:/JmsXA
tegration.domain.management.util.DomainLifecycleUtil] ServerIdentity{name=failover-one, host[Server:failover-one] 13:04:28,926 INFO  [org.jboss.as.messaging] (ServerService Thread Pool -- 57) JBAS011601: Bound messaging object to jndi name java:/ConnectionFactory
=failover-h1, server-group=main-server-[Server:failover-one] 13:04:28,935 INFO  [org.jboss.as.messaging] (ServerService Thread Pool -- 56) JBAS011601: Bound messaging object to jndi name java:/RemoteConnectionFactory
group} status is[Server:failover-one] 13:04:28,936 INFO  [org.hornetq.core.server] (ServerService Thread Pool -- 55) HQ221003: trying to deploy queue jms.topic.testTopic
 starting
[Server:failover-one] 13:04:28,952 INFO  [org.jboss.as.messaging] (ServerService Thread Pool -- 55) JBAS011601: Bound messaging object to jndi name java:/topic/test
[Server:failover-one] 13:04:28,977 INFO  [org.hornetq.ra] (MSC service thread 1-2) HornetQ resource adaptor started
[Server:failover-one] 13:04:28,978 INFO  [org.jboss.as.connector.services.resourceadapters.ResourceAdapterActivatorService$ResourceAdapterActivator] (MSC service thread 1-2) IJ020002: Deployed: file://RaActivatorhornetq-ra
[Server:failover-one] 13:04:28,980 INFO  [org.jboss.as.connector.deployment] (MSC service thread 1-4) JBAS010401: Bound JCA ConnectionFactory [java:/JmsXA]
[Server:failover-one] 13:04:29,028 INFO  [org.jboss.as] (Controller Boot Thread) JBAS015874: JBoss EAP 6.2.0.Alpha1 (AS 7.3.0.Final-redhat-SNAPSHOT) started in 7409ms - Started 131 of 221 services (89 services are passive or on-demand)
INFO  [org.jboss.as.test.integration.domain.management.util.DomainLifecycleUtil] All servers started in 13749 ms
INFO  [org.jboss.as.test.integration.domain.management.util.DomainLifecycleUtil] HostController started in 14012 ms
C:\BuildAgent\work\ab7853673ba2ff34\testsuite\domain\target\test-classes\domain-configs\domain-standard.xml
C:\BuildAgent\work\ab7853673ba2ff34\testsuite\domain\target\test-classes\host-configs\host-failover2.xml
13:04:30,253 INFO  [org.jboss.modules] (main) JBoss Modules version 1.3.0.Beta4
13:04:30,394 INFO  [org.jboss.as.process.Host Controller.status] (main) JBAS012017: Starting process 'Host Controller'
[Host Controller] 13:04:31,103 INFO  [org.jboss.modules] (main) JBoss Modules version 1.3.0.Beta4
[Host Controller] 13:04:31,273 INFO  [org.jboss.msc] (main) JBoss MSC version 1.0.4.GA-redhat-1
[Host Controller] 13:04:31,496 INFO  [org.jboss.as] (MSC service thread 1-2) JBAS015899: JBoss EAP 6.2.0.Alpha1 (AS 7.3.0.Final-redhat-SNAPSHOT) starting
[Host Controller] 13:04:32,589 INFO  [org.xnio] (MSC service thread 1-4) XNIO Version 3.0.7.GA-redhat-1
[Host Controller] 13:04:32,595 INFO  [org.xnio.nio] (MSC service thread 1-4) XNIO NIO Implementation Version 3.0.7.GA-redhat-1
[Host Controller] 13:04:32,606 INFO  [org.jboss.remoting] (MSC service thread 1-4) JBoss Remoting version 3.2.16.GA-redhat-1
[Host Controller] 13:04:32,619 INFO  [org.jboss.as] (Controller Boot Thread) JBAS010902: Creating http management service using network interface (public) port (9980) securePort (-1)
[Host Controller] 13:04:32,942 INFO  [org.jboss.as.remoting] (MSC service thread 1-3) JBAS017100: Listening on 127.0.0.1:9989
[Host Controller] 13:04:35,107 INFO  [org.jboss.as.domain] (slave-request-threads - 1) JBAS010918: Registered remote slave host "failover-h2", JBoss EAP 6.2.0.Alpha1 (AS 7.3.0.Final-redhat-SNAPSHOT)
[Host Controller] 13:04:35,311 INFO  [org.jboss.as.host.controller] (Controller Boot Thread) JBAS010922: Starting server failover-two
13:04:35,370 INFO  [org.jboss.as.process.Server:failover-two.status] (ProcessController-threads - 3) JBAS012017: Starting process 'Server:failover-two'
[Server:failover-two] 13:04:36,496 INFO  [org.jboss.modules] (main) JBoss Modules version 1.3.0.Beta4
[Server:failover-two] 13:04:36,772 INFO  [org.jboss.msc] (main) JBoss MSC version 1.0.4.GA-redhat-1
[Server:failover-two] 13:04:36,887 INFO  [org.jboss.as] (MSC service thread 1-2) JBAS015899: JBoss EAP 6.2.0.Alpha1 (AS 7.3.0.Final-redhat-SNAPSHOT) starting
[Server:failover-two] 13:04:36,976 INFO  [org.xnio] (MSC service thread 1-2) XNIO Version 3.0.7.GA-redhat-1
[Server:failover-two] 13:04:36,992 INFO  [org.xnio.nio] (MSC service thread 1-2) XNIO NIO Implementation Version 3.0.7.GA-redhat-1
[Server:failover-two] 13:04:37,033 INFO  [org.jboss.remoting] (MSC service thread 1-2) JBoss Remoting version 3.2.16.GA-redhat-1
[Host Controller] 13:04:38,121 INFO  [org.jboss.as.domain.controller.mgmt] (Remoting "build02-windows:MANAGEMENT" task-1) JBAS010920: Server [Server:failover-two] registered using connection [Channel ID 49b57302 (inbound) of Remoting connection 01c9e523 to /127.0.0.1:64162]
[Host Controller] 13:04:38,200 INFO  [org.jboss.as] (Controller Boot Thread) JBAS015961: Http management interface listening on http://127.0.0.1:9980/management
[Host Controller] 13:04:38,200 INFO  [org.jboss.as] (Controller Boot Thread) JBAS015954: Admin console is not enabled
[Host Controller] 13:04:38,201 INFO  [org.jboss.as] (Controller Boot Thread) JBAS015874: JBoss EAP 6.2.0.Alpha1 (AS 7.3.0.Final-redhat-SNAPSHOT) (Host Controller) started in 7562ms - Started 12 of 12 services (0 services are passive or on-demand)
[Host Controller] 13:04:38,264 INFO  [org.jboss.as.host.controller] (server-registration-threads - 1) JBAS010919: Registering server failover-two
INFO  [org.jboss.as.test.integration.domain.management.util.DomainLifecycleUtil] ServerIdentity{name=failover-two, host=failover-h2, server-group=main-server-group} status is starting
INFO  [org.jboss.as.test.integration.domain.management.util.DomainLifecycleUtil] ServerIdentity{name=failover-two, host=failover-h2, server-group=main-server-group} status is starting
[Server:failover-two] 13:04:39,441 INFO  [org.jboss.as.modcluster] (ServerService Thread Pool -- 28) JBAS011704: Mod_cluster uses default load balancer provider
[Server:failover-two] 13:04:39,490 INFO  [org.jboss.as.jsf] (ServerService Thread Pool -- 52) JBAS012605: Activated the following JSF Implementations: [main, 1.2]
[Server:failover-two] 13:04:39,517 INFO  [org.jboss.as.clustering.jgroups] (ServerService Thread Pool -- 46) JBAS010260: Activating JGroups subsystem.
[Server:failover-two] 13:04:39,536 INFO  [org.jboss.as.naming] (ServerService Thread Pool -- 44) JBAS011800: Activating Naming Subsystem
[Server:failover-two] 13:04:39,558 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 47) JBAS010280: Activating Infinispan subsystem.
[Server:failover-two] 13:04:39,769 INFO  [org.jboss.as.security] (ServerService Thread Pool -- 43) JBAS013171: Activating Security Subsystem
INFO  [org.jboss.as.test.integration.domain.management.util.DomainLifecycleUtil] ServerIdentity{name=failover-two, host=failover-h2, server-group=main-server-group} status is starting
[Server:failover-two] 13:04:40,017 INFO  [org.jboss.as.security] (MSC service thread 1-2) JBAS013170: Current PicketBox version=4.0.17.SP2-redhat-2
[Server:failover-two] 13:04:40,128 INFO  [org.jboss.as.connector.logging] (MSC service thread 1-1) JBAS010408: Starting JCA Subsystem (IronJacamar 1.0.20.Final-redhat-1)
[Server:failover-two] 13:04:40,147 INFO  [org.jboss.as.webservices] (ServerService Thread Pool -- 29) JBAS015537: Activating WebServices Extension
[Server:failover-two] 13:04:40,158 INFO  [org.jboss.as.connector.subsystems.datasources] (ServerService Thread Pool -- 33) JBAS010403: Deploying JDBC-compliant driver class org.h2.Driver (version 1.3)
INFO  [org.jboss.as.test.integration.domain.management.util.DomainLifecycleUtil] ServerIdentity{name=failover-two, host=failover-h2, server-group=main-server-group} status is starting
[Server:failover-two] 13:04:40,356 INFO  [org.apache.catalina.core] (MSC service thread 1-3) JBWEB001065: The native library which allows optimal performance in production environments was not found on the java.library.path: C:\opt\Java\jdk1.7.0_25\bin;C:\Windows\Sun\Java\bin;C:\Windows\system32;C:\Windows;C:\BuildAgent\tools\maven3\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;;.
[Server:failover-two] 13:04:40,430 INFO  [org.jboss.ws.common.management] (MSC service thread 1-1) JBWS022052: Starting JBoss Web Services - Stack CXF Server 4.2.0.Final-redhat-1
[Server:failover-two] 13:04:40,443 INFO  [org.jboss.as.naming] (MSC service thread 1-1) JBAS011802: Starting Naming Service
[Server:failover-two] 13:04:40,537 INFO  [org.jboss.jaxr] (MSC service thread 1-2) JBAS014000: Started JAXR subsystem, binding JAXR connection factory into JNDI as: java:jboss/jaxr/ConnectionFactory
[Server:failover-two] 13:04:40,558 INFO  [org.jboss.as.mail.extension] (MSC service thread 1-1) JBAS015400: Bound mail session [java:jboss/mail/Default]
INFO  [org.jboss.as.test.integration.domain.management.util.DomainLifecycleUtil] ServerIdentity{name=failover-two, host=failover-h2, server-group=main-server-group} status is starting
[Server:failover-two] 13:04:40,928 INFO  [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (ServerService Thread Pool -- 47) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
[Server:failover-two] 13:04:40,935 INFO  [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (ServerService Thread Pool -- 47) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
INFO  [org.jboss.as.test.integration.domain.management.util.DomainLifecycleUtil] ServerIdentity{name=failover-two, host=failover-h2, server-group=main-server-group} status is starting
[Server:failover-two] 13:04:41,134 INFO  [com.arjuna.ats.jbossatx] (MSC service thread 1-4) ARJUNA032010: JBossTS Recovery Service (tag: 449fb1f97e8bce23557820800037cb460091898b) - JBoss Inc.
[Server:failover-two] 13:04:41,153 INFO  [com.arjuna.ats.arjuna] (MSC service thread 1-4) ARJUNA012324: Start RecoveryActivators
[Server:failover-two] 13:04:41,176 INFO  [org.apache.coyote.http11] (MSC service thread 1-1) JBWEB003001: Coyote HTTP/1.1 initializing on : http-127.0.0.1/127.0.0.1:8430
[Server:failover-two] 13:04:41,177 INFO  [org.apache.coyote.http11] (MSC service thread 1-1) JBWEB003000: Coyote HTTP/1.1 starting on: http-127.0.0.1/127.0.0.1:8430
[Server:failover-two] 13:04:41,172 INFO  [com.arjuna.ats.arjuna] (Transaction Expired Entry Monitor) ARJUNA012296: ExpiredEntryMonitor running at Fri, 13 Sep 2013 13:04:41
[Server:failover-two] 13:04:41,179 INFO  [org.apache.coyote.ajp] (MSC service thread 1-2) JBWEB003046: Starting Coyote AJP/1.3 on ajp-127.0.0.1/127.0.0.1:8359
[Server:failover-two] 13:04:41,225 INFO  [org.jboss.as.remoting] (MSC service thread 1-2) JBAS017100: Listening on 127.0.0.1:4797
[Server:failover-two] 13:04:41,232 INFO  [com.arjuna.ats.arjuna] (MSC service thread 1-4) ARJUNA012343: RecoveryManagerImple is ready. Socket listener is turned off.
[Server:failover-two] 13:04:41,232 INFO  [com.arjuna.ats.jbossatx] (MSC service thread 1-4) ARJUNA032013: Starting transaction recovery manager
[Server:failover-two] 13:04:41,247 INFO  [org.hornetq.core.server] (MSC service thread 1-3) HQ221000: live server is starting with configuration HornetQ Configuration (clustered=false,backup=false,sharedStore=true,journalDirectory=C:\BuildAgent\work\ab7853673ba2ff34\testsuite\domain\target\domains\DomainControllerMigrationTestCase\failover2\servers\failover-two\data\messagingjournal,bindingsDirectory=C:\BuildAgent\work\ab7853673ba2ff34\testsuite\domain\target\domains\DomainControllerMigrationTestCase\failover2\servers\failover-two\data\messagingbindings,largeMessagesDirectory=C:\BuildAgent\work\ab7853673ba2ff34\testsuite\domain\target\domains\DomainControllerMigrationTestCase\failover2\servers\failover-two\data\messaginglargemessages,pagingDirectory=C:\BuildAgent\work\ab7853673ba2ff34\testsuite\domain\target\domains\DomainControllerMigrationTestCase\failover2\servers\failover-two\data\messagingpaging)
[Server:failover-two] 13:04:41,272 INFO  [com.arjuna.ats.jbossatx] (MSC service thread 1-4) ARJUNA032017: JBossTS Transaction Service (JTA version - tag: 449fb1f97e8bce23557820800037cb460091898b) - JBoss Inc.
[Server:failover-two] 13:04:41,551 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-1) JBAS010400: Bound data source [java:jboss/datasources/ExampleDS]
[Server:failover-two] 13:04:41,645 INFO  [org.hornetq.core.server] (MSC service thread 1-3) HQ221020: Started Netty Acceptor version 3.6.6.Final-redhat-1-fd3c6b7 127.0.0.1:5805 for CORE protocol
[Server:failover-two] 13:04:41,661 INFO  [org.hornetq.core.server] (MSC service thread 1-3) HQ221020: Started Netty Acceptor version 3.6.6.Final-redhat-1-fd3c6b7 127.0.0.1:5795 for CORE protocol
[Server:failover-two] 13:04:41,661 INFO  [org.hornetq.core.server] (MSC service thread 1-3) HQ221007: Server is now live
[Server:failover-two] 13:04:41,661 INFO  [org.hornetq.core.server] (MSC service thread 1-3) HQ221001: HornetQ Server version 2.3.5.Final-redhat-2 (Monster Bee, 123) [9472a4ad-1c96-11e3-9806-69b2683df9fe] 
[Server:failover-two] 13:04:41,728 INFO  [org.jboss.modcluster] (ServerService Thread Pool -- 54) MODCLUSTER000001: Initializing mod_cluster ${project.version}
[Server:failover-two] 13:04:41,743 INFO  [org.jboss.modcluster] (ServerService Thread Pool -- 54) MODCLUSTER000032: Listening to proxy advertisements on /224.0.1.105:23364
INFO  [org.jboss.as.test.integration.domain.management.util.DomainLifecycleUtil] ServerIdentity{name=failover-two, host=failover-h2, server-group=main-server-group} status is starting
[Server:failover-two] 13:04:41,891 INFO  [org.jboss.as.messaging] (ServerService Thread Pool -- 55) JBAS011601: Bound messaging object to jndi name java:/ConnectionFactory
[Server:failover-two] 13:04:41,893 INFO  [org.hornetq.core.server] (ServerService Thread Pool -- 54) HQ221003: trying to deploy queue jms.topic.testTopic
[Server:failover-two] 13:04:41,927 INFO  [org.jboss.as.connector.deployment] (MSC service thread 1-3) JBAS010406: Registered connection factory java:/JmsXA
[Server:failover-two] 13:04:41,967 INFO  [org.jboss.as.messaging] (ServerService Thread Pool -- 54) JBAS011601: Bound messaging object to jndi name java:/topic/test
[Server:failover-two] 13:04:41,970 INFO  [org.jboss.as.messaging] (ServerService Thread Pool -- 56) JBAS011601: Bound messaging object to jndi name java:/RemoteConnectionFactory
[Server:failover-two] 13:04:41,972 INFO  [org.hornetq.core.server] (ServerService Thread Pool -- 57) HQ221003: trying to deploy queue jms.queue.testQueue
[Server:failover-two] 13:04:41,979 INFO  [org.hornetq.ra] (MSC service thread 1-3) HornetQ resource adaptor started
[Server:failover-two] 13:04:41,980 INFO  [org.jboss.as.connector.services.resourceadapters.ResourceAdapterActivatorService$ResourceAdapterActivator] (MSC service thread 1-3) IJ020002: Deployed: file://RaActivatorhornetq-ra
[Server:failover-two] 13:04:41,982 INFO  [org.jboss.as.connector.deployment] (MSC service thread 1-1) JBAS010401: Bound JCA ConnectionFactory [java:/JmsXA]
[Server:failover-two] 13:04:41,986 INFO  [org.jboss.as.messaging] (ServerService Thread Pool -- 57) JBAS011601: Bound messaging object to jndi name java:/queue/test
[Server:failover-two] 13:04:42,025 INFO  [org.jboss.as] (Controller Boot Thread) JBAS015874: JBoss EAP 6.2.0.Alpha1 (AS 7.3.0.Final-redhat-SNAPSHOT) started in 6502ms - Started 131 of 221 services (89 services are passive or on-demand)
INFO  [org.jboss.as.test.integration.domain.management.util.DomainLifecycleUtil] All servers started in 12478 ms
INFO  [org.jboss.as.test.integration.domain.management.util.DomainLifecycleUtil] HostController started in 12734 ms
C:\BuildAgent\work\ab7853673ba2ff34\testsuite\domain\target\test-classes\domain-configs\domain-standard.xml
C:\BuildAgent\work\ab7853673ba2ff34\testsuite\domain\target\test-classes\host-configs\host-failover3.xml
13:04:43,121 INFO  [org.jboss.modules] (main) JBoss Modules version 1.3.0.Beta4
13:04:43,251 INFO  [org.jboss.as.process.Host Controller.status] (main) JBAS012017: Starting process 'Host Controller'
[Host Controller] 13:04:44,091 INFO  [org.jboss.modules] (main) JBoss Modules version 1.3.0.Beta4
[Host Controller] 13:04:44,262 INFO  [org.jboss.msc] (main) JBoss MSC version 1.0.4.GA-redhat-1
[Host Controller] 13:04:44,326 INFO  [org.jboss.as] (MSC service thread 1-1) JBAS015899: JBoss EAP 6.2.0.Alpha1 (AS 7.3.0.Final-redhat-SNAPSHOT) starting
[Host Controller] 13:04:45,237 INFO  [org.jboss.as] (Controller Boot Thread) JBAS010902: Creating http management service using network interface (public) port (19990) securePort (-1)
[Host Controller] 13:04:45,238 INFO  [org.xnio] (MSC service thread 1-1) XNIO Version 3.0.7.GA-redhat-1
[Host Controller] 13:04:45,255 INFO  [org.xnio.nio] (MSC service thread 1-1) XNIO NIO Implementation Version 3.0.7.GA-redhat-1
[Host Controller] 13:04:45,295 INFO  [org.jboss.remoting] (MSC service thread 1-1) JBoss Remoting version 3.2.16.GA-redhat-1
[Host Controller] 13:04:45,642 INFO  [org.jboss.as.remoting] (MSC service thread 1-2) JBAS017100: Listening on 127.0.0.1:19999
[Host Controller] 13:04:47,930 INFO  [org.jboss.as.domain] (slave-request-threads - 1) JBAS010918: Registered remote slave host "failover-h3", JBoss EAP 6.2.0.Alpha1 (AS 7.3.0.Final-redhat-SNAPSHOT)
[Host Controller] 13:04:48,106 INFO  [org.jboss.as.host.controller] (Controller Boot Thread) JBAS010922: Starting server failover-three
13:04:48,133 INFO  [org.jboss.as.process.Server:failover-three.status] (ProcessController-threads - 3) JBAS012017: Starting process 'Server:failover-three'
[Server:failover-three] 13:04:48,752 INFO  [org.jboss.modules] (main) JBoss Modules version 1.3.0.Beta4
[Server:failover-three] 13:04:49,254 INFO  [org.jboss.msc] (main) JBoss MSC version 1.0.4.GA-redhat-1
[Server:failover-three] 13:04:49,361 INFO  [org.jboss.as] (MSC service thread 1-4) JBAS015899: JBoss EAP 6.2.0.Alpha1 (AS 7.3.0.Final-redhat-SNAPSHOT) starting
[Server:failover-three] 13:04:49,436 INFO  [org.xnio] (MSC service thread 1-3) XNIO Version 3.0.7.GA-redhat-1
[Server:failover-three] 13:04:49,442 INFO  [org.xnio.nio] (MSC service thread 1-3) XNIO NIO Implementation Version 3.0.7.GA-redhat-1
[Server:failover-three] 13:04:49,451 INFO  [org.jboss.remoting] (MSC service thread 1-3) JBoss Remoting version 3.2.16.GA-redhat-1
[Host Controller] 13:04:50,425 INFO  [org.jboss.as.domain.controller.mgmt] (Remoting "build02-windows:MANAGEMENT" task-2) JBAS010920: Server [Server:failover-three] registered using connection [Channel ID 68b6cfc4 (inbound) of Remoting connection 00d20052 to /127.0.0.1:64181]
[Host Controller] 13:04:50,499 INFO  [org.jboss.as] (Controller Boot Thread) JBAS015961: Http management interface listening on http://127.0.0.1:19990/management
[Host Controller] 13:04:50,499 INFO  [org.jboss.as] (Controller Boot Thread) JBAS015954: Admin console is not enabled
[Host Controller] 13:04:50,500 INFO  [org.jboss.as] (Controller Boot Thread) JBAS015874: JBoss EAP 6.2.0.Alpha1 (AS 7.3.0.Final-redhat-SNAPSHOT) (Host Controller) started in 7125ms - Started 12 of 12 services (0 services are passive or on-demand)
[Host Controller] 13:04:50,539 INFO  [org.jboss.as.host.controller] (server-registration-threads - 1) JBAS010919: Registering server failover-three
INFO  [org.jboss.as.test.integration.domain.management.util.DomainLifecycleUtil] ServerIdentity{name=failover-three, host=failover-h3, server-group=main-server-group} status is starting
INFO  [org.jboss.as.test.integration.domain.management.util.DomainLifecycleUtil] ServerIdentity{name=failover-three, host=failover-h3, server-group=main-server-group} status is starting
[Server:failover-three] 13:04:51,830 INFO  [org.jboss.as.modcluster] (ServerService Thread Pool -- 28) JBAS011704: Mod_cluster uses default load balancer provider
[Server:failover-three] 13:04:51,913 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 47) JBAS010280: Activating Infinispan subsystem.
[Server:failover-three] 13:04:51,919 INFO  [org.jboss.as.security] (ServerService Thread Pool -- 43) JBAS013171: Activating Security Subsystem
[Server:failover-three] 13:04:51,951 INFO  [org.jboss.as.webservices] (ServerService Thread Pool -- 29) JBAS015537: Activating WebServices Extension
[Server:failover-three] 13:04:51,952 INFO  [org.jboss.as.naming] (ServerService Thread Pool -- 44) JBAS011800: Activating Naming Subsystem
[Server:failover-three] 13:04:51,960 INFO  [org.jboss.as.clustering.jgroups] (ServerService Thread Pool -- 46) JBAS010260: Activating JGroups subsystem.
[Server:failover-three] 13:04:51,971 INFO  [org.jboss.as.connector.subsystems.datasources] (ServerService Thread Pool -- 33) JBAS010403: Deploying JDBC-compliant driver class org.h2.Driver (version 1.3)
[Server:failover-three] 13:04:52,141 INFO  [org.jboss.as.security] (MSC service thread 1-4) JBAS013170: Current PicketBox version=4.0.17.SP2-redhat-2
[Server:failover-three] 13:04:52,152 INFO  [org.apache.catalina.core] (MSC service thread 1-3) JBWEB001065: The native library which allows optimal performance in production environments was not found on the java.library.path: C:\opt\Java\jdk1.7.0_25\bin;C:\Windows\Sun\Java\bin;C:\Windows\system32;C:\Windows;C:\BuildAgent\tools\maven3\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;;.
[Server:failover-three] 13:04:52,190 INFO  [org.jboss.as.jsf] (ServerService Thread Pool -- 52) JBAS012605: Activated the following JSF Implementations: [main, 1.2]
INFO  [org.j[Server:failover-three] 13:04:52,224 INFO  [org.jboss.as.naming] (MSC service thread 1-4) JBAS011802: Starting Naming Service
boss.as.test.integratio[Server:failover-three] 13:04:52,281 INFO  [org.jboss.as.mail.extension] (MSC service thread 1-3) JBAS015400: Bound mail session [java:jboss/mail/Default]
n.domain[Server:failover-three] 13:04:52,299 INFO  [org.jboss.ws.common.management] (MSC service thread 1-4) JBWS022052: Starting JBoss Web Services - Stack CXF Server 4.2.0.Final-redhat-1
.management.util.DomainLifecycleUtil] ServerIdentity{name=failover-three, host=failover-h3, server-group=main-server-group} status is starting
[Server:failover-three] 13:04:52,347 INFO  [org.jboss.jaxr] (MSC service thread 1-2) JBAS014000: Started JAXR subsystem, binding JAXR connection factory into JNDI as: java:jboss/jaxr/ConnectionFactory
[Server:failover-three] 13:04:52,375 INFO  [org.jboss.as.connector.logging] (MSC service thread 1-2) JBAS010408: Starting JCA Subsystem (IronJacamar 1.0.20.Final-redhat-1)
INFO  [org.jboss.as.test.integration.domain.management.util.DomainLifecycleUtil] ServerIdentity{name=failover-three, host=failover-h3, server-group=main-server-group} status is starting
[Server:failover-three] 13:04:53,428 INFO  [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (ServerService Thread Pool -- 47) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
[Server:failover-three] 13:04:53,435 INFO  [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (ServerService Thread Pool -- 47) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
INFO  [org.jboss.as.test.integration.domain.management.util.DomainLifecycleUtil] ServerIdentity{name=failover-three, host=failover-h3, server-group=main-server-group} status is starting
[Server:failover-three] 13:04:53,557 INFO  [org.apache.coyote.http11] (MSC service thread 1-1) JBWEB003001: Coyote HTTP/1.1 initializing on : http-/127.0.0.1:8630
[Server:failover-three] 13:04:53,563 INFO  [org.apache.coyote.http11] (MSC service thread 1-1) JBWEB003000: Coyote HTTP/1.1 starting on: http-/127.0.0.1:8630
[Server:failover-three] 13:04:53,593 INFO  [com.arjuna.ats.jbossatx] (MSC service thread 1-2) ARJUNA032010: JBossTS Recovery Service (tag: 449fb1f97e8bce23557820800037cb460091898b) - JBoss Inc.
[Server:failover-three] 13:04:53,593 INFO  [com.arjuna.ats.arjuna] (MSC service thread 1-2) ARJUNA012324: Start RecoveryActivators
[Server:failover-three] 13:04:53,647 INFO  [com.arjuna.ats.arjuna] (Transaction Expired Entry Monitor) ARJUNA012296: ExpiredEntryMonitor running at Fri, 13 Sep 2013 13:04:53
[Server:failover-three] 13:04:53,677 INFO  [org.apache.coyote.ajp] (MSC service thread 1-4) JBWEB003046: Starting Coyote AJP/1.3 on ajp-127.0.0.1/127.0.0.1:8559
[Server:failover-three] 13:04:53,679 INFO  [org.jboss.as.remoting] (MSC service thread 1-1) JBAS017100: Listening on 127.0.0.1:4997
[Server:failover-three] 13:04:53,679 INFO  [com.arjuna.ats.arjuna] (MSC service thread 1-2) ARJUNA012343: RecoveryManagerImple is ready. Socket listener is turned off.
[Server:failover-three] 13:04:53,680 INFO  [com.arjuna.ats.jbossatx] (MSC service thread 1-2) ARJUNA032013: Starting transaction recovery manager
[Server:failover-three] 13:04:53,735 INFO  [com.arjuna.ats.jbossatx] (MSC service thread 1-2) ARJUNA032017: JBossTS Transaction Service (JTA version - tag: 449fb1f97e8bce23557820800037cb460091898b) - JBoss Inc.
[Server:failover-three] 13:04:53,736 INFO  [org.hornetq.core.server] (MSC service thread 1-3) HQ221000: live server is starting with configuration HornetQ Configuration (clustered=false,backup=false,sharedStore=true,journalDirectory=C:\BuildAgent\work\ab7853673ba2ff34\testsuite\domain\target\domains\DomainControllerMigrationTestCase\failover3\servers\failover-three\data\messagingjournal,bindingsDirectory=C:\BuildAgent\work\ab7853673ba2ff34\testsuite\domain\target\domains\DomainControllerMigrationTestCase\failover3\servers\failover-three\data\messagingbindings,largeMessagesDirectory=C:\BuildAgent\work\ab7853673ba2ff34\testsuite\domain\target\domains\DomainControllerMigrationTestCase\failover3\servers\failover-three\data\messaginglargemessages,pagingDirectory=C:\BuildAgent\work\ab7853673ba2ff34\testsuite\domain\target\domains\DomainControllerMigrationTestCase\failover3\servers\failover-three\data\messagingpaging)
[Server:failover-three] 13:04:53,995 INFO  [org.jboss.modcluster] (ServerService Thread Pool -- 54) MODCLUSTER000001: Initializing mod_cluster ${project.version}
[Server:failover-three] 13:04:54,020 INFO  [org.jboss.modcluster] (ServerService Thread Pool -- 54) MODCLUSTER000032: Listening to proxy advertisements on /224.0.1.105:23364
INFO  [org.jboss.as.test.integration.domain.management.util.DomainLifecycleUtil] ServerIdentity{name=f[Server:failover-three] 13:04:54,081 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-1) JBAS010400: Bound data source [java:jboss/datasources/ExampleDS]
ailover-three, host=failover-h3, server-group=main-server-group} status is starting
[Server:failover-three] 13:04:54,149 INFO  [org.hornetq.core.server] (MSC service thread 1-3) HQ221020: Started Netty Acceptor version 3.6.6.Final-redhat-1-fd3c6b7 127.0.0.1:6005 for CORE protocol
[Server:failover-three] 13:04:54,151 INFO  [org.hornetq.core.server] (MSC service thread 1-3) HQ221020: Started Netty Acceptor version 3.6.6.Final-redhat-1-fd3c6b7 127.0.0.1:5995 for CORE protocol
[Server:failover-three] 13:04:54,153 INFO  [org.hornetq.core.server] (MSC service thread 1-3) HQ221007: Server is now live
[Server:failover-three] 13:04:54,154 INFO  [org.hornetq.core.server] (MSC service thread 1-3) HQ221001: HornetQ Server version 2.3.5.Final-redhat-2 (Monster Bee, 123) [9be42909-1c96-11e3-83aa-8db5c91cf96a] 
[Server:failover-three] 13:04:54,433 INFO  [org.jboss.as.connector.deployment] (MSC service thread 1-1) JBAS010406: Registered connection factory java:/JmsXA
[Server:failover-three] 13:04:54,510 INFO  [org.jboss.as.messaging] (ServerService Thread Pool -- 56) JBAS011601: Bound messaging object to jndi name java:/ConnectionFactory
INFO  [org.jboss.as.test.integration.domain.management.util.DomainLifecycleUtil] ServerIdentity{name=failover-three, host=failover-h3, server-group=main-server-group} status is starting
[Server:failover-three] 13:04:54,528 INFO  [org.hornetq.core.server] (ServerService Thread Pool -- 54) HQ221003: trying to deploy queue jms.topic.testTopic
[Server:failover-three] 13:04:54,556 INFO  [org.hornetq.ra] (MSC service thread 1-1) HornetQ resource adaptor started
[Server:failover-three] 13:04:54,558 INFO  [org.jboss.as.connector.services.resourceadapters.ResourceAdapterActivatorService$ResourceAdapterActivator] (MSC service thread 1-1) IJ020002: Deployed: file://RaActivatorhornetq-ra
[Server:failover-three] 13:04:54,560 INFO  [org.jboss.as.connector.deployment] (MSC service thread 1-1) JBAS010401: Bound JCA ConnectionFactory [java:/JmsXA]
[Server:failover-three] 13:04:54,596 INFO  [org.jboss.as.messaging] (ServerService Thread Pool -- 54) JBAS011601: Bound messaging object to jndi name java:/topic/test
[Server:failover-three] 13:04:54,598 INFO  [org.jboss.as.messaging] (ServerService Thread Pool -- 57) JBAS011601: Bound messaging object to jndi name java:/RemoteConnectionFactory
[Server:failover-three] 13:04:54,599 INFO  [org.hornetq.core.server] (ServerService Thread Pool -- 55) HQ221003: trying to deploy queue jms.queue.testQueue
[Server:failover-three] 13:04:54,610 INFO  [org.jboss.as.messaging] (ServerService Thread Pool -- 55) JBAS011601: Bound messaging object to jndi name java:/queue/test
[Server:failover-three] 13:04:54,650 INFO  [org.jboss.as] (Controller Boot Thread) JBAS015874: JBoss EAP 6.2.0.Alpha1 (AS 7.3.0.Final-redhat-SNAPSHOT) started in 6334ms - Started 131 of 221 services (89 services are passive or on-demand)
INFO  [org.jboss.as.test.integration.domain.management.util.DomainLifecycleUtil] All servers started in 12443 ms
INFO  [org.jboss.as.test.integration.domain.management.util.DomainLifecycleUtil] HostController started in 12704 ms
INFO  [org.jboss.as.test.integration.domain.DomainControllerMigrationTestCase] Stopping the domain controller on failover-h1.
INFO  [org.jboss.as.test.integration.domain.DomainControllerMigrationTestCase] Domain controller on failover-h1 stopped.
INFO  [org.jboss.as.test.integration.domain.DomainControllerMigrationTestCase] Reloading failover-h2 to act as the domain controller.
[Host Controller] 13:04:55,812 INFO  [org.jboss.as.host.controller] (Remoting "build02-windows:MANAGEMENT" task-3) JBAS010928: Unregistered at domain controller
[Host Controller] 13:04:55,820 INFO  [org.jboss.as.host.controller] (MSC service thread 1-4) JBAS010926: Unregistering server failover-two
[Host Controller] 13:04:55,852 INFO  [org.jboss.as] (MSC service thread 1-3) JBAS015950: JBoss EAP 6.2.0.Alpha1 (AS 7.3.0.Final-redhat-SNAPSHOT) stopped in 84ms
[Host Controller] 13:04:55,855 INFO  [org.jboss.as] (MSC service thread 1-3) JBAS015899: JBoss EAP 6.2.0.Alpha1 (AS 7.3.0.Final-redhat-SNAPSHOT) starting
[Host Controller] 13:04:55,923 INFO  [org.jboss.as] (Controller Boot Thread) JBAS010902: Creating http management service using network interface (public) port (9980) securePort (-1)
[Host Controller] 13:04:56,064 WARN  [org.jboss.as.host.controller] (Remoting "build02-windows:MANAGEMENT" read-1) JBAS010914: Connection to remote host-controller closed. Trying to reconnect.
[Host Controller] 13:04:56,160 INFO  [org.jboss.as.remoting] (MSC service thread 1-4) JBAS017100: Listening on 127.0.0.1:9989
[Host Controller] 13:04:56,234 WARN  [org.jboss.as.threads] (Controller Boot Thread) JBAS012400: The 'per-cpu' attribute is no longer supported. The value [10.000000] of the 'count' attribute is being combined with the value [20.000000] of the 'per-cpu' attribute and the current processor count [2] to derive a new value of [50] for 'count'.
INFO  [org.jboss.as.test.integration.domain.DomainControllerMigrationTestCase] Reloading failover-h3 to register to new domain controller.
[Host Controller] 13:04:56,688 INFO  [org.jboss.as.host.controller] (MSC service thread 1-2) JBAS010926: Unregistering server failover-three
[Host Controller] 13:04:56,761 WARN  [org.jboss.as.jmx] (Controller Boot Thread) JBAS011305: <jmx-connector/> is no longer supporting. <remoting-connector/> should be used instead to allow remote connections via JBoss Remoting.
[Host Controller] 13:04:56,807 WARN  [org.jboss.as.threads] (Controller Boot Thread) JBAS012400: The 'per-cpu' attribute is no longer supported. The value [10.000000] of the 'count' attribute is being combined with the value [20.000000] of the 'per-cpu' attribute and the current processor count [2] to derive a new value of [50] for 'count'.
[Host Controller] 13:04:56,825 WARN  [org.jboss.as.jmx] (Controller Boot Thread) JBAS011305: <jmx-connector/> is no longer supporting. <remoting-connector/> should be used instead to allow remote connections via JBoss Remoting.
[Host Controller] 13:04:56,827 WARN  [org.jboss.as.threads] (Controller Boot Thread) JBAS012400: The 'per-cpu' attribute is no longer supported. The value [10.000000] of the 'count' attribute is being combined with the value [20.000000] of the 'per-cpu' attribute and the current processor count [2] to derive a new value of [50] for 'count'.
[Host Controller] 13:04:56,828 WARN  [org.jboss.as.threads] (Controller Boot Thread) JBAS012400: The 'per-cpu' attribute is no longer supported. The value [10.000000] of the 'count' attribute is being combined with the value [20.000000] of the 'per-cpu' attribute and the current processor count [2] to derive a new value of [50] for 'count'.
[Host Controller] 13:04:56,830 WARN  [org.jboss.as.threads] (Controller Boot Thread) JBAS012400: The 'per-cpu' attribute is no longer supported. The value [10.000000] of the 'count' attribute is being combined with the value [20.000000] of the 'per-cpu' attribute and the current processor count [2] to derive a new value of [50] for 'count'.
[Host Controller] 13:04:56,831 WARN  [org.jboss.as.threads] (Controller Boot Thread) JBAS012400: The 'per-cpu' attribute is no longer supported. The value [10.000000] of the 'count' attribute is being combined with the value [20.000000] of the 'per-cpu' attribute and the current processor count [2] to derive a new value of [50] for 'count'.
[Host Controller] 13:04:56,833 WARN  [org.jboss.as.threads] (Controller Boot Thread) JBAS012400: The 'per-cpu' attribute is no longer supported. The value [10.000000] of the 'count' attribute is being combined with the value [20.000000] of the 'per-cpu' attribute and the current processor count [2] to derive a new value of [50] for 'count'.
[Host Controller] 13:04:56,834 WARN  [org.jboss.as.threads] (Controller Boot Thread) JBAS012400: The 'per-cpu' attribute is no longer supported. The value [10.000000] of the 'count' attribute is being combined with the value [20.000000] of the 'per-cpu' attribute and the current processor count [2] to derive a new value of [50] for 'count'.
[Host Controller] 13:04:57,382 INFO  [org.jboss.as.host.controller] (Controller Boot Thread) JBAS010917: Reconnecting server failover-two
[Host Controller] 13:04:57,385 INFO  [org.jboss.as.host.controller] (Controller Boot Thread) JBAS010919: Registering server failover-two
[Host Controller] 13:04:57,392 INFO  [org.jboss.as] (Controller Boot Thread) JBAS015961: Http management interface listening on http://127.0.0.1:9980/management
[Host Controller] 13:04:57,393 INFO  [org.jboss.as] (Controller Boot Thread) JBAS015951: Admin console listening on http://127.0.0.1:9980
[Host Controller] 13:04:57,393 INFO  [org.jboss.as] (Controller Boot Thread) JBAS015874: JBoss EAP 6.2.0.Alpha1 (AS 7.3.0.Final-redhat-SNAPSHOT) (Host Controller) started in 1537ms - Started 12 of 12 services (0 services are passive or on-demand)
[Host Controller] 13:04:57,423 INFO  [org.jboss.as.domain.controller.mgmt] (Remoting "failover-h2:MANAGEMENT" task-3) JBAS010920: Server [Server:failover-two] registered using connection [Channel ID 086619a2 (inbound) of Remoting connection 016e21b4 to /127.0.0.1:64199]
[Host Controller] 13:05:01,169 INFO  [org.jboss.as] (MSC service thread 1-3) JBAS015950: JBoss EAP 6.2.0.Alpha1 (AS 7.3.0.Final-redhat-SNAPSHOT) stopped in 4494ms
[Host Controller] 13:05:01,170 INFO  [org.jboss.as] (MSC service thread 1-3) JBAS015899: JBoss EAP 6.2.0.Alpha1 (AS 7.3.0.Final-redhat-SNAPSHOT) starting
[Host Controller] 13:05:01,223 INFO  [org.jboss.as] (Controller Boot Thread) JBAS010902: Creating http management service using network interface (public) port (19990) securePort (-1)
[Host Controller] 13:05:01,474 INFO  [org.jboss.as.remoting] (MSC service thread 1-3) JBAS017100: Listening on 127.0.0.1:19999
[Host Controller] 13:05:01,852 INFO  [org.jboss.as.domain] (slave-request-threads - 1) JBAS010918: Registered remote slave host "failover-h3", JBoss EAP 6.2.0.Alpha1 (AS 7.3.0.Final-redhat-SNAPSHOT)
[Host Controller] 13:05:01,922 INFO  [org.jboss.as.host.controller] (Controller Boot Thread) JBAS010917: Reconnecting server failover-three
[Host Controller] 13:05:01,924 INFO  [org.jboss.as.host.controller] (Controller Boot Thread) JBAS010919: Registering server failover-three
[Host Controller] 13:05:01,927 INFO  [org.jboss.as] (Controller Boot Thread) JBAS015961: Http management interface listening on http://127.0.0.1:19990/management
[Host Controller] 13:05:01,927 INFO  [org.jboss.as] (Controller Boot Thread) JBAS015954: Admin console is not enabled
[Host Controller] 13:05:01,928 INFO  [org.jboss.as] (Controller Boot Thread) JBAS015874: JBoss EAP 6.2.0.Alpha1 (AS 7.3.0.Final-redhat-SNAPSHOT) (Host Controller) started in 756ms - Started 12 of 12 services (0 services are passive or on-demand)
[Host Controller] 13:05:02,019 INFO  [org.jboss.as.repository] (management-handler-thread - 2) JBAS014900: Content added at location C:\BuildAgent\work\ab7853673ba2ff34\testsuite\domain\target\domains\DomainControllerMigrationTestCase\failover2\data\content\7e\600909f99d93e22e7656d71f12e510762de811\content
[Host Controller] 13:05:02,388 INFO  [org.jboss.as.domain.controller.mgmt] (Remoting "failover-h3:MANAGEMENT" task-1) JBAS010920: Server [Server:failover-three] registered using connection [Channel ID 40e8ed6a (inbound) of Remoting connection 00b9f428 to /127.0.0.1:64209]
[Host Controller] 13:05:02,437 ERROR [org.jboss.as.controller.management-operation] (domain-connection-threads - 2) JBAS014612: Operation ("composite") failed - address: ([
[Host Controller]     ("host" => "failover-h3"),
[Host Controller]     ("server" => "failover-three")
[Host Controller] ]): java.lang.RuntimeException: java.io.IOException: JBAS012175: Channel closed
[Host Controller]     at org.jboss.as.controller.remote.RemoteProxyController.execute(RemoteProxyController.java:183) [jboss-as-controller-7.3.0.Final-redhat-SNAPSHOT.jar:7.3.0.Final-redhat-SNAPSHOT]
[Host Controller]     at org.jboss.as.controller.TransformingProxyController$TransformingProxyControllerImpl.execute(TransformingProxyController.java:149) [jboss-as-controller-7.3.0.Final-redhat-SNAPSHOT.jar:7.3.0.Final-redhat-SNAPSHOT]
[Host Controller]     at org.jboss.as.controller.ProxyStepHandler.execute(ProxyStepHandler.java:130) [jboss-as-controller-7.3.0.Final-redhat-SNAPSHOT.jar:7.3.0.Final-redhat-SNAPSHOT]
[Host Controller]     at org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:625) [jboss-as-controller-7.3.0.Final-redhat-SNAPSHOT.jar:7.3.0.Final-redhat-SNAPSHOT]
[Host Controller]     at org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:503) [jboss-as-controller-7.3.0.Final-redhat-SNAPSHOT.jar:7.3.0.Final-redhat-SNAPSHOT]
[Host Controller]     at org.jboss.as.controller.AbstractOperationContext.completeStepInternal(AbstractOperationContext.java:285) [jboss-as-controller-7.3.0.Final-redhat-SNAPSHOT.jar:7.3.0.Final-redhat-SNAPSHOT]
[Host Controller]     at org.jboss.as.controller.AbstractOperationContext.executeOperation(AbstractOperationContext.java:280) [jboss-as-controller-7.3.0.Final-redhat-SNAPSHOT.jar:7.3.0.Final-redhat-SNAPSHOT]
[Host Controller]     at org.jboss.as.controller.ModelControllerImpl.internalExecute(ModelControllerImpl.java:217) [jboss-as-controller-7.3.0.Final-redhat-SNAPSHOT.jar:7.3.0.Final-redhat-SNAPSHOT]
[Host Controller]     at org.jboss.as.controller.ModelControllerImpl.execute(ModelControllerImpl.java:134) [jboss-as-controller-7.3.0.Final-redhat-SNAPSHOT.jar:7.3.0.Final-redhat-SNAPSHOT]
[Host Controller]     at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler.doExecute(TransactionalProtocolOperationHandler.java:128) [jboss-as-controller-7.3.0.Final-redhat-SNAPSHOT.jar:7.3.0.Final-redhat-SNAPSHOT]
[Host Controller]     at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler$1.run(TransactionalProtocolOperationHandler.java:107) [jboss-as-controller-7.3.0.Final-redhat-SNAPSHOT.jar:7.3.0.Final-redhat-SNAPSHOT]
[Host Controller]     at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$ExecuteRequestHandler$1.run(TransactionalProtocolOperationHandler.java:103) [jboss-as-controller-7.3.0.Final-redhat-SNAPSHOT.jar:7.3.0.Final-redhat-SNAPSHOT]
[Host Controller]     at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_25]
[Host Controller]     at javax.security.auth.Subject.doAs(Subject.java:356) [rt.jar:1.7.0_25]
[Host Controller]     at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$GetSubjectResponseHandler$1$1.run(TransactionalProtocolOperationHandler.java:198) [jboss-as-controller-7.3.0.Final-redhat-SNAPSHOT.jar:7.3.0.Final-redhat-SNAPSHOT]
[Host Controller]     at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$GetSubjectResponseHandler$1$1.run(TransactionalProtocolOperationHandler.java:194) [jboss-as-controller-7.3.0.Final-redhat-SNAPSHOT.jar:7.3.0.Final-redhat-SNAPSHOT]
[Host Controller]     at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0_25]
[Host Controller]     at org.jboss.as.controller.remote.TransactionalProtocolOperationHandler$GetSubjectResponseHandler$1.execute(TransactionalProtocolOperationHandler.java:194) [jboss-as-controller-7.3.0.Final-redhat-SNAPSHOT.jar:7.3.0.Final-redhat-SNAPSHOT]
[Host Controller]     at org.jboss.as.protocol.mgmt.AbstractMessageHandler$2$1.doExecute(AbstractMessageHandler.java:296) [jboss-as-protocol-7.3.0.Final-redhat-SNAPSHOT.jar:7.3.0.Final-redhat-SNAPSHOT]
[Host Controller]     at org.jboss.as.protocol.mgmt.AbstractMessageHandler$AsyncTaskRunner.run(AbstractMessageHandler.java:518) [jboss-as-protocol-7.3.0.Final-redhat-SNAPSHOT.jar:7.3.0.Final-redhat-SNAPSHOT]
[Host Controller]     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25]
[Host Controller]     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25]
[Host Controller]     at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25]
[Host Controller]     at org.jboss.threads.JBossThread.run(JBossThread.java:122) [jboss-threads-2.1.1.Final-redhat-1.jar:2.1.1.Final-redhat-1]
[Host Controller] Caused by: java.io.IOException: JBAS012175: Channel closed
[Host Controller]     at org.jboss.as.host.controller.ManagedServerProxy$DisconnectedProtocolClient.execute(ManagedServerProxy.java:114)
[Host Controller]     at org.jboss.as.host.controller.ManagedServerProxy.execute(ManagedServerProxy.java:92)
[Host Controller]     at org.jboss.as.host.controller.ManagedServerProxy.execute(ManagedServerProxy.java:80)
[Host Controller]     at org.jboss.as.controller.remote.RemoteProxyController.execute(RemoteProxyController.java:139) [jboss-as-controller-7.3.0.Final-redhat-SNAPSHOT.jar:7.3.0.Final-redhat-SNAPSHOT]
[Host Controller]     ... 23 more
[Host Controller] 
[Server:failover-two] 13:05:02,503 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-3) JBAS015876: Starting deployment of "SimpleServlet.war" (runtime-name: "SimpleServlet.war")
[Server:failover-two] 13:05:03,424 INFO  [org.jboss.web] (ServerService Thread Pool -- 64) JBAS018210: Register web context: /SimpleServlet
[Server:failover-two] 13:05:03,547 ERROR [org.jboss.as.server] (host-controller-connection-threads - 1) JBAS015871: Deploy of deployment "SimpleServlet.war" was rolled back with no failure message
[Server:failover-two] 13:05:03,553 INFO  [org.jboss.web] (ServerService Thread Pool -- 64) JBAS018224: Unregister web context: /SimpleServlet
[Server:failover-two] 13:05:03,585 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-3) JBAS015877: Stopped deployment SimpleServlet.war (runtime-name: SimpleServlet.war) in 36ms


The [Host Controller] that wrote the ERROR logging above is failover-3.

Comment 2 Ladislav Thon 2013-11-14 14:05:48 UTC
Verified with EAP 6.2.0.CR1. The test doesn't fail and the fix is present in the code.