Login
[x]
Log in using an account from:
Fedora Account System
Red Hat Associate
Red Hat Customer
Or login using a Red Hat Bugzilla account
Forgot Password
Login:
Hide Forgot
Create an Account
Red Hat Bugzilla – Attachment 882685 Details for
Bug 1084481
Getting lots of concurrent access timeout exceptions: org.jboss.invocation.InterceptorContext$Invocation@27900b06 - could not obtain lock within 5000MILLISECONDS
[?]
New
Simple Search
Advanced Search
My Links
Browse
Requests
Reports
Current State
Search
Tabular reports
Graphical reports
Duplicates
Other Reports
User Changes
Plotly Reports
Bug Status
Bug Severity
Non-Defaults
|
Product Dashboard
Help
Page Help!
Bug Writing Guidelines
What's new
Browser Support Policy
5.0.4.rh83 Release notes
FAQ
Guides index
User guide
Web Services
Contact
Legal
This site requires JavaScript to be enabled to function correctly, please enable it.
JON server log with the error
server.log.2014-04-03 (text/plain), 1.46 MB, created by
Radim Hatlapatka
on 2014-04-04 12:52:43 UTC
(
hide
)
Description:
JON server log with the error
Filename:
MIME Type:
Creator:
Radim Hatlapatka
Created:
2014-04-04 12:52:43 UTC
Size:
1.46 MB
patch
obsolete
>08:00:24,977 INFO [org.jboss.as.ejb3] (EJB default - 4) JBAS014121: Timer: [id=e1f92a5e-faf4-48d7-8e09-3cd7fd5f6f5d timedObjectId=rhq.rhq-server.SystemManagerBean auto-timer?:false persistent?:false timerService=org.jboss.as.ejb3.timerservice.TimerServiceImpl@64bdf1ae initialExpiration=Mon Mar 31 05:19:19 EDT 2014 intervalDuration(in milli sec)=60000 nextExpiration=Thu Apr 03 08:01:19 EDT 2014 timerState=IN_TIMEOUT will be retried >08:00:24,977 INFO [org.jboss.as.ejb3] (EJB default - 4) JBAS014123: Retrying timeout for timer: [id=e1f92a5e-faf4-48d7-8e09-3cd7fd5f6f5d timedObjectId=rhq.rhq-server.SystemManagerBean auto-timer?:false persistent?:false timerService=org.jboss.as.ejb3.timerservice.TimerServiceImpl@64bdf1ae initialExpiration=Mon Mar 31 05:19:19 EDT 2014 intervalDuration(in milli sec)=60000 nextExpiration=Thu Apr 03 08:01:19 EDT 2014 timerState=IN_TIMEOUT >08:00:25,325 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-41) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:00:25,379 ERROR [org.jboss.as.ejb3] (RHQScheduler_Worker-4) javax.ejb.EJBTransactionRolledbackException: JBAS014373: EJB 3.1 PFD2 4.8.5.5.1 concurrent access timeout on org.jboss.invocation.InterceptorContext$Invocation@27900b06 - could not obtain lock within 5000MILLISECONDS >08:00:25,380 ERROR [org.jboss.as.ejb3.invocation] (RHQScheduler_Worker-4) JBAS014134: EJB Invocation failed on component SystemManagerBean for method public abstract org.rhq.core.domain.common.composite.SystemSettings org.rhq.enterprise.server.system.SystemManagerLocal.getUnmaskedSystemSettings(boolean): javax.ejb.EJBTransactionRolledbackException: JBAS014373: EJB 3.1 PFD2 4.8.5.5.1 concurrent access timeout on org.jboss.invocation.InterceptorContext$Invocation@27900b06 - could not obtain lock within 5000MILLISECONDS > at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleInCallerTx(CMTTxInterceptor.java:138) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInCallerTx(CMTTxInterceptor.java:228) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:317) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.system.SystemManagerLocal$$$view165.getUnmaskedSystemSettings(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.enterprise.server.core.AgentManagerBean.checkForSuspectAgents(AgentManagerBean.java:238) [rhq-server.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor244.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.jpa.interceptor.SBInvocationInterceptor.processInvocation(SBInvocationInterceptor.java:47) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.pool.PooledInstanceInterceptor.processInvocation(PooledInstanceInterceptor.java:51) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.core.AgentManagerLocal$$$view108.checkForSuspectAgents(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.enterprise.server.scheduler.jobs.CheckForSuspectedAgentsJob.executeJobCode(CheckForSuspectedAgentsJob.java:40) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.enterprise.server.scheduler.jobs.AbstractStatefulJob.execute(AbstractStatefulJob.java:48) [rhq-server.jar:4.9.0.JON320GA] > at org.quartz.core.JobRunShell.run(JobRunShell.java:202) [quartz-1.6.5.jar:1.6.5] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:525) [quartz-1.6.5.jar:1.6.5] >Caused by: javax.ejb.ConcurrentAccessTimeoutException: JBAS014373: EJB 3.1 PFD2 4.8.5.5.1 concurrent access timeout on org.jboss.invocation.InterceptorContext$Invocation@27900b06 - could not obtain lock within 5000MILLISECONDS > at org.jboss.as.ejb3.concurrency.ContainerManagedConcurrencyInterceptor.processInvocation(ContainerManagedConcurrencyInterceptor.java:100) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInCallerTx(CMTTxInterceptor.java:226) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > ... 72 more > >08:00:25,442 WARN [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-4) Agent quiet time config is invalid in DB, defaulting to: 300000: javax.ejb.EJBTransactionRolledbackException: JBAS014373: EJB 3.1 PFD2 4.8.5.5.1 concurrent access timeout on org.jboss.invocation.InterceptorContext$Invocation@27900b06 - could not obtain lock within 5000MILLISECONDS > at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleInCallerTx(CMTTxInterceptor.java:138) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInCallerTx(CMTTxInterceptor.java:228) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:317) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.system.SystemManagerLocal$$$view165.getUnmaskedSystemSettings(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.enterprise.server.core.AgentManagerBean.checkForSuspectAgents(AgentManagerBean.java:238) [rhq-server.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor244.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.jpa.interceptor.SBInvocationInterceptor.processInvocation(SBInvocationInterceptor.java:47) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.pool.PooledInstanceInterceptor.processInvocation(PooledInstanceInterceptor.java:51) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.core.AgentManagerLocal$$$view108.checkForSuspectAgents(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.enterprise.server.scheduler.jobs.CheckForSuspectedAgentsJob.executeJobCode(CheckForSuspectedAgentsJob.java:40) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.enterprise.server.scheduler.jobs.AbstractStatefulJob.execute(AbstractStatefulJob.java:48) [rhq-server.jar:4.9.0.JON320GA] > at org.quartz.core.JobRunShell.run(JobRunShell.java:202) [quartz-1.6.5.jar:1.6.5] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:525) [quartz-1.6.5.jar:1.6.5] >Caused by: javax.ejb.ConcurrentAccessTimeoutException: JBAS014373: EJB 3.1 PFD2 4.8.5.5.1 concurrent access timeout on org.jboss.invocation.InterceptorContext$Invocation@27900b06 - could not obtain lock within 5000MILLISECONDS > at org.jboss.as.ejb3.concurrency.ContainerManagedConcurrencyInterceptor.processInvocation(ContainerManagedConcurrencyInterceptor.java:100) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInCallerTx(CMTTxInterceptor.java:226) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > ... 72 more > >08:00:25,528 WARN [org.hibernate.engine.jdbc.spi.SqlExceptionHelper] (RHQScheduler_Worker-4) SQL Error: 0, SQLState: null >08:00:25,528 ERROR [org.hibernate.engine.jdbc.spi.SqlExceptionHelper] (RHQScheduler_Worker-4) javax.resource.ResourceException: IJ000460: Error checking for a transaction >08:00:25,569 ERROR [org.jboss.as.ejb3.invocation] (RHQScheduler_Worker-4) JBAS014134: EJB Invocation failed on component AgentManagerBean for method public abstract void org.rhq.enterprise.server.core.AgentManagerLocal.checkForSuspectAgents(): javax.ejb.EJBException: javax.persistence.PersistenceException: org.hibernate.exception.GenericJDBCException: Could not open connection > at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleExceptionInOurTx(CMTTxInterceptor.java:165) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:250) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.core.AgentManagerLocal$$$view108.checkForSuspectAgents(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.enterprise.server.scheduler.jobs.CheckForSuspectedAgentsJob.executeJobCode(CheckForSuspectedAgentsJob.java:40) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.enterprise.server.scheduler.jobs.AbstractStatefulJob.execute(AbstractStatefulJob.java:48) [rhq-server.jar:4.9.0.JON320GA] > at org.quartz.core.JobRunShell.run(JobRunShell.java:202) [quartz-1.6.5.jar:1.6.5] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:525) [quartz-1.6.5.jar:1.6.5] >Caused by: javax.persistence.PersistenceException: org.hibernate.exception.GenericJDBCException: Could not open connection > at org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1387) [hibernate-entitymanager-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1310) [hibernate-entitymanager-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.ejb.QueryImpl.getResultList(QueryImpl.java:273) [hibernate-entitymanager-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.jboss.as.jpa.container.QueryNonTxInvocationDetacher.getResultList(QueryNonTxInvocationDetacher.java:58) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.core.AgentManagerBean.checkForSuspectAgents(AgentManagerBean.java:253) [rhq-server.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor244.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.jpa.interceptor.SBInvocationInterceptor.processInvocation(SBInvocationInterceptor.java:47) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.pool.PooledInstanceInterceptor.processInvocation(PooledInstanceInterceptor.java:51) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > ... 26 more >Caused by: org.hibernate.exception.GenericJDBCException: Could not open connection > at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:54) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:125) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:110) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.obtainConnection(LogicalConnectionImpl.java:221) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.getConnection(LogicalConnectionImpl.java:157) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.engine.jdbc.internal.StatementPreparerImpl.connection(StatementPreparerImpl.java:56) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.engine.jdbc.internal.StatementPreparerImpl$5.doPrepare(StatementPreparerImpl.java:161) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.engine.jdbc.internal.StatementPreparerImpl$StatementPreparationTemplate.prepareStatement(StatementPreparerImpl.java:182) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.engine.jdbc.internal.StatementPreparerImpl.prepareQueryStatement(StatementPreparerImpl.java:159) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.loader.Loader.prepareQueryStatement(Loader.java:1854) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1831) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1811) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.loader.Loader.doQuery(Loader.java:899) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:341) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.loader.Loader.doList(Loader.java:2516) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.loader.Loader.doList(Loader.java:2502) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2332) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.loader.Loader.list(Loader.java:2327) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.loader.hql.QueryLoader.list(QueryLoader.java:490) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.hql.internal.ast.QueryTranslatorImpl.list(QueryTranslatorImpl.java:355) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.rhq.core.domain.util.JoinFetchReportingQueryTranslator.list(JoinFetchReportingQueryTranslator.java:84) [rhq-core-domain-ejb3.jar:4.9.0.JON320GA] > at org.hibernate.engine.query.spi.HQLQueryPlan.performList(HQLQueryPlan.java:195) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.internal.SessionImpl.list(SessionImpl.java:1269) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.internal.QueryImpl.list(QueryImpl.java:101) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.ejb.QueryImpl.getResultList(QueryImpl.java:264) [hibernate-entitymanager-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > ... 51 more >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000460: Error checking for a transaction > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:147) > at org.hibernate.ejb.connection.InjectedDataSourceConnectionProvider.getConnection(InjectedDataSourceConnectionProvider.java:70) [hibernate-entitymanager-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.internal.AbstractSessionImpl$NonContextualJdbcConnectionAccess.obtainConnection(AbstractSessionImpl.java:292) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.obtainConnection(LogicalConnectionImpl.java:214) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > ... 72 more >Caused by: javax.resource.ResourceException: IJ000460: Error checking for a transaction > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:362) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:139) > ... 75 more >Caused by: javax.resource.ResourceException: IJ000459: Transaction is not active: tx=TransactionImple < ac, BasicAction: 0:ffff0a10b492:-289ef26c:533931c9:22f0ae status: ActionStatus.ABORT_ONLY > > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:352) > ... 77 more > >08:00:25,685 ERROR [org.rhq.enterprise.server.scheduler.jobs.AbstractStatefulJob] (RHQScheduler_Worker-4) Failed to execute job [CheckForSuspectedAgentsJob]: javax.ejb.EJBException: javax.persistence.PersistenceException: org.hibernate.exception.GenericJDBCException: Could not open connection > at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleExceptionInOurTx(CMTTxInterceptor.java:165) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:250) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.core.AgentManagerLocal$$$view108.checkForSuspectAgents(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.enterprise.server.scheduler.jobs.CheckForSuspectedAgentsJob.executeJobCode(CheckForSuspectedAgentsJob.java:40) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.enterprise.server.scheduler.jobs.AbstractStatefulJob.execute(AbstractStatefulJob.java:48) [rhq-server.jar:4.9.0.JON320GA] > at org.quartz.core.JobRunShell.run(JobRunShell.java:202) [quartz-1.6.5.jar:1.6.5] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:525) [quartz-1.6.5.jar:1.6.5] >Caused by: javax.persistence.PersistenceException: org.hibernate.exception.GenericJDBCException: Could not open connection > at org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1387) [hibernate-entitymanager-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1310) [hibernate-entitymanager-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.ejb.QueryImpl.getResultList(QueryImpl.java:273) [hibernate-entitymanager-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.jboss.as.jpa.container.QueryNonTxInvocationDetacher.getResultList(QueryNonTxInvocationDetacher.java:58) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.core.AgentManagerBean.checkForSuspectAgents(AgentManagerBean.java:253) [rhq-server.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor244.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.jpa.interceptor.SBInvocationInterceptor.processInvocation(SBInvocationInterceptor.java:47) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.pool.PooledInstanceInterceptor.processInvocation(PooledInstanceInterceptor.java:51) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > ... 26 more >Caused by: org.hibernate.exception.GenericJDBCException: Could not open connection > at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:54) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:125) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:110) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.obtainConnection(LogicalConnectionImpl.java:221) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.getConnection(LogicalConnectionImpl.java:157) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.engine.jdbc.internal.StatementPreparerImpl.connection(StatementPreparerImpl.java:56) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.engine.jdbc.internal.StatementPreparerImpl$5.doPrepare(StatementPreparerImpl.java:161) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.engine.jdbc.internal.StatementPreparerImpl$StatementPreparationTemplate.prepareStatement(StatementPreparerImpl.java:182) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.engine.jdbc.internal.StatementPreparerImpl.prepareQueryStatement(StatementPreparerImpl.java:159) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.loader.Loader.prepareQueryStatement(Loader.java:1854) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1831) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1811) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.loader.Loader.doQuery(Loader.java:899) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:341) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.loader.Loader.doList(Loader.java:2516) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.loader.Loader.doList(Loader.java:2502) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2332) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.loader.Loader.list(Loader.java:2327) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.loader.hql.QueryLoader.list(QueryLoader.java:490) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.hql.internal.ast.QueryTranslatorImpl.list(QueryTranslatorImpl.java:355) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.rhq.core.domain.util.JoinFetchReportingQueryTranslator.list(JoinFetchReportingQueryTranslator.java:84) [rhq-core-domain-ejb3.jar:4.9.0.JON320GA] > at org.hibernate.engine.query.spi.HQLQueryPlan.performList(HQLQueryPlan.java:195) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.internal.SessionImpl.list(SessionImpl.java:1269) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.internal.QueryImpl.list(QueryImpl.java:101) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.ejb.QueryImpl.getResultList(QueryImpl.java:264) [hibernate-entitymanager-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > ... 51 more >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000460: Error checking for a transaction > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:147) > at org.hibernate.ejb.connection.InjectedDataSourceConnectionProvider.getConnection(InjectedDataSourceConnectionProvider.java:70) [hibernate-entitymanager-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.internal.AbstractSessionImpl$NonContextualJdbcConnectionAccess.obtainConnection(AbstractSessionImpl.java:292) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > at org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.obtainConnection(LogicalConnectionImpl.java:214) [hibernate-core-4.2.0.SP1-redhat-1.jar:4.2.0.SP1-redhat-1] > ... 72 more >Caused by: javax.resource.ResourceException: IJ000460: Error checking for a transaction > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:362) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:139) > ... 75 more >Caused by: javax.resource.ResourceException: IJ000459: Transaction is not active: tx=TransactionImple < ac, BasicAction: 0:ffff0a10b492:-289ef26c:533931c9:22f0ae status: ActionStatus.ABORT_ONLY > > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:352) > ... 77 more > >08:00:28,208 ERROR [org.jboss.as.ejb3.invocation] (http-/0.0.0.0:7080-44) JBAS014134: EJB Invocation failed on component SystemManagerBean for method public abstract org.rhq.core.domain.common.ProductInfo org.rhq.enterprise.server.system.SystemManagerRemote.getProductInfo(org.rhq.core.domain.auth.Subject): javax.ejb.ConcurrentAccessTimeoutException: JBAS014373: EJB 3.1 PFD2 4.8.5.5.1 concurrent access timeout on org.jboss.invocation.InterceptorContext$Invocation@7a5f4cbe - could not obtain lock within 5000MILLISECONDS > at org.jboss.as.ejb3.concurrency.ContainerManagedConcurrencyInterceptor.processInvocation(ContainerManagedConcurrencyInterceptor.java:100) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.system.SystemManagerLocal$$$view165.getProductInfo(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.SystemGWTServiceImpl.getProductInfo(SystemGWTServiceImpl.java:59) > at sun.reflect.GeneratedMethodAccessor561.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] > >08:00:28,227 ERROR [org.jboss.as.ejb3.invocation] (http-/0.0.0.0:7080-74) JBAS014134: EJB Invocation failed on component SystemManagerBean for method public abstract org.rhq.core.domain.common.composite.SystemSettings org.rhq.enterprise.server.system.SystemManagerLocal.getUnmaskedSystemSettings(boolean): javax.ejb.ConcurrentAccessTimeoutException: JBAS014373: EJB 3.1 PFD2 4.8.5.5.1 concurrent access timeout on org.jboss.invocation.InterceptorContext$Invocation@31cde3b2 - could not obtain lock within 5000MILLISECONDS > at org.jboss.as.ejb3.concurrency.ContainerManagedConcurrencyInterceptor.processInvocation(ContainerManagedConcurrencyInterceptor.java:100) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.system.SystemManagerLocal$$$view165.getUnmaskedSystemSettings(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.SystemGWTServiceImpl.getSessionTimeout(SystemGWTServiceImpl.java:77) > at sun.reflect.GeneratedMethodAccessor560.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] > >08:00:28,352 WARN [org.rhq.coregui.server.gwt.SystemGWTServiceImpl] (http-/0.0.0.0:7080-44) Sending exception to client: [1396526428352] : javax.ejb.ConcurrentAccessTimeoutException: JBAS014373: EJB 3.1 PFD2 4.8.5.5.1 concurrent access timeout on org.jboss.invocation.InterceptorContext$Invocation@7a5f4cbe - could not obtain lock within 5000MILLISECONDS > at org.jboss.as.ejb3.concurrency.ContainerManagedConcurrencyInterceptor.processInvocation(ContainerManagedConcurrencyInterceptor.java:100) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.system.SystemManagerLocal$$$view165.getProductInfo(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.SystemGWTServiceImpl.getProductInfo(SystemGWTServiceImpl.java:59) [classes:] > at sun.reflect.GeneratedMethodAccessor561.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) [gwt-servlet-2.5.0.jar:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) [classes:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) [classes:] > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) [jboss-as-web-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] > >08:00:28,353 WARN [org.rhq.coregui.server.gwt.SystemGWTServiceImpl] (http-/0.0.0.0:7080-74) Sending exception to client: [1396526428353] : javax.ejb.ConcurrentAccessTimeoutException: JBAS014373: EJB 3.1 PFD2 4.8.5.5.1 concurrent access timeout on org.jboss.invocation.InterceptorContext$Invocation@31cde3b2 - could not obtain lock within 5000MILLISECONDS > at org.jboss.as.ejb3.concurrency.ContainerManagedConcurrencyInterceptor.processInvocation(ContainerManagedConcurrencyInterceptor.java:100) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.system.SystemManagerLocal$$$view165.getUnmaskedSystemSettings(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.SystemGWTServiceImpl.getSessionTimeout(SystemGWTServiceImpl.java:77) [classes:] > at sun.reflect.GeneratedMethodAccessor560.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) [gwt-servlet-2.5.0.jar:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) [classes:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) [classes:] > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) [jboss-as-web-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] > >08:00:28,818 INFO [org.rhq.enterprise.server.core.CoreServerServiceImpl] (http-/0.0.0.0:7080-80) Got agent registration request for new agent: dhcp180-58-agent-16163[10.16.180.47:16163][4.9.0.JON320GA(734bd56)] >08:00:28,753 ERROR [org.jboss.as.ejb3.invocation] (http-/0.0.0.0:7080-11) JBAS014134: EJB Invocation failed on component SystemManagerBean for method public abstract org.rhq.core.domain.common.ProductInfo org.rhq.enterprise.server.system.SystemManagerRemote.getProductInfo(org.rhq.core.domain.auth.Subject): javax.ejb.ConcurrentAccessTimeoutException: JBAS014373: EJB 3.1 PFD2 4.8.5.5.1 concurrent access timeout on org.jboss.invocation.InterceptorContext$Invocation@350ba25e - could not obtain lock within 5000MILLISECONDS > at org.jboss.as.ejb3.concurrency.ContainerManagedConcurrencyInterceptor.processInvocation(ContainerManagedConcurrencyInterceptor.java:100) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.system.SystemManagerLocal$$$view165.getProductInfo(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.SystemGWTServiceImpl.getProductInfo(SystemGWTServiceImpl.java:59) > at sun.reflect.GeneratedMethodAccessor561.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] > >08:00:28,746 ERROR [org.jboss.as.ejb3.invocation] (http-/0.0.0.0:7080-9) JBAS014134: EJB Invocation failed on component SystemManagerBean for method public abstract org.rhq.core.domain.common.composite.SystemSettings org.rhq.enterprise.server.system.SystemManagerLocal.getUnmaskedSystemSettings(boolean): javax.ejb.ConcurrentAccessTimeoutException: JBAS014373: EJB 3.1 PFD2 4.8.5.5.1 concurrent access timeout on org.jboss.invocation.InterceptorContext$Invocation@5d584bde - could not obtain lock within 5000MILLISECONDS > at org.jboss.as.ejb3.concurrency.ContainerManagedConcurrencyInterceptor.processInvocation(ContainerManagedConcurrencyInterceptor.java:100) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.system.SystemManagerLocal$$$view165.getUnmaskedSystemSettings(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.SystemGWTServiceImpl.getSessionTimeout(SystemGWTServiceImpl.java:77) > at sun.reflect.GeneratedMethodAccessor560.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] > >08:00:28,799 ERROR [org.jboss.as.ejb3.invocation] (http-/0.0.0.0:7080-37) JBAS014134: EJB Invocation failed on component SystemManagerBean for method public abstract org.rhq.core.domain.common.composite.SystemSettings org.rhq.enterprise.server.system.SystemManagerLocal.getUnmaskedSystemSettings(boolean): javax.ejb.ConcurrentAccessTimeoutException: JBAS014373: EJB 3.1 PFD2 4.8.5.5.1 concurrent access timeout on org.jboss.invocation.InterceptorContext$Invocation@59f81917 - could not obtain lock within 5000MILLISECONDS > at org.jboss.as.ejb3.concurrency.ContainerManagedConcurrencyInterceptor.processInvocation(ContainerManagedConcurrencyInterceptor.java:100) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.system.SystemManagerLocal$$$view165.getUnmaskedSystemSettings(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.SystemGWTServiceImpl.getSessionTimeout(SystemGWTServiceImpl.java:77) > at sun.reflect.GeneratedMethodAccessor560.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] > >08:00:28,814 ERROR [org.jboss.as.ejb3.invocation] (http-/0.0.0.0:7080-7) JBAS014134: EJB Invocation failed on component SystemManagerBean for method public abstract org.rhq.core.domain.common.ProductInfo org.rhq.enterprise.server.system.SystemManagerRemote.getProductInfo(org.rhq.core.domain.auth.Subject): javax.ejb.ConcurrentAccessTimeoutException: JBAS014373: EJB 3.1 PFD2 4.8.5.5.1 concurrent access timeout on org.jboss.invocation.InterceptorContext$Invocation@67de85 - could not obtain lock within 5000MILLISECONDS > at org.jboss.as.ejb3.concurrency.ContainerManagedConcurrencyInterceptor.processInvocation(ContainerManagedConcurrencyInterceptor.java:100) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.system.SystemManagerLocal$$$view165.getProductInfo(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.SystemGWTServiceImpl.getProductInfo(SystemGWTServiceImpl.java:59) > at sun.reflect.GeneratedMethodAccessor561.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] > >08:00:28,900 WARN [org.rhq.coregui.server.gwt.SystemGWTServiceImpl] (http-/0.0.0.0:7080-9) Sending exception to client: [1396526428900] : javax.ejb.ConcurrentAccessTimeoutException: JBAS014373: EJB 3.1 PFD2 4.8.5.5.1 concurrent access timeout on org.jboss.invocation.InterceptorContext$Invocation@5d584bde - could not obtain lock within 5000MILLISECONDS > at org.jboss.as.ejb3.concurrency.ContainerManagedConcurrencyInterceptor.processInvocation(ContainerManagedConcurrencyInterceptor.java:100) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.system.SystemManagerLocal$$$view165.getUnmaskedSystemSettings(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.SystemGWTServiceImpl.getSessionTimeout(SystemGWTServiceImpl.java:77) [classes:] > at sun.reflect.GeneratedMethodAccessor560.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) [gwt-servlet-2.5.0.jar:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) [classes:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) [classes:] > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) [jboss-as-web-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] > >08:00:28,899 WARN [org.rhq.coregui.server.gwt.SystemGWTServiceImpl] (http-/0.0.0.0:7080-11) Sending exception to client: [1396526428899] : javax.ejb.ConcurrentAccessTimeoutException: JBAS014373: EJB 3.1 PFD2 4.8.5.5.1 concurrent access timeout on org.jboss.invocation.InterceptorContext$Invocation@350ba25e - could not obtain lock within 5000MILLISECONDS > at org.jboss.as.ejb3.concurrency.ContainerManagedConcurrencyInterceptor.processInvocation(ContainerManagedConcurrencyInterceptor.java:100) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.system.SystemManagerLocal$$$view165.getProductInfo(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.SystemGWTServiceImpl.getProductInfo(SystemGWTServiceImpl.java:59) [classes:] > at sun.reflect.GeneratedMethodAccessor561.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) [gwt-servlet-2.5.0.jar:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) [classes:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) [classes:] > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) [jboss-as-web-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] > >08:00:28,960 WARN [org.rhq.coregui.server.gwt.SystemGWTServiceImpl] (http-/0.0.0.0:7080-37) Sending exception to client: [1396526428960] : javax.ejb.ConcurrentAccessTimeoutException: JBAS014373: EJB 3.1 PFD2 4.8.5.5.1 concurrent access timeout on org.jboss.invocation.InterceptorContext$Invocation@59f81917 - could not obtain lock within 5000MILLISECONDS > at org.jboss.as.ejb3.concurrency.ContainerManagedConcurrencyInterceptor.processInvocation(ContainerManagedConcurrencyInterceptor.java:100) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.system.SystemManagerLocal$$$view165.getUnmaskedSystemSettings(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.SystemGWTServiceImpl.getSessionTimeout(SystemGWTServiceImpl.java:77) [classes:] > at sun.reflect.GeneratedMethodAccessor560.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) [gwt-servlet-2.5.0.jar:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) [classes:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) [classes:] > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) [jboss-as-web-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] > >08:00:28,986 WARN [org.rhq.coregui.server.gwt.SystemGWTServiceImpl] (http-/0.0.0.0:7080-7) Sending exception to client: [1396526428986] : javax.ejb.ConcurrentAccessTimeoutException: JBAS014373: EJB 3.1 PFD2 4.8.5.5.1 concurrent access timeout on org.jboss.invocation.InterceptorContext$Invocation@67de85 - could not obtain lock within 5000MILLISECONDS > at org.jboss.as.ejb3.concurrency.ContainerManagedConcurrencyInterceptor.processInvocation(ContainerManagedConcurrencyInterceptor.java:100) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.system.SystemManagerLocal$$$view165.getProductInfo(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.SystemGWTServiceImpl.getProductInfo(SystemGWTServiceImpl.java:59) [classes:] > at sun.reflect.GeneratedMethodAccessor561.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) [gwt-servlet-2.5.0.jar:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) [classes:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) [classes:] > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) [jboss-as-web-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] > >08:00:29,121 INFO [org.rhq.enterprise.server.core.CoreServerServiceImpl] (http-/0.0.0.0:7080-30) Agent [dhcp180-58-agent-16163][4.9.0.JON320GA(734bd56)] would like to connect to this server >08:00:29,633 INFO [org.rhq.enterprise.server.core.CoreServerServiceImpl] (http-/0.0.0.0:7080-30) Agent [dhcp180-58-agent-16163] has connected to this server at Thu Apr 03 08:00:29 EDT 2014 >08:00:29,581 ERROR [org.jboss.as.ejb3.invocation] (http-/0.0.0.0:7080-50) JBAS014134: EJB Invocation failed on component SystemManagerBean for method public abstract org.rhq.core.domain.common.composite.SystemSettings org.rhq.enterprise.server.system.SystemManagerLocal.getUnmaskedSystemSettings(boolean): javax.ejb.ConcurrentAccessTimeoutException: JBAS014373: EJB 3.1 PFD2 4.8.5.5.1 concurrent access timeout on org.jboss.invocation.InterceptorContext$Invocation@b5aec6c - could not obtain lock within 5000MILLISECONDS > at org.jboss.as.ejb3.concurrency.ContainerManagedConcurrencyInterceptor.processInvocation(ContainerManagedConcurrencyInterceptor.java:100) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.system.SystemManagerLocal$$$view165.getUnmaskedSystemSettings(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.SystemGWTServiceImpl.getSessionTimeout(SystemGWTServiceImpl.java:77) > at sun.reflect.GeneratedMethodAccessor560.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] > >08:00:29,577 ERROR [org.jboss.as.ejb3.invocation] (http-/0.0.0.0:7080-34) JBAS014134: EJB Invocation failed on component SystemManagerBean for method public abstract org.rhq.core.domain.common.ProductInfo org.rhq.enterprise.server.system.SystemManagerRemote.getProductInfo(org.rhq.core.domain.auth.Subject): javax.ejb.ConcurrentAccessTimeoutException: JBAS014373: EJB 3.1 PFD2 4.8.5.5.1 concurrent access timeout on org.jboss.invocation.InterceptorContext$Invocation@fc013e8 - could not obtain lock within 5000MILLISECONDS > at org.jboss.as.ejb3.concurrency.ContainerManagedConcurrencyInterceptor.processInvocation(ContainerManagedConcurrencyInterceptor.java:100) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.system.SystemManagerLocal$$$view165.getProductInfo(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.SystemGWTServiceImpl.getProductInfo(SystemGWTServiceImpl.java:59) > at sun.reflect.GeneratedMethodAccessor561.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] > >08:00:29,742 WARN [org.rhq.coregui.server.gwt.SystemGWTServiceImpl] (http-/0.0.0.0:7080-50) Sending exception to client: [1396526429742] : javax.ejb.ConcurrentAccessTimeoutException: JBAS014373: EJB 3.1 PFD2 4.8.5.5.1 concurrent access timeout on org.jboss.invocation.InterceptorContext$Invocation@b5aec6c - could not obtain lock within 5000MILLISECONDS > at org.jboss.as.ejb3.concurrency.ContainerManagedConcurrencyInterceptor.processInvocation(ContainerManagedConcurrencyInterceptor.java:100) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.system.SystemManagerLocal$$$view165.getUnmaskedSystemSettings(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.SystemGWTServiceImpl.getSessionTimeout(SystemGWTServiceImpl.java:77) [classes:] > at sun.reflect.GeneratedMethodAccessor560.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) [gwt-servlet-2.5.0.jar:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) [classes:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) [classes:] > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) [jboss-as-web-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] > >08:00:29,750 WARN [org.rhq.coregui.server.gwt.SystemGWTServiceImpl] (http-/0.0.0.0:7080-34) Sending exception to client: [1396526429750] : javax.ejb.ConcurrentAccessTimeoutException: JBAS014373: EJB 3.1 PFD2 4.8.5.5.1 concurrent access timeout on org.jboss.invocation.InterceptorContext$Invocation@fc013e8 - could not obtain lock within 5000MILLISECONDS > at org.jboss.as.ejb3.concurrency.ContainerManagedConcurrencyInterceptor.processInvocation(ContainerManagedConcurrencyInterceptor.java:100) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.system.SystemManagerLocal$$$view165.getProductInfo(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.SystemGWTServiceImpl.getProductInfo(SystemGWTServiceImpl.java:59) [classes:] > at sun.reflect.GeneratedMethodAccessor561.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) [gwt-servlet-2.5.0.jar:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) [classes:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) [classes:] > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) [jboss-as-web-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] > >08:00:29,989 ERROR [org.jboss.as.ejb3] (EJB default - 4) JBAS014122: Error during retrying timeout for timer: [id=e1f92a5e-faf4-48d7-8e09-3cd7fd5f6f5d timedObjectId=rhq.rhq-server.SystemManagerBean auto-timer?:false persistent?:false timerService=org.jboss.as.ejb3.timerservice.TimerServiceImpl@64bdf1ae initialExpiration=Mon Mar 31 05:19:19 EDT 2014 intervalDuration(in milli sec)=60000 nextExpiration=Thu Apr 03 08:01:19 EDT 2014 timerState=RETRY_TIMEOUT: javax.ejb.ConcurrentAccessTimeoutException: JBAS014373: EJB 3.1 PFD2 4.8.5.5.1 concurrent access timeout on org.jboss.invocation.InterceptorContext$Invocation@47c7803d - could not obtain lock within 5000MILLISECONDS > at org.jboss.as.ejb3.concurrency.ContainerManagedConcurrencyInterceptor.processInvocation(ContainerManagedConcurrencyInterceptor.java:100) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.timerservice.TimedObjectInvokerImpl.callTimeout(TimedObjectInvokerImpl.java:101) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.timerservice.TimedObjectInvokerImpl.callTimeout(TimedObjectInvokerImpl.java:111) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.timerservice.task.TimerTask.callTimeout(TimerTask.java:157) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.timerservice.task.TimerTask.retryTimeout(TimerTask.java:184) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.timerservice.task.TimerTask.run(TimerTask.java:140) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439) [rt.jar:1.6.0_45] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) [rt.jar:1.6.0_45] > at java.util.concurrent.FutureTask.run(FutureTask.java:138) [rt.jar:1.6.0_45] > at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) [rt.jar:1.6.0_45] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) [rt.jar:1.6.0_45] > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] > at org.jboss.threads.JBossThread.run(JBossThread.java:122) [jboss-threads-2.1.0.Final-redhat-1.jar:2.1.0.Final-redhat-1] > >08:00:34,212 INFO [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/coregui]] (http-/0.0.0.0:7080-24) org.rhq.coregui.CoreGUI PackageVersionFileUploadServlet: file was uploaded: mock-ejb3.jar >08:00:34,763 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-5) Database maintenance completed in [19331]ms >08:00:34,764 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-5) Auto-calculation of baselines starting at Thu Apr 03 08:00:34 EDT 2014 >08:00:34,774 INFO [org.rhq.enterprise.server.measurement.MeasurementBaselineManagerBean] (RHQScheduler_Worker-5) Calculating auto baselines >08:00:34,775 INFO [org.rhq.enterprise.server.measurement.MeasurementBaselineManagerBean] (RHQScheduler_Worker-5) Deleting baselines computations older than Mon Mar 31 08:00:34 EDT 2014 >08:00:34,775 INFO [org.rhq.enterprise.server.measurement.MeasurementBaselineManagerBean] (RHQScheduler_Worker-5) Inserting new baselines using last 7 days of 1H data >08:00:34,784 INFO [org.rhq.enterprise.server.measurement.MeasurementBaselineManagerBean] (RHQScheduler_Worker-5) Removed [0] old baselines - they will now be recalculated (9)ms >08:00:41,478 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-12) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:00:56,637 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-12) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:00:59,791 INFO [org.rhq.enterprise.server.measurement.MeasurementBaselineManagerBean] (RHQScheduler_Worker-5) Calculated and inserted [4698] new baselines. (25007)ms >08:00:59,938 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-5) Auto-calculation of baselines completed in [25174]ms >08:00:59,939 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-5) Auto-calculation of OOBs starting >08:00:59,975 INFO [org.rhq.enterprise.server.measurement.MeasurementOOBManagerBean] (RHQScheduler_Worker-5) Removed [0] outdated OOBs >08:01:00,090 INFO [org.rhq.enterprise.server.measurement.MeasurementOOBManagerBean] (RHQScheduler_Worker-5) Computing OOBs >08:01:12,003 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-12) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:01:20,375 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-4) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:01:33,181 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 4) dhcp180-146.mw.lab.eng.bos.redhat.com took [12862]ms to reload cache for 21 agents >08:01:47,692 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-28) Received create resource response: CreateResourceResponse[RequestId=10043, Status=Success] >08:02:11,886 INFO [org.rhq.enterprise.server.measurement.MeasurementOOBManagerBean] (RHQScheduler_Worker-5) Finished calculating 86 OOBs in 71796 ms >08:02:11,886 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-5) Auto-calculation of OOBs completed in [71947]ms >08:02:11,886 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-5) Data Purge Job FINISHED [131794]ms >08:02:13,581 INFO [org.rhq.enterprise.server.scheduler.jobs.AsyncResourceDeleteJob] (RHQScheduler_Worker-1) Async resource deletion - 142 successful, 0 failed, took [173447] ms >08:02:14,144 WARN [com.arjuna.ats.arjuna] (Transaction Reaper) ARJUNA012117: TransactionReaper::check timeout for TX 0:ffff0a10b492:-289ef26c:533931c9:2132b5 in state RUN >08:02:14,146 WARN [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012095: Abort of action id 0:ffff0a10b492:-289ef26c:533931c9:2132b5 invoked while multiple threads active within it. >08:02:14,149 INFO [org.rhq.enterprise.server.common.TransactionInterruptInterceptor] (Transaction Reaper Worker 0) Transaction [0:ffff0a10b492:-289ef26c:533931c9:2132b5] is aborting with active thread [http-/0.0.0.0:7080-27]. interrupting=[false]: java.lang.Throwable: STACK TRACE OF ACTIVE THREAD IN TERMINATING TRANSACTION > at sun.misc.Unsafe.park(Native Method) [rt.jar:1.6.0_45] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:196) [rt.jar:1.6.0_45] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1011) [rt.jar:1.6.0_45] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.6.0_45] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:227) [rt.jar:1.6.0_45] > at java.util.concurrent.FutureTask.get(FutureTask.java:91) [rt.jar:1.6.0_45] > at org.rhq.enterprise.communications.command.client.ClientCommandSenderTask.run(ClientCommandSenderTask.java:143) > at org.rhq.enterprise.communications.command.client.ClientCommandSender.sendSynch(ClientCommandSender.java:647) > at org.rhq.enterprise.communications.command.client.ClientRemotePojoFactory$RemotePojoProxyHandler.invoke(ClientRemotePojoFactory.java:418) > at com.sun.proxy.$Proxy5869.createResource(Unknown Source) > at org.rhq.enterprise.server.resource.ResourceFactoryManagerBean.doCreatePackageBackedResource(ResourceFactoryManagerBean.java:644) > at org.rhq.enterprise.server.resource.ResourceFactoryManagerBean.createPackageBackedResourceViaPackageVersion(ResourceFactoryManagerBean.java:612) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.6.0_45] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) [rt.jar:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) > at org.rhq.enterprise.server.common.PerformanceMonitorInterceptor.monitorHibernatePerformance(PerformanceMonitorInterceptor.java:32) > at sun.reflect.GeneratedMethodAccessor93.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) > at org.rhq.enterprise.server.common.TransactionInterruptInterceptor.addCheckedActionToTransactionManager(TransactionInterruptInterceptor.java:78) > at sun.reflect.GeneratedMethodAccessor92.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) > at org.rhq.enterprise.server.authz.RequiredPermissionsInterceptor.checkRequiredPermissions(RequiredPermissionsInterceptor.java:156) > at sun.reflect.GeneratedMethodAccessor91.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.jpa.interceptor.SBInvocationInterceptor.processInvocation(SBInvocationInterceptor.java:47) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.pool.PooledInstanceInterceptor.processInvocation(PooledInstanceInterceptor.java:51) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.resource.ResourceFactoryManagerLocal$$$view153.createPackageBackedResourceViaPackageVersion(Unknown Source) > at org.rhq.coregui.server.gwt.ResourceGWTServiceImpl.createResource(ResourceGWTServiceImpl.java:341) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.6.0_45] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) [rt.jar:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] > >08:02:14,198 WARN [org.hibernate.engine.transaction.synchronization.internal.SynchronizationCallbackCoordinatorImpl] (Transaction Reaper Worker 0) Transaction afterCompletion called by a background thread! Delaying action until the original thread can handle it. [status=4] >08:02:14,198 WARN [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012121: TransactionReaper::doCancellations worker Thread[Transaction Reaper Worker 0,5,main] successfully canceled TX 0:ffff0a10b492:-289ef26c:533931c9:2132b5 >08:02:14,250 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-27) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[createResource], targetInterfaceName=org.rhq.core.clientapi.agent.inventory.ResourceFactoryAgentService}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:02:14,251 ERROR [org.rhq.enterprise.server.resource.ResourceFactoryManagerBean] (http-/0.0.0.0:7080-27) Error while sending create resource request to agent service: java.lang.reflect.UndeclaredThrowableException > at com.sun.proxy.$Proxy5869.createResource(Unknown Source) > at org.rhq.enterprise.server.resource.ResourceFactoryManagerBean.doCreatePackageBackedResource(ResourceFactoryManagerBean.java:644) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.enterprise.server.resource.ResourceFactoryManagerBean.createPackageBackedResourceViaPackageVersion(ResourceFactoryManagerBean.java:612) [rhq-server.jar:4.9.0.JON320GA] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.6.0_45] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) [rt.jar:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.rhq.enterprise.server.common.PerformanceMonitorInterceptor.monitorHibernatePerformance(PerformanceMonitorInterceptor.java:32) [rhq-server.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor93.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.rhq.enterprise.server.common.TransactionInterruptInterceptor.addCheckedActionToTransactionManager(TransactionInterruptInterceptor.java:78) [rhq-server.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor92.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.rhq.enterprise.server.authz.RequiredPermissionsInterceptor.checkRequiredPermissions(RequiredPermissionsInterceptor.java:156) [rhq-server.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor91.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.jpa.interceptor.SBInvocationInterceptor.processInvocation(SBInvocationInterceptor.java:47) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.pool.PooledInstanceInterceptor.processInvocation(PooledInstanceInterceptor.java:51) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.resource.ResourceFactoryManagerLocal$$$view153.createPackageBackedResourceViaPackageVersion(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.ResourceGWTServiceImpl.createResource(ResourceGWTServiceImpl.java:341) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.6.0_45] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) [rt.jar:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] >Caused by: java.util.concurrent.TimeoutException > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:228) [rt.jar:1.6.0_45] > at java.util.concurrent.FutureTask.get(FutureTask.java:91) [rt.jar:1.6.0_45] > at org.rhq.enterprise.communications.command.client.ClientCommandSenderTask.run(ClientCommandSenderTask.java:143) [rhq-enterprise-comm-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at org.rhq.enterprise.communications.command.client.ClientCommandSender.sendSynch(ClientCommandSender.java:647) [rhq-enterprise-comm-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at org.rhq.enterprise.communications.command.client.ClientRemotePojoFactory$RemotePojoProxyHandler.invoke(ClientRemotePojoFactory.java:418) [rhq-enterprise-comm-4.9.0.JON320GA.jar:4.9.0.JON320GA] > ... 139 more > >08:02:14,377 WARN [com.arjuna.ats.arjuna] (http-/0.0.0.0:7080-27) ARJUNA012077: Abort called on already aborted atomic action 0:ffff0a10b492:-289ef26c:533931c9:2132b5 >08:02:14,378 ERROR [org.jboss.as.ejb3.invocation] (http-/0.0.0.0:7080-27) JBAS014134: EJB Invocation failed on component ResourceFactoryManagerBean for method public abstract org.rhq.core.domain.resource.CreateResourceHistory org.rhq.enterprise.server.resource.ResourceFactoryManagerRemote.createPackageBackedResourceViaPackageVersion(org.rhq.core.domain.auth.Subject,int,int,java.lang.String,org.rhq.core.domain.configuration.Configuration,org.rhq.core.domain.configuration.Configuration,int,java.lang.Integer): javax.ejb.EJBTransactionRolledbackException: Transaction rolled back > at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleEndTransactionException(CMTTxInterceptor.java:114) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:94) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:252) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.resource.ResourceFactoryManagerLocal$$$view153.createPackageBackedResourceViaPackageVersion(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.ResourceGWTServiceImpl.createResource(ResourceGWTServiceImpl.java:341) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.6.0_45] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) [rt.jar:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] >Caused by: javax.transaction.RollbackException: ARJUNA016102: The transaction is not active! Uid is 0:ffff0a10b492:-289ef26c:533931c9:2132b5 > at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1155) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:75) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > ... 92 more > >08:02:14,418 WARN [org.rhq.coregui.server.gwt.ResourceGWTServiceImpl] (http-/0.0.0.0:7080-27) Sending exception to client: [1396526534418] : javax.ejb.EJBTransactionRolledbackException: Transaction rolled back > at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleEndTransactionException(CMTTxInterceptor.java:114) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:94) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:252) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.resource.ResourceFactoryManagerLocal$$$view153.createPackageBackedResourceViaPackageVersion(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.ResourceGWTServiceImpl.createResource(ResourceGWTServiceImpl.java:341) [classes:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.6.0_45] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) [rt.jar:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) [gwt-servlet-2.5.0.jar:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) [classes:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) [classes:] > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) [jboss-as-web-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] >Caused by: javax.transaction.RollbackException: ARJUNA016102: The transaction is not active! Uid is 0:ffff0a10b492:-289ef26c:533931c9:2132b5 > at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1155) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:75) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > ... 92 more > >08:02:20,463 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 8) dhcp180-146.mw.lab.eng.bos.redhat.com took [342]ms to reload cache for 1 agents >08:02:50,761 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 8) dhcp180-146.mw.lab.eng.bos.redhat.com took [663]ms to reload cache for 1 agents >08:03:09,701 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-35) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:03:20,133 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-1) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:03:31,055 INFO [org.rhq.enterprise.server.util.concurrent.InventoryReportSerializer] (http-/0.0.0.0:7080-19) tid=3591; agent=dev20-02-agent-16224: releasing write lock after being locked for millis=12699 >08:03:31,237 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 10) dhcp180-146.mw.lab.eng.bos.redhat.com took [627]ms to reload cache for 1 agents >08:03:39,608 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-74) Received create resource response: CreateResourceResponse[RequestId=10143, Status=Success] >08:03:40,769 INFO [org.rhq.enterprise.server.util.concurrent.InventoryReportSerializer] (http-/0.0.0.0:7080-19) tid=3591; agent=dev20-02-agent-16224: releasing write lock after being locked for millis=7595 >08:03:51,065 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 1) dhcp180-146.mw.lab.eng.bos.redhat.com took [930]ms to reload cache for 2 agents >08:04:06,924 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-80) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:04:11,042 INFO [org.rhq.enterprise.server.util.concurrent.InventoryReportSerializer] (http-/0.0.0.0:7080-42) tid=3712; agent=dhcp180-58-agent-16163: releasing write lock after being locked for millis=5884 >08:04:20,242 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-1) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:04:20,800 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 1) dhcp180-146.mw.lab.eng.bos.redhat.com took [601]ms to reload cache for 2 agents >08:04:22,968 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-95) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:04:35,962 INFO [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/coregui]] (http-/0.0.0.0:7080-12) org.rhq.coregui.CoreGUI PackageVersionFileUploadServlet: file was uploaded: mock-war.war >08:04:37,615 INFO [org.rhq.enterprise.server.scheduler.jobs.AsyncResourceDeleteJob] (RHQScheduler_Worker-5) Async resource deletion - 48 successful, 0 failed, took [17570] ms >08:04:37,935 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-78) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:04:47,423 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-53) Received create resource response: CreateResourceResponse[RequestId=10067, Status=Success] >08:04:53,017 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-78) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:04:58,598 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-2) Received create resource response: CreateResourceResponse[RequestId=10122, Status=Success] >08:05:03,918 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-22) Received create resource response: CreateResourceResponse[RequestId=10171, Status=Success] >08:05:08,305 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-82) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:05:20,165 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-5) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:05:21,063 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 5) dhcp180-146.mw.lab.eng.bos.redhat.com took [956]ms to reload cache for 3 agents >08:05:23,984 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-82) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:05:39,114 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-42) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:05:53,392 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-30) Received delete resource response: DeleteResourceResponse[RequestId=10013, Status=Success] >08:05:54,256 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-21) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:05:58,270 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-74) Received delete resource response: DeleteResourceResponse[RequestId=10141, Status=Success] >08:06:07,314 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-41) Received create resource response: CreateResourceResponse[RequestId=10014, Status=Success] >08:06:10,038 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-15) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:06:20,131 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-4) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:06:20,248 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 7) dhcp180-146.mw.lab.eng.bos.redhat.com took [119]ms to reload cache for 1 agents >08:06:25,158 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-29) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:06:40,756 INFO [org.rhq.enterprise.server.core.CoreServerServiceImpl] (http-/0.0.0.0:7080-45) Got agent registration request for new agent: dhcp95-226-agent-16163[10.16.180.44:16163][4.9.0.JON320GA(734bd56)] >08:06:40,860 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-81) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:06:40,913 INFO [org.rhq.enterprise.server.core.CoreServerServiceImpl] (http-/0.0.0.0:7080-54) Agent [dhcp95-226-agent-16163][4.9.0.JON320GA(734bd56)] would like to connect to this server >08:06:41,219 INFO [org.rhq.enterprise.server.core.CoreServerServiceImpl] (http-/0.0.0.0:7080-54) Agent [dhcp95-226-agent-16163] has connected to this server at Thu Apr 03 08:06:41 EDT 2014 >08:06:50,955 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 1) dhcp180-146.mw.lab.eng.bos.redhat.com took [849]ms to reload cache for 1 agents >08:06:56,012 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-19) Received delete resource response: DeleteResourceResponse[RequestId=10131, Status=Success] >08:06:56,051 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-85) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:07:04,314 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-35) Received create resource response: CreateResourceResponse[RequestId=10181, Status=Success] >08:07:11,621 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-45) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:07:20,225 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-5) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:07:20,391 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 4) dhcp180-146.mw.lab.eng.bos.redhat.com took [280]ms to reload cache for 1 agents >08:07:26,608 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-59) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:07:28,339 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-47) Received delete resource response: DeleteResourceResponse[RequestId=10151, Status=Success] >08:07:32,656 INFO [org.rhq.enterprise.server.core.CoreServerServiceImpl] (http-/0.0.0.0:7080-50) Got agent registration request for new agent: dev87-agent-16201[10.16.95.142:16201][4.9.0.JON320GA(734bd56)] >08:07:32,803 INFO [org.rhq.enterprise.server.core.CoreServerServiceImpl] (http-/0.0.0.0:7080-88) Agent [dev87-agent-16201][4.9.0.JON320GA(734bd56)] would like to connect to this server >08:07:32,928 INFO [org.rhq.enterprise.server.core.CoreServerServiceImpl] (http-/0.0.0.0:7080-88) Agent [dev87-agent-16201] has connected to this server at Thu Apr 03 08:07:32 EDT 2014 >08:07:42,343 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-4) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:07:46,587 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-37) Received create resource response: CreateResourceResponse[RequestId=10191, Status=Success] >08:07:50,489 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 3) dhcp180-146.mw.lab.eng.bos.redhat.com took [377]ms to reload cache for 1 agents >08:07:57,568 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-63) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:08:12,666 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-63) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:08:21,850 INFO [org.rhq.enterprise.server.util.concurrent.AvailabilityReportSerializer] (http-/0.0.0.0:7080-12) tid=3563; agent=dev117-agent-16219: releasing write lock after being locked for millis=6262 >08:08:21,978 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-5) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:08:22,895 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-46) Received delete resource response: DeleteResourceResponse[RequestId=10142, Status=Success] >08:08:23,604 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 10) dhcp180-146.mw.lab.eng.bos.redhat.com took [1947]ms to reload cache for 2 agents >08:08:28,608 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-61) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:08:43,495 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-31) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:08:48,400 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-13) Received create resource response: CreateResourceResponse[RequestId=10053, Status=Success] >08:08:59,232 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-21) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:09:14,343 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-21) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:09:20,176 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-1) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:09:20,580 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 1) dhcp180-146.mw.lab.eng.bos.redhat.com took [222]ms to reload cache for 1 agents >08:09:22,347 INFO [org.rhq.enterprise.server.scheduler.jobs.AsyncResourceDeleteJob] (RHQScheduler_Worker-3) Async resource deletion - 3 successful, 0 failed, took [2325] ms >08:09:29,945 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-46) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:09:45,098 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-4) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:09:47,330 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-22) Received create resource response: CreateResourceResponse[RequestId=10132, Status=Success] >08:09:57,205 ERROR [org.rhq.enterprise.server.alert.engine.internal.AlertConditionCacheCoordinator] (EJB default - 7) Could not find agent for scheduleId = 35558 >08:10:13,167 INFO [org.rhq.enterprise.server.core.CoreServerServiceImpl] (http-/0.0.0.0:7080-38) Got agent registration request for new agent: dev02-agent-16222[10.16.93.33:16222][4.9.0.JON320GA(734bd56)] >08:10:13,608 INFO [org.rhq.enterprise.server.core.CoreServerServiceImpl] (http-/0.0.0.0:7080-86) Agent [dev02-agent-16222][4.9.0.JON320GA(734bd56)] would like to connect to this server >08:10:14,279 INFO [org.rhq.enterprise.server.core.CoreServerServiceImpl] (http-/0.0.0.0:7080-86) Agent [dev02-agent-16222] has connected to this server at Thu Apr 03 08:10:14 EDT 2014 >08:10:18,518 INFO [org.rhq.enterprise.server.util.concurrent.InventoryReportSerializer] (http-/0.0.0.0:7080-79) tid=3949; agent=dhcp95-226-agent-16163: releasing write lock after being locked for millis=5933 >08:10:20,265 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-4) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:10:21,475 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 9) dhcp180-146.mw.lab.eng.bos.redhat.com took [1330]ms to reload cache for 2 agents >08:10:42,665 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-80) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:10:57,773 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-34) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:10:59,243 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-25) Received create resource response: CreateResourceResponse[RequestId=10201, Status=Success] >08:11:20,401 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-4) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:11:21,033 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 2) dhcp180-146.mw.lab.eng.bos.redhat.com took [894]ms to reload cache for 1 agents >08:11:59,000 INFO [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/coregui]] (http-/0.0.0.0:7080-70) org.rhq.coregui.CoreGUI PackageVersionFileUploadServlet: file was uploaded: mock-ejb2.jar >08:12:10,381 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-12) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:12:20,151 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-4) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:12:20,246 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 7) dhcp180-146.mw.lab.eng.bos.redhat.com took [123]ms to reload cache for 1 agents >08:12:21,431 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-56) Received create resource response: CreateResourceResponse[RequestId=10003, Status=Success] >08:12:25,666 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-12) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:12:26,643 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-14) Received delete resource response: DeleteResourceResponse[RequestId=10161, Status=Success] >08:12:49,658 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-18) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:12:50,487 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 9) dhcp180-146.mw.lab.eng.bos.redhat.com took [385]ms to reload cache for 1 agents >08:13:04,674 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-18) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:13:16,493 INFO [org.rhq.enterprise.server.core.CoreServerServiceImpl] (http-/0.0.0.0:7080-20) Got agent registration request for new agent: dev07-agent-16226[10.16.93.25:16226][4.9.0.JON320GA(734bd56)] >08:13:16,648 INFO [org.rhq.enterprise.server.core.CoreServerServiceImpl] (http-/0.0.0.0:7080-34) Agent [dev07-agent-16226][4.9.0.JON320GA(734bd56)] would like to connect to this server >08:13:16,931 INFO [org.rhq.enterprise.server.core.CoreServerServiceImpl] (http-/0.0.0.0:7080-34) Agent [dev07-agent-16226] has connected to this server at Thu Apr 03 08:13:16 EDT 2014 >08:13:21,199 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-75) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:13:21,683 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-3) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:13:34,687 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-75) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:13:38,158 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-53) Received create resource response: CreateResourceResponse[RequestId=10211, Status=Success] >08:13:47,620 WARN [org.rhq.helpers.rtfilter.filter.RtFilter] (http-/0.0.0.0:7080-76) Response time log '/qa/hudson_workspace/workspace/jon-32-server-install/dhcp180-146.mw.lab.eng.bos.redhat.com/jon-server-3.2.0.GA/logs/rt/10.16.180.234_coregui_rt.log' has exceeded maximum file size (5242880 bytes) - truncating it... >08:13:48,141 INFO [org.rhq.enterprise.server.util.concurrent.InventoryReportSerializer] (http-/0.0.0.0:7080-3) tid=243; agent=dhcp180-58-agent-16163: releasing write lock after being locked for millis=5472 >08:13:49,727 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-57) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:13:50,698 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 5) dhcp180-146.mw.lab.eng.bos.redhat.com took [582]ms to reload cache for 1 agents >08:13:54,525 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-36) Received delete resource response: DeleteResourceResponse[RequestId=10171, Status=Success] >08:13:58,207 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-84) Received delete resource response: DeleteResourceResponse[RequestId=10034, Status=Success] >08:14:03,717 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-36) Received delete resource response: DeleteResourceResponse[RequestId=10122, Status=Success] >08:14:04,742 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-57) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:14:09,542 INFO [org.rhq.enterprise.server.util.concurrent.InventoryReportSerializer] (http-/0.0.0.0:7080-3) tid=243; agent=dhcp180-58-agent-16163: releasing write lock after being locked for millis=8206 >08:14:15,519 INFO [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/coregui]] (http-/0.0.0.0:7080-53) org.rhq.coregui.CoreGUI PackageVersionFileUploadServlet: file was uploaded: mock-ejb3.jar >08:14:16,535 WARN [com.arjuna.ats.arjuna] (Transaction Reaper) ARJUNA012117: TransactionReaper::check timeout for TX 0:ffff0a10b492:-289ef26c:533931c9:23fdca in state RUN >08:14:16,536 WARN [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012095: Abort of action id 0:ffff0a10b492:-289ef26c:533931c9:23fdca invoked while multiple threads active within it. >08:14:16,607 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-1) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[deleteResource], targetInterfaceName=org.rhq.core.clientapi.agent.inventory.ResourceFactoryAgentService}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:14:16,538 INFO [org.rhq.enterprise.server.common.TransactionInterruptInterceptor] (Transaction Reaper Worker 0) Transaction [0:ffff0a10b492:-289ef26c:533931c9:23fdca] is aborting with active thread [http-/0.0.0.0:7080-1]. interrupting=[false]: java.lang.Throwable: STACK TRACE OF ACTIVE THREAD IN TERMINATING TRANSACTION > at sun.misc.Unsafe.park(Native Method) [rt.jar:1.6.0_45] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:196) [rt.jar:1.6.0_45] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1011) [rt.jar:1.6.0_45] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.6.0_45] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:227) [rt.jar:1.6.0_45] > at java.util.concurrent.FutureTask.get(FutureTask.java:91) [rt.jar:1.6.0_45] > at org.rhq.enterprise.communications.command.client.ClientCommandSenderTask.run(ClientCommandSenderTask.java:143) > at org.rhq.enterprise.communications.command.client.ClientCommandSender.sendSynch(ClientCommandSender.java:647) > at org.rhq.enterprise.communications.command.client.ClientRemotePojoFactory$RemotePojoProxyHandler.invoke(ClientRemotePojoFactory.java:418) > at com.sun.proxy.$Proxy5869.deleteResource(Unknown Source) > at org.rhq.enterprise.server.resource.ResourceFactoryManagerBean.deleteResource(ResourceFactoryManagerBean.java:707) > at org.rhq.enterprise.server.resource.ResourceFactoryManagerBean.deleteResources(ResourceFactoryManagerBean.java:679) > at sun.reflect.GeneratedMethodAccessor814.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) > at org.rhq.enterprise.server.common.PerformanceMonitorInterceptor.monitorHibernatePerformance(PerformanceMonitorInterceptor.java:32) > at sun.reflect.GeneratedMethodAccessor93.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) > at org.rhq.enterprise.server.common.TransactionInterruptInterceptor.addCheckedActionToTransactionManager(TransactionInterruptInterceptor.java:78) > at sun.reflect.GeneratedMethodAccessor92.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) > at org.rhq.enterprise.server.authz.RequiredPermissionsInterceptor.checkRequiredPermissions(RequiredPermissionsInterceptor.java:156) > at sun.reflect.GeneratedMethodAccessor91.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.jpa.interceptor.SBInvocationInterceptor.processInvocation(SBInvocationInterceptor.java:47) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.pool.PooledInstanceInterceptor.processInvocation(PooledInstanceInterceptor.java:51) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.resource.ResourceFactoryManagerLocal$$$view153.deleteResources(Unknown Source) > at org.rhq.coregui.server.gwt.ResourceGWTServiceImpl.deleteResources(ResourceGWTServiceImpl.java:354) > at sun.reflect.GeneratedMethodAccessor813.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] > >08:14:16,638 WARN [org.hibernate.engine.transaction.synchronization.internal.SynchronizationCallbackCoordinatorImpl] (Transaction Reaper Worker 0) Transaction afterCompletion called by a background thread! Delaying action until the original thread can handle it. [status=4] >08:14:16,638 WARN [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012121: TransactionReaper::doCancellations worker Thread[Transaction Reaper Worker 0,5,main] successfully canceled TX 0:ffff0a10b492:-289ef26c:533931c9:23fdca >08:14:16,607 ERROR [org.rhq.enterprise.server.resource.ResourceFactoryManagerBean] (http-/0.0.0.0:7080-1) Error while sending delete resource request to agent service: java.lang.reflect.UndeclaredThrowableException > at com.sun.proxy.$Proxy5869.deleteResource(Unknown Source) > at org.rhq.enterprise.server.resource.ResourceFactoryManagerBean.deleteResource(ResourceFactoryManagerBean.java:707) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.enterprise.server.resource.ResourceFactoryManagerBean.deleteResources(ResourceFactoryManagerBean.java:679) [rhq-server.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor814.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.rhq.enterprise.server.common.PerformanceMonitorInterceptor.monitorHibernatePerformance(PerformanceMonitorInterceptor.java:32) [rhq-server.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor93.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.rhq.enterprise.server.common.TransactionInterruptInterceptor.addCheckedActionToTransactionManager(TransactionInterruptInterceptor.java:78) [rhq-server.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor92.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.rhq.enterprise.server.authz.RequiredPermissionsInterceptor.checkRequiredPermissions(RequiredPermissionsInterceptor.java:156) [rhq-server.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor91.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.jpa.interceptor.SBInvocationInterceptor.processInvocation(SBInvocationInterceptor.java:47) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.pool.PooledInstanceInterceptor.processInvocation(PooledInstanceInterceptor.java:51) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.resource.ResourceFactoryManagerLocal$$$view153.deleteResources(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.ResourceGWTServiceImpl.deleteResources(ResourceGWTServiceImpl.java:354) > at sun.reflect.GeneratedMethodAccessor813.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] >Caused by: java.util.concurrent.TimeoutException > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:228) [rt.jar:1.6.0_45] > at java.util.concurrent.FutureTask.get(FutureTask.java:91) [rt.jar:1.6.0_45] > at org.rhq.enterprise.communications.command.client.ClientCommandSenderTask.run(ClientCommandSenderTask.java:143) [rhq-enterprise-comm-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at org.rhq.enterprise.communications.command.client.ClientCommandSender.sendSynch(ClientCommandSender.java:647) [rhq-enterprise-comm-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at org.rhq.enterprise.communications.command.client.ClientRemotePojoFactory$RemotePojoProxyHandler.invoke(ClientRemotePojoFactory.java:418) [rhq-enterprise-comm-4.9.0.JON320GA.jar:4.9.0.JON320GA] > ... 137 more > >08:14:16,728 WARN [com.arjuna.ats.arjuna] (http-/0.0.0.0:7080-1) ARJUNA012077: Abort called on already aborted atomic action 0:ffff0a10b492:-289ef26c:533931c9:23fdca >08:14:16,728 ERROR [org.jboss.as.ejb3.invocation] (http-/0.0.0.0:7080-1) JBAS014134: EJB Invocation failed on component ResourceFactoryManagerBean for method public abstract java.util.List org.rhq.enterprise.server.resource.ResourceFactoryManagerRemote.deleteResources(org.rhq.core.domain.auth.Subject,int[]): javax.ejb.EJBTransactionRolledbackException: Transaction rolled back > at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleEndTransactionException(CMTTxInterceptor.java:114) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:94) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:252) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.resource.ResourceFactoryManagerLocal$$$view153.deleteResources(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.ResourceGWTServiceImpl.deleteResources(ResourceGWTServiceImpl.java:354) > at sun.reflect.GeneratedMethodAccessor813.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] >Caused by: javax.transaction.RollbackException: ARJUNA016102: The transaction is not active! Uid is 0:ffff0a10b492:-289ef26c:533931c9:23fdca > at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1155) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:75) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > ... 91 more > >08:14:16,820 WARN [org.rhq.coregui.server.gwt.ResourceGWTServiceImpl] (http-/0.0.0.0:7080-1) Sending exception to client: [1396527256820] : javax.ejb.EJBTransactionRolledbackException: Transaction rolled back > at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleEndTransactionException(CMTTxInterceptor.java:114) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:94) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:252) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.resource.ResourceFactoryManagerLocal$$$view153.deleteResources(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.ResourceGWTServiceImpl.deleteResources(ResourceGWTServiceImpl.java:354) [classes:] > at sun.reflect.GeneratedMethodAccessor813.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) [gwt-servlet-2.5.0.jar:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) [classes:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) [classes:] > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) [jboss-as-web-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] >Caused by: javax.transaction.RollbackException: ARJUNA016102: The transaction is not active! Uid is 0:ffff0a10b492:-289ef26c:533931c9:23fdca > at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1155) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:75) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > ... 91 more > >08:14:19,648 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-89) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:14:20,300 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-5) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:14:21,896 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 8) dhcp180-146.mw.lab.eng.bos.redhat.com took [868]ms to reload cache for 3 agents >08:14:34,775 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-89) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:14:47,568 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-49) Received create resource response: CreateResourceResponse[RequestId=10212, Status=Success] >08:14:49,781 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-89) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:14:50,691 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 8) dhcp180-146.mw.lab.eng.bos.redhat.com took [567]ms to reload cache for 1 agents >08:15:04,772 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-89) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:15:19,791 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-89) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:15:20,170 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-2) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:15:21,662 INFO [org.rhq.enterprise.server.util.concurrent.InventoryReportSerializer] (http-/0.0.0.0:7080-71) tid=3741; agent=dhcp180-58-agent-16163: releasing write lock after being locked for millis=11804 >08:15:22,497 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 9) dhcp180-146.mw.lab.eng.bos.redhat.com took [1192]ms to reload cache for 2 agents >08:15:23,400 INFO [org.rhq.enterprise.server.util.concurrent.AvailabilityReportSerializer] (http-/0.0.0.0:7080-93) tid=4149; agent=dhcp180-58-agent-16163: releasing write lock after being locked for millis=7435 >08:15:33,177 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-14) Received delete resource response: DeleteResourceResponse[RequestId=10132, Status=Success] >08:15:34,767 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-89) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:15:40,224 INFO [org.rhq.enterprise.server.util.concurrent.InventoryReportSerializer] (http-/0.0.0.0:7080-71) tid=3741; agent=dhcp180-58-agent-16163: releasing write lock after being locked for millis=10496 >08:15:42,908 INFO [org.rhq.enterprise.server.scheduler.jobs.AsyncResourceDeleteJob] (RHQScheduler_Worker-3) Async resource deletion - 96 successful, 0 failed, took [82818] ms >08:15:49,704 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-79) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:15:50,249 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 4) dhcp180-146.mw.lab.eng.bos.redhat.com took [136]ms to reload cache for 1 agents >08:16:05,809 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-5) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:16:19,811 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-85) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:16:20,335 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-3) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:16:32,891 INFO [org.rhq.enterprise.server.util.concurrent.InventoryReportSerializer] (http-/0.0.0.0:7080-72) tid=3742; agent=dspvmg01-agent-16205: releasing write lock after being locked for millis=19249 >08:16:33,011 INFO [org.rhq.enterprise.server.util.concurrent.AvailabilityReportSerializer] (http-/0.0.0.0:7080-99) tid=4327; agent=dspvmg01-agent-16205: releasing write lock after being locked for millis=15595 >08:16:34,036 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 4) dhcp180-146.mw.lab.eng.bos.redhat.com took [1330]ms to reload cache for 2 agents >08:16:34,792 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-85) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:16:49,846 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-68) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:16:52,893 WARN [org.rhq.enterprise.server.discovery.DiscoveryServerServiceImpl] (http-/0.0.0.0:7080-7) Performance: inventory merge (33082)ms >08:16:52,893 INFO [org.rhq.enterprise.server.util.concurrent.InventoryReportSerializer] (http-/0.0.0.0:7080-7) tid=3477; agent=dev109-agent-16214: releasing write lock after being locked for millis=33083 >08:16:53,059 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 2) dhcp180-146.mw.lab.eng.bos.redhat.com took [1735]ms to reload cache for 2 agents >08:16:53,200 INFO [org.rhq.enterprise.server.util.concurrent.InventoryReportSerializer] (http-/0.0.0.0:7080-72) tid=3742; agent=dspvmg01-agent-16205: releasing write lock after being locked for millis=12721 >08:16:53,325 INFO [org.rhq.enterprise.server.util.concurrent.AvailabilityReportSerializer] (http-/0.0.0.0:7080-34) tid=3606; agent=dev109-agent-16214: releasing write lock after being locked for millis=13648 >08:17:04,948 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-19) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:17:08,385 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-11) Received create resource response: CreateResourceResponse[RequestId=10144, Status=Success] >08:17:11,380 INFO [org.rhq.enterprise.server.util.concurrent.InventoryReportSerializer] (http-/0.0.0.0:7080-34) tid=3606; agent=dev109-agent-16214: releasing write lock after being locked for millis=11349 >08:17:19,810 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-19) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:17:20,190 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-1) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:17:21,018 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 3) dhcp180-146.mw.lab.eng.bos.redhat.com took [903]ms to reload cache for 3 agents >08:17:34,870 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-19) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:17:51,386 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-103) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:17:57,047 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (RHQScheduler_Worker-3) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=2000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:17:57,047 ERROR [org.rhq.enterprise.server.measurement.MeasurementScheduleManagerBean] (RHQScheduler_Worker-3) Could not send measurement schedule updates to agent[id=10041], marking resources for update; agent ping failed >08:18:05,829 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-90) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:18:10,122 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-41) Received delete resource response: DeleteResourceResponse[RequestId=10181, Status=Success] >08:18:17,822 INFO [org.rhq.enterprise.server.util.concurrent.AvailabilityReportSerializer] (http-/0.0.0.0:7080-101) tid=4329; agent=dev07-agent-16226: releasing write lock after being locked for millis=5088 >08:18:17,938 INFO [org.rhq.enterprise.server.util.concurrent.InventoryReportSerializer] (http-/0.0.0.0:7080-65) tid=3735; agent=dev07-agent-16226: releasing write lock after being locked for millis=7547 >08:18:18,121 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-105) Received create resource response: CreateResourceResponse[RequestId=10172, Status=Success] >08:18:19,272 INFO [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/coregui]] (http-/0.0.0.0:7080-38) org.rhq.coregui.CoreGUI PackageVersionFileUploadServlet: file was uploaded: mock-war.war >08:18:20,428 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-4) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:18:20,945 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-4) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:18:21,169 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 2) dhcp180-146.mw.lab.eng.bos.redhat.com took [1021]ms to reload cache for 1 agents >08:18:35,751 INFO [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/coregui]] (http-/0.0.0.0:7080-51) org.rhq.coregui.CoreGUI PackageVersionFileUploadServlet: file was uploaded: mock-ejb2.jar >08:18:35,872 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-4) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:18:50,361 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 7) dhcp180-146.mw.lab.eng.bos.redhat.com took [240]ms to reload cache for 2 agents >08:18:50,800 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-26) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:18:57,261 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-25) Received delete resource response: DeleteResourceResponse[RequestId=10191, Status=Success] >08:18:58,043 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-92) Received create resource response: CreateResourceResponse[RequestId=10221, Status=Success] >08:19:05,875 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-26) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:19:20,339 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-3) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:19:20,896 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 5) dhcp180-146.mw.lab.eng.bos.redhat.com took [374]ms to reload cache for 1 agents >08:19:20,965 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-69) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:19:21,541 INFO [org.rhq.enterprise.server.scheduler.jobs.AsyncResourceDeleteJob] (RHQScheduler_Worker-2) Async resource deletion - 1 successful, 0 failed, took [1471] ms >08:19:29,662 WARN [org.rhq.enterprise.server.measurement.AvailabilityManagerBean] (http-/0.0.0.0:7080-41) Resource [Resource[id=13221, uuid=null, type=<null>, key=null, name=null, parent=<null>]] has multiple availabilities without an endtime [result returns more than one elements] - will attempt to remove the extra ones >AV:[dev20-02-agent-16224][49][full] >08:19:31,418 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-68) Received create resource response: CreateResourceResponse[RequestId=10182, Status=Success] >08:19:36,024 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-69) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:19:50,322 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 4) dhcp180-146.mw.lab.eng.bos.redhat.com took [218]ms to reload cache for 1 agents >08:19:50,954 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-69) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:20:01,203 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-101) Received delete resource response: DeleteResourceResponse[RequestId=10082, Status=Success] >08:20:05,989 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-41) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:20:20,447 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-2) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:20:21,017 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-41) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:20:27,376 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-99) Received delete resource response: DeleteResourceResponse[RequestId=10133, Status=Success] >08:20:35,998 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-41) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:20:49,443 INFO [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/coregui]] (http-/0.0.0.0:7080-98) org.rhq.coregui.CoreGUI PackageVersionFileUploadServlet: file was uploaded: mock-ejb3.jar >08:20:51,000 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-14) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:21:06,030 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-14) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:21:20,185 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-1) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:21:20,959 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-14) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:21:35,987 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-79) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:21:50,956 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-79) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:21:56,583 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-63) Received create resource response: CreateResourceResponse[RequestId=10145, Status=Success] >08:22:02,836 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-1) Received delete resource response: DeleteResourceResponse[RequestId=10182, Status=Success] >08:22:05,975 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-79) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:22:20,320 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-3) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:22:20,614 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 6) dhcp180-146.mw.lab.eng.bos.redhat.com took [510]ms to reload cache for 1 agents >08:22:21,116 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-31) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:22:26,756 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (RHQScheduler_Worker-5) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=2000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:22:26,756 ERROR [org.rhq.enterprise.server.measurement.MeasurementScheduleManagerBean] (RHQScheduler_Worker-5) Could not send measurement schedule updates to agent[id=10041], marking resources for update; agent ping failed >08:22:34,909 WARN [com.arjuna.ats.arjuna] (Transaction Reaper) ARJUNA012117: TransactionReaper::check timeout for TX 0:ffff0a10b492:-289ef26c:533931c9:25669a in state RUN >08:22:34,910 WARN [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012095: Abort of action id 0:ffff0a10b492:-289ef26c:533931c9:25669a invoked while multiple threads active within it. >08:22:34,930 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-86) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:22:34,911 INFO [org.rhq.enterprise.server.common.TransactionInterruptInterceptor] (Transaction Reaper Worker 0) Transaction [0:ffff0a10b492:-289ef26c:533931c9:25669a] is aborting with active thread [http-/0.0.0.0:7080-86]. interrupting=[false]: java.lang.Throwable: STACK TRACE OF ACTIVE THREAD IN TERMINATING TRANSACTION > at sun.misc.Unsafe.park(Native Method) [rt.jar:1.6.0_45] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:196) [rt.jar:1.6.0_45] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1011) [rt.jar:1.6.0_45] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.6.0_45] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:227) [rt.jar:1.6.0_45] > at java.util.concurrent.FutureTask.get(FutureTask.java:91) [rt.jar:1.6.0_45] > at org.rhq.enterprise.communications.command.client.ClientCommandSenderTask.run(ClientCommandSenderTask.java:143) > at org.rhq.enterprise.communications.command.client.ClientCommandSender.sendSynch(ClientCommandSender.java:647) > at org.rhq.enterprise.communications.command.client.ClientRemotePojoFactory$RemotePojoProxyHandler.invoke(ClientRemotePojoFactory.java:418) > at com.sun.proxy.$Proxy5944.getRealTimeMeasurementValue(Unknown Source) > at org.rhq.enterprise.server.measurement.MeasurementDataManagerBean.findLiveData(MeasurementDataManagerBean.java:892) > at org.rhq.enterprise.server.measurement.MeasurementDataManagerBean.findLiveData(MeasurementDataManagerBean.java:858) > at sun.reflect.GeneratedMethodAccessor757.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) > at org.rhq.enterprise.server.common.PerformanceMonitorInterceptor.monitorHibernatePerformance(PerformanceMonitorInterceptor.java:32) > at sun.reflect.GeneratedMethodAccessor93.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) > at org.rhq.enterprise.server.common.TransactionInterruptInterceptor.addCheckedActionToTransactionManager(TransactionInterruptInterceptor.java:78) > at sun.reflect.GeneratedMethodAccessor92.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) > at org.rhq.enterprise.server.authz.RequiredPermissionsInterceptor.checkRequiredPermissions(RequiredPermissionsInterceptor.java:156) > at sun.reflect.GeneratedMethodAccessor91.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.jpa.interceptor.SBInvocationInterceptor.processInvocation(SBInvocationInterceptor.java:47) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.pool.PooledInstanceInterceptor.processInvocation(PooledInstanceInterceptor.java:51) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.measurement.MeasurementDataManagerLocal$$$view123.findLiveData(Unknown Source) > at org.rhq.coregui.server.gwt.MeasurementDataGWTServiceImpl.findLiveData(MeasurementDataGWTServiceImpl.java:92) > at sun.reflect.GeneratedMethodAccessor756.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] > >08:22:35,074 WARN [com.arjuna.ats.arjuna] (http-/0.0.0.0:7080-86) ARJUNA012077: Abort called on already aborted atomic action 0:ffff0a10b492:-289ef26c:533931c9:25669a >08:22:35,078 WARN [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012121: TransactionReaper::doCancellations worker Thread[Transaction Reaper Worker 0,5,main] successfully canceled TX 0:ffff0a10b492:-289ef26c:533931c9:25669a >08:22:35,084 ERROR [org.jboss.as.ejb3.invocation] (http-/0.0.0.0:7080-86) JBAS014134: EJB Invocation failed on component MeasurementDataManagerBean for method public abstract java.util.Set org.rhq.enterprise.server.measurement.MeasurementDataManagerRemote.findLiveData(org.rhq.core.domain.auth.Subject,int,int[]): javax.ejb.EJBTransactionRolledbackException: Transaction rolled back > at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleEndTransactionException(CMTTxInterceptor.java:114) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:94) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:252) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.measurement.MeasurementDataManagerLocal$$$view123.findLiveData(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.MeasurementDataGWTServiceImpl.findLiveData(MeasurementDataGWTServiceImpl.java:92) > at sun.reflect.GeneratedMethodAccessor756.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] >Caused by: javax.transaction.RollbackException: ARJUNA016102: The transaction is not active! Uid is 0:ffff0a10b492:-289ef26c:533931c9:25669a > at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1155) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:75) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > ... 91 more > >08:22:35,165 WARN [org.rhq.coregui.server.gwt.MeasurementDataGWTServiceImpl] (http-/0.0.0.0:7080-86) Sending exception to client: [1396527755165] : javax.ejb.EJBTransactionRolledbackException: Transaction rolled back > at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleEndTransactionException(CMTTxInterceptor.java:114) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:94) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:252) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.measurement.MeasurementDataManagerLocal$$$view123.findLiveData(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.MeasurementDataGWTServiceImpl.findLiveData(MeasurementDataGWTServiceImpl.java:92) [classes:] > at sun.reflect.GeneratedMethodAccessor756.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) [gwt-servlet-2.5.0.jar:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) [classes:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) [classes:] > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) [jboss-as-web-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] >Caused by: javax.transaction.RollbackException: ARJUNA016102: The transaction is not active! Uid is 0:ffff0a10b492:-289ef26c:533931c9:25669a > at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1155) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:75) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > ... 91 more > >08:22:35,271 WARN [org.rhq.enterprise.server.measurement.AvailabilityManagerBean] (http-/0.0.0.0:7080-93) Resource [Resource[id=12854, uuid=null, type=<null>, key=null, name=null, parent=<null>]] has multiple availabilities without an endtime [result returns more than one elements] - will attempt to remove the extra ones >AV:[dev20-03-agent-16296][53][full] >08:22:37,184 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-31) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:22:50,239 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 10) dhcp180-146.mw.lab.eng.bos.redhat.com took [123]ms to reload cache for 1 agents >08:22:52,214 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-31) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:22:53,233 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-12) Received delete resource response: DeleteResourceResponse[RequestId=10201, Status=Success] >08:23:07,205 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-31) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:23:09,812 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-44) Received create resource response: CreateResourceResponse[RequestId=10004, Status=Success] >08:23:20,170 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-4) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:23:20,286 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 6) dhcp180-146.mw.lab.eng.bos.redhat.com took [176]ms to reload cache for 1 agents >08:23:22,192 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-31) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:23:36,464 WARN [com.arjuna.ats.arjuna] (Transaction Reaper) ARJUNA012117: TransactionReaper::check timeout for TX 0:ffff0a10b492:-289ef26c:533931c9:25895a in state RUN >08:23:36,477 WARN [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012095: Abort of action id 0:ffff0a10b492:-289ef26c:533931c9:25895a invoked while multiple threads active within it. >08:23:36,550 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-17) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:23:36,482 INFO [org.rhq.enterprise.server.common.TransactionInterruptInterceptor] (Transaction Reaper Worker 0) Transaction [0:ffff0a10b492:-289ef26c:533931c9:25895a] is aborting with active thread [http-/0.0.0.0:7080-17]. interrupting=[false]: java.lang.Throwable: STACK TRACE OF ACTIVE THREAD IN TERMINATING TRANSACTION > at sun.misc.Unsafe.park(Native Method) [rt.jar:1.6.0_45] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:196) [rt.jar:1.6.0_45] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1011) [rt.jar:1.6.0_45] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.6.0_45] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:227) [rt.jar:1.6.0_45] > at java.util.concurrent.FutureTask.get(FutureTask.java:91) [rt.jar:1.6.0_45] > at org.rhq.enterprise.communications.command.client.ClientCommandSenderTask.run(ClientCommandSenderTask.java:143) > at org.rhq.enterprise.communications.command.client.ClientCommandSender.sendSynch(ClientCommandSender.java:647) > at org.rhq.enterprise.communications.command.client.ClientRemotePojoFactory$RemotePojoProxyHandler.invoke(ClientRemotePojoFactory.java:418) > at com.sun.proxy.$Proxy5944.getRealTimeMeasurementValue(Unknown Source) > at org.rhq.enterprise.server.measurement.MeasurementDataManagerBean.findLiveData(MeasurementDataManagerBean.java:892) > at org.rhq.enterprise.server.measurement.MeasurementDataManagerBean.findLiveData(MeasurementDataManagerBean.java:858) > at sun.reflect.GeneratedMethodAccessor757.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) > at org.rhq.enterprise.server.common.PerformanceMonitorInterceptor.monitorHibernatePerformance(PerformanceMonitorInterceptor.java:32) > at sun.reflect.GeneratedMethodAccessor93.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) > at org.rhq.enterprise.server.common.TransactionInterruptInterceptor.addCheckedActionToTransactionManager(TransactionInterruptInterceptor.java:78) > at sun.reflect.GeneratedMethodAccessor92.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) > at org.rhq.enterprise.server.authz.RequiredPermissionsInterceptor.checkRequiredPermissions(RequiredPermissionsInterceptor.java:156) > at sun.reflect.GeneratedMethodAccessor91.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.jpa.interceptor.SBInvocationInterceptor.processInvocation(SBInvocationInterceptor.java:47) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.pool.PooledInstanceInterceptor.processInvocation(PooledInstanceInterceptor.java:51) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.measurement.MeasurementDataManagerLocal$$$view123.findLiveData(Unknown Source) > at org.rhq.coregui.server.gwt.MeasurementDataGWTServiceImpl.findLiveData(MeasurementDataGWTServiceImpl.java:92) > at sun.reflect.GeneratedMethodAccessor756.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] > >08:23:36,804 WARN [com.arjuna.ats.arjuna] (http-/0.0.0.0:7080-17) ARJUNA012077: Abort called on already aborted atomic action 0:ffff0a10b492:-289ef26c:533931c9:25895a >08:23:36,811 WARN [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012121: TransactionReaper::doCancellations worker Thread[Transaction Reaper Worker 0,5,main] successfully canceled TX 0:ffff0a10b492:-289ef26c:533931c9:25895a >08:23:36,816 ERROR [org.jboss.as.ejb3.invocation] (http-/0.0.0.0:7080-17) JBAS014134: EJB Invocation failed on component MeasurementDataManagerBean for method public abstract java.util.Set org.rhq.enterprise.server.measurement.MeasurementDataManagerRemote.findLiveData(org.rhq.core.domain.auth.Subject,int,int[]): javax.ejb.EJBTransactionRolledbackException: Transaction rolled back > at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleEndTransactionException(CMTTxInterceptor.java:114) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:94) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:252) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.measurement.MeasurementDataManagerLocal$$$view123.findLiveData(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.MeasurementDataGWTServiceImpl.findLiveData(MeasurementDataGWTServiceImpl.java:92) > at sun.reflect.GeneratedMethodAccessor756.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] >Caused by: javax.transaction.RollbackException: ARJUNA016102: The transaction is not active! Uid is 0:ffff0a10b492:-289ef26c:533931c9:25895a > at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1155) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:75) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > ... 91 more > >08:23:37,016 WARN [org.rhq.coregui.server.gwt.MeasurementDataGWTServiceImpl] (http-/0.0.0.0:7080-17) Sending exception to client: [1396527817016] : javax.ejb.EJBTransactionRolledbackException: Transaction rolled back > at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleEndTransactionException(CMTTxInterceptor.java:114) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:94) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:252) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.measurement.MeasurementDataManagerLocal$$$view123.findLiveData(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.MeasurementDataGWTServiceImpl.findLiveData(MeasurementDataGWTServiceImpl.java:92) [classes:] > at sun.reflect.GeneratedMethodAccessor756.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) [gwt-servlet-2.5.0.jar:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) [classes:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) [classes:] > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) [jboss-as-web-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] >Caused by: javax.transaction.RollbackException: ARJUNA016102: The transaction is not active! Uid is 0:ffff0a10b492:-289ef26c:533931c9:25895a > at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1155) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:75) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > ... 91 more > >08:23:37,180 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-7) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:23:43,030 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-60) Received create resource response: CreateResourceResponse[RequestId=10025, Status=Success] >08:23:50,391 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 5) dhcp180-146.mw.lab.eng.bos.redhat.com took [278]ms to reload cache for 2 agents >08:23:52,196 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-30) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:24:07,190 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-30) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:24:21,017 INFO [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/coregui]] (http-/0.0.0.0:7080-40) org.rhq.coregui.CoreGUI PackageVersionFileUploadServlet: file was uploaded: mock-ejb2.jar >08:24:21,325 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-5) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:24:22,242 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-30) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:24:23,745 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 4) dhcp180-146.mw.lab.eng.bos.redhat.com took [2468]ms to reload cache for 2 agents >08:24:27,056 INFO [org.rhq.enterprise.server.util.concurrent.AvailabilityReportSerializer] (http-/0.0.0.0:7080-94) tid=4150; agent=dhcp95-226-agent-16163: releasing write lock after being locked for millis=5034 >08:24:32,755 INFO [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/coregui]] (http-/0.0.0.0:7080-106) org.rhq.coregui.CoreGUI PackageVersionFileUploadServlet: file was uploaded: mock-war.war >08:24:35,506 WARN [com.arjuna.ats.arjuna] (Transaction Reaper) ARJUNA012117: TransactionReaper::check timeout for TX 0:ffff0a10b492:-289ef26c:533931c9:25d28c in state RUN >08:24:35,507 WARN [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012095: Abort of action id 0:ffff0a10b492:-289ef26c:533931c9:25d28c invoked while multiple threads active within it. >08:24:35,612 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-81) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:24:35,508 INFO [org.rhq.enterprise.server.common.TransactionInterruptInterceptor] (Transaction Reaper Worker 0) Transaction [0:ffff0a10b492:-289ef26c:533931c9:25d28c] is aborting with active thread [http-/0.0.0.0:7080-81]. interrupting=[false]: java.lang.Throwable: STACK TRACE OF ACTIVE THREAD IN TERMINATING TRANSACTION > at sun.misc.Unsafe.park(Native Method) [rt.jar:1.6.0_45] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:196) [rt.jar:1.6.0_45] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1011) [rt.jar:1.6.0_45] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.6.0_45] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:227) [rt.jar:1.6.0_45] > at java.util.concurrent.FutureTask.get(FutureTask.java:91) [rt.jar:1.6.0_45] > at org.rhq.enterprise.communications.command.client.ClientCommandSenderTask.run(ClientCommandSenderTask.java:143) > at org.rhq.enterprise.communications.command.client.ClientCommandSender.sendSynch(ClientCommandSender.java:647) > at org.rhq.enterprise.communications.command.client.ClientRemotePojoFactory$RemotePojoProxyHandler.invoke(ClientRemotePojoFactory.java:418) > at com.sun.proxy.$Proxy5944.getRealTimeMeasurementValue(Unknown Source) > at org.rhq.enterprise.server.measurement.MeasurementDataManagerBean.findLiveData(MeasurementDataManagerBean.java:892) > at org.rhq.enterprise.server.measurement.MeasurementDataManagerBean.findLiveData(MeasurementDataManagerBean.java:858) > at sun.reflect.GeneratedMethodAccessor757.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) > at org.rhq.enterprise.server.common.PerformanceMonitorInterceptor.monitorHibernatePerformance(PerformanceMonitorInterceptor.java:32) > at sun.reflect.GeneratedMethodAccessor93.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) > at org.rhq.enterprise.server.common.TransactionInterruptInterceptor.addCheckedActionToTransactionManager(TransactionInterruptInterceptor.java:78) > at sun.reflect.GeneratedMethodAccessor92.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) > at org.rhq.enterprise.server.authz.RequiredPermissionsInterceptor.checkRequiredPermissions(RequiredPermissionsInterceptor.java:156) > at sun.reflect.GeneratedMethodAccessor91.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.jpa.interceptor.SBInvocationInterceptor.processInvocation(SBInvocationInterceptor.java:47) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.pool.PooledInstanceInterceptor.processInvocation(PooledInstanceInterceptor.java:51) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.measurement.MeasurementDataManagerLocal$$$view123.findLiveData(Unknown Source) > at org.rhq.coregui.server.gwt.MeasurementDataGWTServiceImpl.findLiveData(MeasurementDataGWTServiceImpl.java:92) > at sun.reflect.GeneratedMethodAccessor756.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] > >08:24:35,706 WARN [com.arjuna.ats.arjuna] (http-/0.0.0.0:7080-81) ARJUNA012077: Abort called on already aborted atomic action 0:ffff0a10b492:-289ef26c:533931c9:25d28c >08:24:35,718 WARN [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012121: TransactionReaper::doCancellations worker Thread[Transaction Reaper Worker 0,5,main] successfully canceled TX 0:ffff0a10b492:-289ef26c:533931c9:25d28c >08:24:35,707 ERROR [org.jboss.as.ejb3.invocation] (http-/0.0.0.0:7080-81) JBAS014134: EJB Invocation failed on component MeasurementDataManagerBean for method public abstract java.util.Set org.rhq.enterprise.server.measurement.MeasurementDataManagerRemote.findLiveData(org.rhq.core.domain.auth.Subject,int,int[]): javax.ejb.EJBTransactionRolledbackException: Transaction rolled back > at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleEndTransactionException(CMTTxInterceptor.java:114) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:94) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:252) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.measurement.MeasurementDataManagerLocal$$$view123.findLiveData(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.MeasurementDataGWTServiceImpl.findLiveData(MeasurementDataGWTServiceImpl.java:92) > at sun.reflect.GeneratedMethodAccessor756.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] >Caused by: javax.transaction.RollbackException: ARJUNA016102: The transaction is not active! Uid is 0:ffff0a10b492:-289ef26c:533931c9:25d28c > at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1155) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:75) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > ... 91 more > >08:24:35,842 WARN [org.rhq.coregui.server.gwt.MeasurementDataGWTServiceImpl] (http-/0.0.0.0:7080-81) Sending exception to client: [1396527875842] : javax.ejb.EJBTransactionRolledbackException: Transaction rolled back > at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleEndTransactionException(CMTTxInterceptor.java:114) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:94) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:252) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.measurement.MeasurementDataManagerLocal$$$view123.findLiveData(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.MeasurementDataGWTServiceImpl.findLiveData(MeasurementDataGWTServiceImpl.java:92) [classes:] > at sun.reflect.GeneratedMethodAccessor756.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) [gwt-servlet-2.5.0.jar:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) [classes:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) [classes:] > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) [jboss-as-web-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] >Caused by: javax.transaction.RollbackException: ARJUNA016102: The transaction is not active! Uid is 0:ffff0a10b492:-289ef26c:533931c9:25d28c > at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1155) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:75) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > ... 91 more > >08:24:37,315 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-30) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:24:38,111 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-6) Received delete resource response: DeleteResourceResponse[RequestId=10094, Status=Success] >08:24:44,150 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-71) Received create resource response: CreateResourceResponse[RequestId=10231, Status=Success] >08:24:52,401 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-101) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:25:07,284 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-101) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:25:20,560 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-3) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:25:20,820 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 8) dhcp180-146.mw.lab.eng.bos.redhat.com took [692]ms to reload cache for 1 agents >08:25:22,261 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-101) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:25:26,672 INFO [org.rhq.enterprise.server.scheduler.jobs.AsyncResourceDeleteJob] (RHQScheduler_Worker-4) Async resource deletion - 49 successful, 0 failed, took [65778] ms >08:25:36,408 WARN [com.arjuna.ats.arjuna] (Transaction Reaper) ARJUNA012117: TransactionReaper::check timeout for TX 0:ffff0a10b492:-289ef26c:533931c9:2647b7 in state RUN >08:25:36,413 WARN [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012095: Abort of action id 0:ffff0a10b492:-289ef26c:533931c9:2647b7 invoked while multiple threads active within it. >08:25:36,486 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-37) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:25:36,414 INFO [org.rhq.enterprise.server.common.TransactionInterruptInterceptor] (Transaction Reaper Worker 0) Transaction [0:ffff0a10b492:-289ef26c:533931c9:2647b7] is aborting with active thread [http-/0.0.0.0:7080-37]. interrupting=[false]: java.lang.Throwable: STACK TRACE OF ACTIVE THREAD IN TERMINATING TRANSACTION > at sun.misc.Unsafe.park(Native Method) [rt.jar:1.6.0_45] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:196) [rt.jar:1.6.0_45] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1011) [rt.jar:1.6.0_45] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.6.0_45] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:227) [rt.jar:1.6.0_45] > at java.util.concurrent.FutureTask.get(FutureTask.java:91) [rt.jar:1.6.0_45] > at org.rhq.enterprise.communications.command.client.ClientCommandSenderTask.run(ClientCommandSenderTask.java:143) > at org.rhq.enterprise.communications.command.client.ClientCommandSender.sendSynch(ClientCommandSender.java:647) > at org.rhq.enterprise.communications.command.client.ClientRemotePojoFactory$RemotePojoProxyHandler.invoke(ClientRemotePojoFactory.java:418) > at com.sun.proxy.$Proxy5944.getRealTimeMeasurementValue(Unknown Source) > at org.rhq.enterprise.server.measurement.MeasurementDataManagerBean.findLiveData(MeasurementDataManagerBean.java:892) > at org.rhq.enterprise.server.measurement.MeasurementDataManagerBean.findLiveData(MeasurementDataManagerBean.java:858) > at sun.reflect.GeneratedMethodAccessor757.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) > at org.rhq.enterprise.server.common.PerformanceMonitorInterceptor.monitorHibernatePerformance(PerformanceMonitorInterceptor.java:32) > at sun.reflect.GeneratedMethodAccessor93.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) > at org.rhq.enterprise.server.common.TransactionInterruptInterceptor.addCheckedActionToTransactionManager(TransactionInterruptInterceptor.java:78) > at sun.reflect.GeneratedMethodAccessor92.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) > at org.rhq.enterprise.server.authz.RequiredPermissionsInterceptor.checkRequiredPermissions(RequiredPermissionsInterceptor.java:156) > at sun.reflect.GeneratedMethodAccessor91.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.jpa.interceptor.SBInvocationInterceptor.processInvocation(SBInvocationInterceptor.java:47) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.pool.PooledInstanceInterceptor.processInvocation(PooledInstanceInterceptor.java:51) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.measurement.MeasurementDataManagerLocal$$$view123.findLiveData(Unknown Source) > at org.rhq.coregui.server.gwt.MeasurementDataGWTServiceImpl.findLiveData(MeasurementDataGWTServiceImpl.java:92) > at sun.reflect.GeneratedMethodAccessor756.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] > >08:25:36,553 WARN [com.arjuna.ats.arjuna] (http-/0.0.0.0:7080-37) ARJUNA012077: Abort called on already aborted atomic action 0:ffff0a10b492:-289ef26c:533931c9:2647b7 >08:25:36,556 WARN [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012121: TransactionReaper::doCancellations worker Thread[Transaction Reaper Worker 0,5,main] successfully canceled TX 0:ffff0a10b492:-289ef26c:533931c9:2647b7 >08:25:36,554 ERROR [org.jboss.as.ejb3.invocation] (http-/0.0.0.0:7080-37) JBAS014134: EJB Invocation failed on component MeasurementDataManagerBean for method public abstract java.util.Set org.rhq.enterprise.server.measurement.MeasurementDataManagerRemote.findLiveData(org.rhq.core.domain.auth.Subject,int,int[]): javax.ejb.EJBTransactionRolledbackException: Transaction rolled back > at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleEndTransactionException(CMTTxInterceptor.java:114) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:94) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:252) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.measurement.MeasurementDataManagerLocal$$$view123.findLiveData(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.MeasurementDataGWTServiceImpl.findLiveData(MeasurementDataGWTServiceImpl.java:92) > at sun.reflect.GeneratedMethodAccessor756.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] >Caused by: javax.transaction.RollbackException: ARJUNA016102: The transaction is not active! Uid is 0:ffff0a10b492:-289ef26c:533931c9:2647b7 > at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1155) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:75) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > ... 91 more > >08:25:36,604 WARN [org.rhq.coregui.server.gwt.MeasurementDataGWTServiceImpl] (http-/0.0.0.0:7080-37) Sending exception to client: [1396527936604] : javax.ejb.EJBTransactionRolledbackException: Transaction rolled back > at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleEndTransactionException(CMTTxInterceptor.java:114) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:94) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:252) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.measurement.MeasurementDataManagerLocal$$$view123.findLiveData(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.MeasurementDataGWTServiceImpl.findLiveData(MeasurementDataGWTServiceImpl.java:92) [classes:] > at sun.reflect.GeneratedMethodAccessor756.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) [gwt-servlet-2.5.0.jar:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) [classes:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) [classes:] > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) [jboss-as-web-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] >Caused by: javax.transaction.RollbackException: ARJUNA016102: The transaction is not active! Uid is 0:ffff0a10b492:-289ef26c:533931c9:2647b7 > at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1155) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:75) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > ... 91 more > >08:25:37,190 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-6) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:25:52,206 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-6) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:25:55,614 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-96) Received create resource response: CreateResourceResponse[RequestId=10251, Status=Success] >08:26:07,215 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-6) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:26:08,577 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-67) Received delete resource response: DeleteResourceResponse[RequestId=10211, Status=Success] >08:26:20,167 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-1) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:26:20,244 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 2) dhcp180-146.mw.lab.eng.bos.redhat.com took [136]ms to reload cache for 1 agents >08:26:22,193 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-70) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:26:32,169 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-34) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >08:26:34,690 INFO [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/coregui]] (http-/0.0.0.0:7080-24) org.rhq.coregui.CoreGUI PackageVersionFileUploadServlet: file was uploaded: mock-ejb3.jar >08:26:37,235 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-34) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:26:38,148 WARN [com.arjuna.ats.arjuna] (Transaction Reaper) ARJUNA012117: TransactionReaper::check timeout for TX 0:ffff0a10b492:-289ef26c:533931c9:2688dd in state RUN >08:26:38,149 WARN [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012095: Abort of action id 0:ffff0a10b492:-289ef26c:533931c9:2688dd invoked while multiple threads active within it. >08:26:38,150 INFO [org.rhq.enterprise.server.common.TransactionInterruptInterceptor] (Transaction Reaper Worker 0) Transaction [0:ffff0a10b492:-289ef26c:533931c9:2688dd] is aborting with active thread [http-/0.0.0.0:7080-85]. interrupting=[false]: java.lang.Throwable: STACK TRACE OF ACTIVE THREAD IN TERMINATING TRANSACTION > at sun.misc.Unsafe.park(Native Method) [rt.jar:1.6.0_45] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:196) [rt.jar:1.6.0_45] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1011) [rt.jar:1.6.0_45] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.6.0_45] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:227) [rt.jar:1.6.0_45] > at java.util.concurrent.FutureTask.get(FutureTask.java:91) [rt.jar:1.6.0_45] > at org.rhq.enterprise.communications.command.client.ClientCommandSenderTask.run(ClientCommandSenderTask.java:143) > at org.rhq.enterprise.communications.command.client.ClientCommandSender.sendSynch(ClientCommandSender.java:647) > at org.rhq.enterprise.communications.command.client.ClientRemotePojoFactory$RemotePojoProxyHandler.invoke(ClientRemotePojoFactory.java:418) > at com.sun.proxy.$Proxy5944.getRealTimeMeasurementValue(Unknown Source) > at org.rhq.enterprise.server.measurement.MeasurementDataManagerBean.findLiveData(MeasurementDataManagerBean.java:892) > at org.rhq.enterprise.server.measurement.MeasurementDataManagerBean.findLiveData(MeasurementDataManagerBean.java:858) > at sun.reflect.GeneratedMethodAccessor757.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) > at org.rhq.enterprise.server.common.PerformanceMonitorInterceptor.monitorHibernatePerformance(PerformanceMonitorInterceptor.java:32) > at sun.reflect.GeneratedMethodAccessor93.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) > at org.rhq.enterprise.server.common.TransactionInterruptInterceptor.addCheckedActionToTransactionManager(TransactionInterruptInterceptor.java:78) > at sun.reflect.GeneratedMethodAccessor92.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) > at org.rhq.enterprise.server.authz.RequiredPermissionsInterceptor.checkRequiredPermissions(RequiredPermissionsInterceptor.java:156) > at sun.reflect.GeneratedMethodAccessor91.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.jpa.interceptor.SBInvocationInterceptor.processInvocation(SBInvocationInterceptor.java:47) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.pool.PooledInstanceInterceptor.processInvocation(PooledInstanceInterceptor.java:51) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.measurement.MeasurementDataManagerLocal$$$view123.findLiveData(Unknown Source) > at org.rhq.coregui.server.gwt.MeasurementDataGWTServiceImpl.findLiveData(MeasurementDataGWTServiceImpl.java:92) > at sun.reflect.GeneratedMethodAccessor756.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] > >08:26:38,232 WARN [org.hibernate.engine.transaction.synchronization.internal.SynchronizationCallbackCoordinatorImpl] (Transaction Reaper Worker 0) Transaction afterCompletion called by a background thread! Delaying action until the original thread can handle it. [status=4] >08:26:38,233 WARN [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012121: TransactionReaper::doCancellations worker Thread[Transaction Reaper Worker 0,5,main] successfully canceled TX 0:ffff0a10b492:-289ef26c:533931c9:2688dd >08:26:38,296 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-85) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:26:38,298 WARN [com.arjuna.ats.arjuna] (http-/0.0.0.0:7080-85) ARJUNA012077: Abort called on already aborted atomic action 0:ffff0a10b492:-289ef26c:533931c9:2688dd >08:26:38,298 ERROR [org.jboss.as.ejb3.invocation] (http-/0.0.0.0:7080-85) JBAS014134: EJB Invocation failed on component MeasurementDataManagerBean for method public abstract java.util.Set org.rhq.enterprise.server.measurement.MeasurementDataManagerRemote.findLiveData(org.rhq.core.domain.auth.Subject,int,int[]): javax.ejb.EJBTransactionRolledbackException: Transaction rolled back > at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleEndTransactionException(CMTTxInterceptor.java:114) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:94) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:252) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.measurement.MeasurementDataManagerLocal$$$view123.findLiveData(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.MeasurementDataGWTServiceImpl.findLiveData(MeasurementDataGWTServiceImpl.java:92) > at sun.reflect.GeneratedMethodAccessor756.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] >Caused by: javax.transaction.RollbackException: ARJUNA016102: The transaction is not active! Uid is 0:ffff0a10b492:-289ef26c:533931c9:2688dd > at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1155) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:75) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > ... 91 more > >08:26:38,319 WARN [org.rhq.coregui.server.gwt.MeasurementDataGWTServiceImpl] (http-/0.0.0.0:7080-85) Sending exception to client: [1396527998319] : javax.ejb.EJBTransactionRolledbackException: Transaction rolled back > at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleEndTransactionException(CMTTxInterceptor.java:114) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:94) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:252) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.measurement.MeasurementDataManagerLocal$$$view123.findLiveData(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.MeasurementDataGWTServiceImpl.findLiveData(MeasurementDataGWTServiceImpl.java:92) [classes:] > at sun.reflect.GeneratedMethodAccessor756.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) [gwt-servlet-2.5.0.jar:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) [classes:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) [classes:] > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) [jboss-as-web-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] >Caused by: javax.transaction.RollbackException: ARJUNA016102: The transaction is not active! Uid is 0:ffff0a10b492:-289ef26c:533931c9:2688dd > at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1155) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:75) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > ... 91 more > >08:26:41,203 WARN [com.arjuna.ats.arjuna] (Transaction Reaper) ARJUNA012117: TransactionReaper::check timeout for TX 0:ffff0a10b492:-289ef26c:533931c9:268b93 in state RUN >08:26:41,205 WARN [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012095: Abort of action id 0:ffff0a10b492:-289ef26c:533931c9:268b93 invoked while multiple threads active within it. >08:26:41,206 INFO [org.rhq.enterprise.server.common.TransactionInterruptInterceptor] (Transaction Reaper Worker 0) Transaction [0:ffff0a10b492:-289ef26c:533931c9:268b93] is aborting with active thread [http-/0.0.0.0:7080-61]. interrupting=[false]: java.lang.Throwable: STACK TRACE OF ACTIVE THREAD IN TERMINATING TRANSACTION > at sun.misc.Unsafe.park(Native Method) [rt.jar:1.6.0_45] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:196) [rt.jar:1.6.0_45] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1011) [rt.jar:1.6.0_45] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.6.0_45] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:227) [rt.jar:1.6.0_45] > at java.util.concurrent.FutureTask.get(FutureTask.java:91) [rt.jar:1.6.0_45] > at org.rhq.enterprise.communications.command.client.ClientCommandSenderTask.run(ClientCommandSenderTask.java:143) > at org.rhq.enterprise.communications.command.client.ClientCommandSender.sendSynch(ClientCommandSender.java:647) > at org.rhq.enterprise.communications.command.client.ClientRemotePojoFactory$RemotePojoProxyHandler.invoke(ClientRemotePojoFactory.java:418) > at com.sun.proxy.$Proxy5944.getRealTimeMeasurementValue(Unknown Source) > at org.rhq.enterprise.server.measurement.MeasurementDataManagerBean.findLiveData(MeasurementDataManagerBean.java:892) > at org.rhq.enterprise.server.measurement.MeasurementDataManagerBean.findLiveData(MeasurementDataManagerBean.java:858) > at sun.reflect.GeneratedMethodAccessor757.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) > at org.rhq.enterprise.server.common.PerformanceMonitorInterceptor.monitorHibernatePerformance(PerformanceMonitorInterceptor.java:32) > at sun.reflect.GeneratedMethodAccessor93.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) > at org.rhq.enterprise.server.common.TransactionInterruptInterceptor.addCheckedActionToTransactionManager(TransactionInterruptInterceptor.java:78) > at sun.reflect.GeneratedMethodAccessor92.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) > at org.rhq.enterprise.server.authz.RequiredPermissionsInterceptor.checkRequiredPermissions(RequiredPermissionsInterceptor.java:156) > at sun.reflect.GeneratedMethodAccessor91.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.jpa.interceptor.SBInvocationInterceptor.processInvocation(SBInvocationInterceptor.java:47) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.pool.PooledInstanceInterceptor.processInvocation(PooledInstanceInterceptor.java:51) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.measurement.MeasurementDataManagerLocal$$$view123.findLiveData(Unknown Source) > at org.rhq.coregui.server.gwt.MeasurementDataGWTServiceImpl.findLiveData(MeasurementDataGWTServiceImpl.java:92) > at sun.reflect.GeneratedMethodAccessor756.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] > >08:26:41,310 WARN [org.hibernate.engine.transaction.synchronization.internal.SynchronizationCallbackCoordinatorImpl] (Transaction Reaper Worker 0) Transaction afterCompletion called by a background thread! Delaying action until the original thread can handle it. [status=4] >08:26:41,312 WARN [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012121: TransactionReaper::doCancellations worker Thread[Transaction Reaper Worker 0,5,main] successfully canceled TX 0:ffff0a10b492:-289ef26c:533931c9:268b93 >08:26:41,318 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-61) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:26:41,320 WARN [com.arjuna.ats.arjuna] (http-/0.0.0.0:7080-61) ARJUNA012077: Abort called on already aborted atomic action 0:ffff0a10b492:-289ef26c:533931c9:268b93 >08:26:41,321 ERROR [org.jboss.as.ejb3.invocation] (http-/0.0.0.0:7080-61) JBAS014134: EJB Invocation failed on component MeasurementDataManagerBean for method public abstract java.util.Set org.rhq.enterprise.server.measurement.MeasurementDataManagerRemote.findLiveData(org.rhq.core.domain.auth.Subject,int,int[]): javax.ejb.EJBTransactionRolledbackException: Transaction rolled back > at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleEndTransactionException(CMTTxInterceptor.java:114) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:94) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:252) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.measurement.MeasurementDataManagerLocal$$$view123.findLiveData(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.MeasurementDataGWTServiceImpl.findLiveData(MeasurementDataGWTServiceImpl.java:92) > at sun.reflect.GeneratedMethodAccessor756.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] >Caused by: javax.transaction.RollbackException: ARJUNA016102: The transaction is not active! Uid is 0:ffff0a10b492:-289ef26c:533931c9:268b93 > at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1155) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:75) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > ... 91 more > >08:26:41,390 WARN [org.rhq.coregui.server.gwt.MeasurementDataGWTServiceImpl] (http-/0.0.0.0:7080-61) Sending exception to client: [1396528001390] : javax.ejb.EJBTransactionRolledbackException: Transaction rolled back > at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleEndTransactionException(CMTTxInterceptor.java:114) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:94) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:252) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.measurement.MeasurementDataManagerLocal$$$view123.findLiveData(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.MeasurementDataGWTServiceImpl.findLiveData(MeasurementDataGWTServiceImpl.java:92) [classes:] > at sun.reflect.GeneratedMethodAccessor756.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) [gwt-servlet-2.5.0.jar:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) [classes:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) [classes:] > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) [jboss-as-web-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] >Caused by: javax.transaction.RollbackException: ARJUNA016102: The transaction is not active! Uid is 0:ffff0a10b492:-289ef26c:533931c9:268b93 > at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1155) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:75) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > ... 91 more > >08:26:42,387 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-89) Received delete resource response: DeleteResourceResponse[RequestId=10172, Status=Success] >08:26:52,219 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-34) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:27:07,625 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-23) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:27:20,737 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-3) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:27:22,337 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-111) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:27:37,255 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-111) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:27:48,924 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-64) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >08:27:50,360 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 9) dhcp180-146.mw.lab.eng.bos.redhat.com took [260]ms to reload cache for 1 agents >08:27:52,278 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-104) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:28:00,172 WARN [com.arjuna.ats.arjuna] (Transaction Reaper) ARJUNA012117: TransactionReaper::check timeout for TX 0:ffff0a10b492:-289ef26c:533931c9:26c02b in state RUN >08:28:00,177 WARN [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012095: Abort of action id 0:ffff0a10b492:-289ef26c:533931c9:26c02b invoked while multiple threads active within it. >08:28:00,258 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-22) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:28:00,185 INFO [org.rhq.enterprise.server.common.TransactionInterruptInterceptor] (Transaction Reaper Worker 0) Transaction [0:ffff0a10b492:-289ef26c:533931c9:26c02b] is aborting with active thread [http-/0.0.0.0:7080-22]. interrupting=[false]: java.lang.Throwable: STACK TRACE OF ACTIVE THREAD IN TERMINATING TRANSACTION > at sun.misc.Unsafe.park(Native Method) [rt.jar:1.6.0_45] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:196) [rt.jar:1.6.0_45] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1011) [rt.jar:1.6.0_45] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.6.0_45] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:227) [rt.jar:1.6.0_45] > at java.util.concurrent.FutureTask.get(FutureTask.java:91) [rt.jar:1.6.0_45] > at org.rhq.enterprise.communications.command.client.ClientCommandSenderTask.run(ClientCommandSenderTask.java:143) > at org.rhq.enterprise.communications.command.client.ClientCommandSender.sendSynch(ClientCommandSender.java:647) > at org.rhq.enterprise.communications.command.client.ClientRemotePojoFactory$RemotePojoProxyHandler.invoke(ClientRemotePojoFactory.java:418) > at com.sun.proxy.$Proxy5944.getRealTimeMeasurementValue(Unknown Source) > at org.rhq.enterprise.server.measurement.MeasurementDataManagerBean.findLiveData(MeasurementDataManagerBean.java:892) > at org.rhq.enterprise.server.measurement.MeasurementDataManagerBean.findLiveData(MeasurementDataManagerBean.java:858) > at sun.reflect.GeneratedMethodAccessor757.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) > at org.rhq.enterprise.server.common.PerformanceMonitorInterceptor.monitorHibernatePerformance(PerformanceMonitorInterceptor.java:32) > at sun.reflect.GeneratedMethodAccessor93.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) > at org.rhq.enterprise.server.common.TransactionInterruptInterceptor.addCheckedActionToTransactionManager(TransactionInterruptInterceptor.java:78) > at sun.reflect.GeneratedMethodAccessor92.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) > at org.rhq.enterprise.server.authz.RequiredPermissionsInterceptor.checkRequiredPermissions(RequiredPermissionsInterceptor.java:156) > at sun.reflect.GeneratedMethodAccessor91.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.jpa.interceptor.SBInvocationInterceptor.processInvocation(SBInvocationInterceptor.java:47) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.pool.PooledInstanceInterceptor.processInvocation(PooledInstanceInterceptor.java:51) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.measurement.MeasurementDataManagerLocal$$$view123.findLiveData(Unknown Source) > at org.rhq.coregui.server.gwt.MeasurementDataGWTServiceImpl.findLiveData(MeasurementDataGWTServiceImpl.java:92) > at sun.reflect.GeneratedMethodAccessor756.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] > >08:28:00,322 WARN [com.arjuna.ats.arjuna] (http-/0.0.0.0:7080-22) ARJUNA012077: Abort called on already aborted atomic action 0:ffff0a10b492:-289ef26c:533931c9:26c02b >08:28:00,330 WARN [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012121: TransactionReaper::doCancellations worker Thread[Transaction Reaper Worker 0,5,main] successfully canceled TX 0:ffff0a10b492:-289ef26c:533931c9:26c02b >08:28:00,327 ERROR [org.jboss.as.ejb3.invocation] (http-/0.0.0.0:7080-22) JBAS014134: EJB Invocation failed on component MeasurementDataManagerBean for method public abstract java.util.Set org.rhq.enterprise.server.measurement.MeasurementDataManagerRemote.findLiveData(org.rhq.core.domain.auth.Subject,int,int[]): javax.ejb.EJBTransactionRolledbackException: Transaction rolled back > at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleEndTransactionException(CMTTxInterceptor.java:114) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:94) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:252) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.measurement.MeasurementDataManagerLocal$$$view123.findLiveData(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.MeasurementDataGWTServiceImpl.findLiveData(MeasurementDataGWTServiceImpl.java:92) > at sun.reflect.GeneratedMethodAccessor756.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] >Caused by: javax.transaction.RollbackException: ARJUNA016102: The transaction is not active! Uid is 0:ffff0a10b492:-289ef26c:533931c9:26c02b > at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1155) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:75) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > ... 91 more > >08:28:00,413 WARN [org.rhq.coregui.server.gwt.MeasurementDataGWTServiceImpl] (http-/0.0.0.0:7080-22) Sending exception to client: [1396528080413] : javax.ejb.EJBTransactionRolledbackException: Transaction rolled back > at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleEndTransactionException(CMTTxInterceptor.java:114) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:94) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:252) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.measurement.MeasurementDataManagerLocal$$$view123.findLiveData(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.MeasurementDataGWTServiceImpl.findLiveData(MeasurementDataGWTServiceImpl.java:92) [classes:] > at sun.reflect.GeneratedMethodAccessor756.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) [gwt-servlet-2.5.0.jar:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) [classes:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) [classes:] > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) [jboss-as-web-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] >Caused by: javax.transaction.RollbackException: ARJUNA016102: The transaction is not active! Uid is 0:ffff0a10b492:-289ef26c:533931c9:26c02b > at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1155) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:75) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > ... 91 more > >08:28:07,288 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-22) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:28:20,153 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-2) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:28:20,247 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 7) dhcp180-146.mw.lab.eng.bos.redhat.com took [128]ms to reload cache for 1 agents >08:28:22,245 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-86) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:28:24,664 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-94) Received delete resource response: DeleteResourceResponse[RequestId=10212, Status=Success] >08:28:27,069 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-23) Received create resource response: CreateResourceResponse[RequestId=10241, Status=Success] >08:28:27,913 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-34) Received create resource response: CreateResourceResponse[RequestId=10152, Status=Success] >08:28:37,313 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-86) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:28:51,654 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 3) dhcp180-146.mw.lab.eng.bos.redhat.com took [1543]ms to reload cache for 3 agents >08:28:51,750 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-31) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >08:28:52,290 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-86) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:28:55,858 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-7) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >08:28:55,969 INFO [org.rhq.enterprise.server.util.concurrent.InventoryReportSerializer] (http-/0.0.0.0:7080-72) tid=3742; agent=dhcp95-226-agent-16163: releasing write lock after being locked for millis=6984 >08:29:07,386 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-86) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:29:08,922 WARN [com.arjuna.ats.arjuna] (Transaction Reaper) ARJUNA012117: TransactionReaper::check timeout for TX 0:ffff0a10b492:-289ef26c:533931c9:26eddb in state RUN >08:29:08,924 WARN [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012095: Abort of action id 0:ffff0a10b492:-289ef26c:533931c9:26eddb invoked while multiple threads active within it. >08:29:08,974 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-26) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:29:08,925 INFO [org.rhq.enterprise.server.common.TransactionInterruptInterceptor] (Transaction Reaper Worker 0) Transaction [0:ffff0a10b492:-289ef26c:533931c9:26eddb] is aborting with active thread [http-/0.0.0.0:7080-26]. interrupting=[false]: java.lang.Throwable: STACK TRACE OF ACTIVE THREAD IN TERMINATING TRANSACTION > at sun.misc.Unsafe.park(Native Method) [rt.jar:1.6.0_45] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:196) [rt.jar:1.6.0_45] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1011) [rt.jar:1.6.0_45] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.6.0_45] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:227) [rt.jar:1.6.0_45] > at java.util.concurrent.FutureTask.get(FutureTask.java:91) [rt.jar:1.6.0_45] > at org.rhq.enterprise.communications.command.client.ClientCommandSenderTask.run(ClientCommandSenderTask.java:143) > at org.rhq.enterprise.communications.command.client.ClientCommandSender.sendSynch(ClientCommandSender.java:647) > at org.rhq.enterprise.communications.command.client.ClientRemotePojoFactory$RemotePojoProxyHandler.invoke(ClientRemotePojoFactory.java:418) > at com.sun.proxy.$Proxy5944.getRealTimeMeasurementValue(Unknown Source) > at org.rhq.enterprise.server.measurement.MeasurementDataManagerBean.findLiveData(MeasurementDataManagerBean.java:892) > at org.rhq.enterprise.server.measurement.MeasurementDataManagerBean.findLiveData(MeasurementDataManagerBean.java:858) > at sun.reflect.GeneratedMethodAccessor757.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) > at org.rhq.enterprise.server.common.PerformanceMonitorInterceptor.monitorHibernatePerformance(PerformanceMonitorInterceptor.java:32) > at sun.reflect.GeneratedMethodAccessor93.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) > at org.rhq.enterprise.server.common.TransactionInterruptInterceptor.addCheckedActionToTransactionManager(TransactionInterruptInterceptor.java:78) > at sun.reflect.GeneratedMethodAccessor92.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) > at org.rhq.enterprise.server.authz.RequiredPermissionsInterceptor.checkRequiredPermissions(RequiredPermissionsInterceptor.java:156) > at sun.reflect.GeneratedMethodAccessor91.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.jpa.interceptor.SBInvocationInterceptor.processInvocation(SBInvocationInterceptor.java:47) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.pool.PooledInstanceInterceptor.processInvocation(PooledInstanceInterceptor.java:51) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.measurement.MeasurementDataManagerLocal$$$view123.findLiveData(Unknown Source) > at org.rhq.coregui.server.gwt.MeasurementDataGWTServiceImpl.findLiveData(MeasurementDataGWTServiceImpl.java:92) > at sun.reflect.GeneratedMethodAccessor756.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] > >08:29:09,173 WARN [com.arjuna.ats.arjuna] (http-/0.0.0.0:7080-26) ARJUNA012077: Abort called on already aborted atomic action 0:ffff0a10b492:-289ef26c:533931c9:26eddb >08:29:09,176 WARN [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012121: TransactionReaper::doCancellations worker Thread[Transaction Reaper Worker 0,5,main] successfully canceled TX 0:ffff0a10b492:-289ef26c:533931c9:26eddb >08:29:09,178 ERROR [org.jboss.as.ejb3.invocation] (http-/0.0.0.0:7080-26) JBAS014134: EJB Invocation failed on component MeasurementDataManagerBean for method public abstract java.util.Set org.rhq.enterprise.server.measurement.MeasurementDataManagerRemote.findLiveData(org.rhq.core.domain.auth.Subject,int,int[]): javax.ejb.EJBTransactionRolledbackException: Transaction rolled back > at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleEndTransactionException(CMTTxInterceptor.java:114) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:94) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:252) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.measurement.MeasurementDataManagerLocal$$$view123.findLiveData(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.MeasurementDataGWTServiceImpl.findLiveData(MeasurementDataGWTServiceImpl.java:92) > at sun.reflect.GeneratedMethodAccessor756.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] >Caused by: javax.transaction.RollbackException: ARJUNA016102: The transaction is not active! Uid is 0:ffff0a10b492:-289ef26c:533931c9:26eddb > at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1155) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:75) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > ... 91 more > >08:29:09,364 WARN [org.rhq.coregui.server.gwt.MeasurementDataGWTServiceImpl] (http-/0.0.0.0:7080-26) Sending exception to client: [1396528149364] : javax.ejb.EJBTransactionRolledbackException: Transaction rolled back > at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleEndTransactionException(CMTTxInterceptor.java:114) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:94) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:252) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.measurement.MeasurementDataManagerLocal$$$view123.findLiveData(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.MeasurementDataGWTServiceImpl.findLiveData(MeasurementDataGWTServiceImpl.java:92) [classes:] > at sun.reflect.GeneratedMethodAccessor756.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) [gwt-servlet-2.5.0.jar:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) [classes:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) [classes:] > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) [jboss-as-web-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] >Caused by: javax.transaction.RollbackException: ARJUNA016102: The transaction is not active! Uid is 0:ffff0a10b492:-289ef26c:533931c9:26eddb > at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1155) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:75) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > ... 91 more > >08:29:13,644 INFO [org.rhq.enterprise.server.util.concurrent.InventoryReportSerializer] (http-/0.0.0.0:7080-108) tid=4338; agent=dev87-agent-16201: releasing write lock after being locked for millis=13523 >08:29:13,730 INFO [org.rhq.enterprise.server.util.concurrent.AvailabilityReportSerializer] (http-/0.0.0.0:7080-30) tid=3602; agent=dev87-agent-16201: releasing write lock after being locked for millis=12501 >08:29:13,865 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-55) Received create resource response: CreateResourceResponse[RequestId=10112, Status=Success] >08:29:20,383 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-5) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:29:21,793 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 7) dhcp180-146.mw.lab.eng.bos.redhat.com took [1154]ms to reload cache for 3 agents >08:29:22,242 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-86) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:29:26,497 INFO [org.rhq.enterprise.server.util.concurrent.InventoryReportSerializer] (http-/0.0.0.0:7080-108) tid=4338; agent=dev87-agent-16201: releasing write lock after being locked for millis=6845 >08:29:37,727 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-82) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:29:38,069 WARN [com.arjuna.ats.arjuna] (Transaction Reaper) ARJUNA012117: TransactionReaper::check timeout for TX 0:ffff0a10b492:-289ef26c:533931c9:27001e in state RUN >08:29:38,123 WARN [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012095: Abort of action id 0:ffff0a10b492:-289ef26c:533931c9:27001e invoked while multiple threads active within it. >08:29:38,139 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-42) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:29:38,591 WARN [com.arjuna.ats.arjuna] (Transaction Reaper) ARJUNA012117: TransactionReaper::check timeout for TX 0:ffff0a10b492:-289ef26c:533931c9:27001e in state CANCEL >08:29:38,163 INFO [org.rhq.enterprise.server.common.TransactionInterruptInterceptor] (Transaction Reaper Worker 0) Transaction [0:ffff0a10b492:-289ef26c:533931c9:27001e] is aborting with active thread [http-/0.0.0.0:7080-42]. interrupting=[false]: java.lang.Throwable: STACK TRACE OF ACTIVE THREAD IN TERMINATING TRANSACTION > at sun.misc.Unsafe.park(Native Method) [rt.jar:1.6.0_45] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:196) [rt.jar:1.6.0_45] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1011) [rt.jar:1.6.0_45] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.6.0_45] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:227) [rt.jar:1.6.0_45] > at java.util.concurrent.FutureTask.get(FutureTask.java:91) [rt.jar:1.6.0_45] > at org.rhq.enterprise.communications.command.client.ClientCommandSenderTask.run(ClientCommandSenderTask.java:143) > at org.rhq.enterprise.communications.command.client.ClientCommandSender.sendSynch(ClientCommandSender.java:647) > at org.rhq.enterprise.communications.command.client.ClientRemotePojoFactory$RemotePojoProxyHandler.invoke(ClientRemotePojoFactory.java:418) > at com.sun.proxy.$Proxy5944.getRealTimeMeasurementValue(Unknown Source) > at org.rhq.enterprise.server.measurement.MeasurementDataManagerBean.findLiveData(MeasurementDataManagerBean.java:892) > at org.rhq.enterprise.server.measurement.MeasurementDataManagerBean.findLiveData(MeasurementDataManagerBean.java:858) > at sun.reflect.GeneratedMethodAccessor757.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) > at org.rhq.enterprise.server.common.PerformanceMonitorInterceptor.monitorHibernatePerformance(PerformanceMonitorInterceptor.java:32) > at sun.reflect.GeneratedMethodAccessor93.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) > at org.rhq.enterprise.server.common.TransactionInterruptInterceptor.addCheckedActionToTransactionManager(TransactionInterruptInterceptor.java:78) > at sun.reflect.GeneratedMethodAccessor92.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) > at org.rhq.enterprise.server.authz.RequiredPermissionsInterceptor.checkRequiredPermissions(RequiredPermissionsInterceptor.java:156) > at sun.reflect.GeneratedMethodAccessor91.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.jpa.interceptor.SBInvocationInterceptor.processInvocation(SBInvocationInterceptor.java:47) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.pool.PooledInstanceInterceptor.processInvocation(PooledInstanceInterceptor.java:51) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.measurement.MeasurementDataManagerLocal$$$view123.findLiveData(Unknown Source) > at org.rhq.coregui.server.gwt.MeasurementDataGWTServiceImpl.findLiveData(MeasurementDataGWTServiceImpl.java:92) > at sun.reflect.GeneratedMethodAccessor756.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] > >08:29:39,035 WARN [com.arjuna.ats.arjuna] (http-/0.0.0.0:7080-42) ARJUNA012077: Abort called on already aborted atomic action 0:ffff0a10b492:-289ef26c:533931c9:27001e >08:29:39,045 ERROR [org.jboss.as.ejb3.invocation] (http-/0.0.0.0:7080-42) JBAS014134: EJB Invocation failed on component MeasurementDataManagerBean for method public abstract java.util.Set org.rhq.enterprise.server.measurement.MeasurementDataManagerRemote.findLiveData(org.rhq.core.domain.auth.Subject,int,int[]): javax.ejb.EJBTransactionRolledbackException: Transaction rolled back > at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleEndTransactionException(CMTTxInterceptor.java:114) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:94) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:252) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.measurement.MeasurementDataManagerLocal$$$view123.findLiveData(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.MeasurementDataGWTServiceImpl.findLiveData(MeasurementDataGWTServiceImpl.java:92) > at sun.reflect.GeneratedMethodAccessor756.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] >Caused by: javax.transaction.RollbackException: ARJUNA016102: The transaction is not active! Uid is 0:ffff0a10b492:-289ef26c:533931c9:27001e > at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1155) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:75) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > ... 91 more > >08:29:39,459 WARN [org.rhq.coregui.server.gwt.MeasurementDataGWTServiceImpl] (http-/0.0.0.0:7080-42) Sending exception to client: [1396528179459] : javax.ejb.EJBTransactionRolledbackException: Transaction rolled back > at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleEndTransactionException(CMTTxInterceptor.java:114) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:94) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:252) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.measurement.MeasurementDataManagerLocal$$$view123.findLiveData(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.MeasurementDataGWTServiceImpl.findLiveData(MeasurementDataGWTServiceImpl.java:92) [classes:] > at sun.reflect.GeneratedMethodAccessor756.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) [gwt-servlet-2.5.0.jar:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) [classes:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) [classes:] > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) [jboss-as-web-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] >Caused by: javax.transaction.RollbackException: ARJUNA016102: The transaction is not active! Uid is 0:ffff0a10b492:-289ef26c:533931c9:27001e > at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1155) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:75) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > ... 91 more > >08:29:48,289 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-80) Received create resource response: CreateResourceResponse[RequestId=10261, Status=Success] >08:29:51,062 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 6) dhcp180-146.mw.lab.eng.bos.redhat.com took [897]ms to reload cache for 1 agents >08:29:51,846 INFO [org.rhq.enterprise.server.util.concurrent.AvailabilityReportSerializer] (http-/0.0.0.0:7080-72) tid=3742; agent=dev87-agent-16201: releasing write lock after being locked for millis=6523 >08:29:52,390 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-59) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:29:56,373 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-54) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >08:29:57,015 INFO [org.rhq.enterprise.server.util.concurrent.InventoryReportSerializer] (http-/0.0.0.0:7080-80) tid=3950; agent=dev117-agent-16219: releasing write lock after being locked for millis=5079 >08:30:07,491 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-54) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:30:20,202 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-32) Received create resource response: CreateResourceResponse[RequestId=10044, Status=Success] >08:30:20,908 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-4) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:30:21,145 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 9) dhcp180-146.mw.lab.eng.bos.redhat.com took [757]ms to reload cache for 1 agents >08:30:22,322 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-54) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:30:36,932 WARN [com.arjuna.ats.arjuna] (Transaction Reaper) ARJUNA012117: TransactionReaper::check timeout for TX 0:ffff0a10b492:-289ef26c:533931c9:272492 in state RUN >08:30:36,937 WARN [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012095: Abort of action id 0:ffff0a10b492:-289ef26c:533931c9:272492 invoked while multiple threads active within it. >08:30:37,098 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-2) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:30:36,938 INFO [org.rhq.enterprise.server.common.TransactionInterruptInterceptor] (Transaction Reaper Worker 0) Transaction [0:ffff0a10b492:-289ef26c:533931c9:272492] is aborting with active thread [http-/0.0.0.0:7080-2]. interrupting=[false]: java.lang.Throwable: STACK TRACE OF ACTIVE THREAD IN TERMINATING TRANSACTION > at sun.misc.Unsafe.park(Native Method) [rt.jar:1.6.0_45] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:196) [rt.jar:1.6.0_45] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1011) [rt.jar:1.6.0_45] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1303) [rt.jar:1.6.0_45] > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:227) [rt.jar:1.6.0_45] > at java.util.concurrent.FutureTask.get(FutureTask.java:91) [rt.jar:1.6.0_45] > at org.rhq.enterprise.communications.command.client.ClientCommandSenderTask.run(ClientCommandSenderTask.java:143) > at org.rhq.enterprise.communications.command.client.ClientCommandSender.sendSynch(ClientCommandSender.java:647) > at org.rhq.enterprise.communications.command.client.ClientRemotePojoFactory$RemotePojoProxyHandler.invoke(ClientRemotePojoFactory.java:418) > at com.sun.proxy.$Proxy5944.getRealTimeMeasurementValue(Unknown Source) > at org.rhq.enterprise.server.measurement.MeasurementDataManagerBean.findLiveData(MeasurementDataManagerBean.java:892) > at org.rhq.enterprise.server.measurement.MeasurementDataManagerBean.findLiveData(MeasurementDataManagerBean.java:858) > at sun.reflect.GeneratedMethodAccessor757.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) > at org.rhq.enterprise.server.common.PerformanceMonitorInterceptor.monitorHibernatePerformance(PerformanceMonitorInterceptor.java:32) > at sun.reflect.GeneratedMethodAccessor93.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) > at org.rhq.enterprise.server.common.TransactionInterruptInterceptor.addCheckedActionToTransactionManager(TransactionInterruptInterceptor.java:78) > at sun.reflect.GeneratedMethodAccessor92.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) > at org.rhq.enterprise.server.authz.RequiredPermissionsInterceptor.checkRequiredPermissions(RequiredPermissionsInterceptor.java:156) > at sun.reflect.GeneratedMethodAccessor91.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.jpa.interceptor.SBInvocationInterceptor.processInvocation(SBInvocationInterceptor.java:47) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.pool.PooledInstanceInterceptor.processInvocation(PooledInstanceInterceptor.java:51) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.measurement.MeasurementDataManagerLocal$$$view123.findLiveData(Unknown Source) > at org.rhq.coregui.server.gwt.MeasurementDataGWTServiceImpl.findLiveData(MeasurementDataGWTServiceImpl.java:92) > at sun.reflect.GeneratedMethodAccessor756.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] > >08:30:37,125 WARN [com.arjuna.ats.arjuna] (http-/0.0.0.0:7080-2) ARJUNA012077: Abort called on already aborted atomic action 0:ffff0a10b492:-289ef26c:533931c9:272492 >08:30:37,138 WARN [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012121: TransactionReaper::doCancellations worker Thread[Transaction Reaper Worker 0,5,main] successfully canceled TX 0:ffff0a10b492:-289ef26c:533931c9:272492 >08:30:37,129 ERROR [org.jboss.as.ejb3.invocation] (http-/0.0.0.0:7080-2) JBAS014134: EJB Invocation failed on component MeasurementDataManagerBean for method public abstract java.util.Set org.rhq.enterprise.server.measurement.MeasurementDataManagerRemote.findLiveData(org.rhq.core.domain.auth.Subject,int,int[]): javax.ejb.EJBTransactionRolledbackException: Transaction rolled back > at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleEndTransactionException(CMTTxInterceptor.java:114) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:94) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:252) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.measurement.MeasurementDataManagerLocal$$$view123.findLiveData(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.MeasurementDataGWTServiceImpl.findLiveData(MeasurementDataGWTServiceImpl.java:92) > at sun.reflect.GeneratedMethodAccessor756.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] >Caused by: javax.transaction.RollbackException: ARJUNA016102: The transaction is not active! Uid is 0:ffff0a10b492:-289ef26c:533931c9:272492 > at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1155) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:75) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > ... 91 more > >08:30:37,232 WARN [org.rhq.coregui.server.gwt.MeasurementDataGWTServiceImpl] (http-/0.0.0.0:7080-2) Sending exception to client: [1396528237232] : javax.ejb.EJBTransactionRolledbackException: Transaction rolled back > at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleEndTransactionException(CMTTxInterceptor.java:114) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:94) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:252) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.measurement.MeasurementDataManagerLocal$$$view123.findLiveData(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.MeasurementDataGWTServiceImpl.findLiveData(MeasurementDataGWTServiceImpl.java:92) [classes:] > at sun.reflect.GeneratedMethodAccessor756.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) [gwt-servlet-2.5.0.jar:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) [classes:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) [classes:] > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) [jboss-as-web-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] >Caused by: javax.transaction.RollbackException: ARJUNA016102: The transaction is not active! Uid is 0:ffff0a10b492:-289ef26c:533931c9:272492 > at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1155) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) [jbossjts-jacorb-4.17.7.Final-redhat-4.jar:4.17.7.Final-redhat-4] > at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:75) > at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > ... 91 more > >08:30:37,332 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-23) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:30:44,104 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-26) Received create resource response: CreateResourceResponse[RequestId=10005, Status=Success] >08:30:50,835 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 6) dhcp180-146.mw.lab.eng.bos.redhat.com took [723]ms to reload cache for 2 agents >08:30:52,368 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-2) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:30:58,306 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-58) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >08:31:07,323 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-58) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:31:08,749 INFO [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/coregui]] (http-/0.0.0.0:7080-96) org.rhq.coregui.CoreGUI PackageVersionFileUploadServlet: file was uploaded: mock-war.war >08:31:20,531 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-2) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:31:22,353 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-58) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:31:36,771 INFO [org.rhq.enterprise.server.scheduler.jobs.AsyncResourceDeleteJob] (RHQScheduler_Worker-3) Async resource deletion - 97 successful, 0 failed, took [136592] ms >08:31:37,443 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-14) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:31:51,124 INFO [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/coregui]] (http-/0.0.0.0:7080-96) org.rhq.coregui.CoreGUI PackageVersionFileUploadServlet: file was uploaded: mock-ejb2.jar >08:31:52,342 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-88) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:31:52,695 INFO [org.rhq.enterprise.server.util.concurrent.InventoryReportSerializer] (http-/0.0.0.0:7080-1) tid=229; agent=dhcp180-58-agent-16163: releasing write lock after being locked for millis=9477 >08:31:52,793 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 8) dhcp180-146.mw.lab.eng.bos.redhat.com took [259]ms to reload cache for 1 agents >08:31:57,168 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-5) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >08:32:05,724 INFO [org.rhq.enterprise.server.util.concurrent.InventoryReportSerializer] (http-/0.0.0.0:7080-53) tid=3723; agent=dhcp180-58-agent-16163: releasing write lock after being locked for millis=5775 >08:32:07,356 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-88) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:32:17,409 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-54) Received create resource response: CreateResourceResponse[RequestId=10045, Status=Success] >08:32:20,232 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-5) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:32:20,522 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 9) dhcp180-146.mw.lab.eng.bos.redhat.com took [400]ms to reload cache for 1 agents >08:32:24,247 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-88) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:32:37,333 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-88) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:32:50,370 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 3) dhcp180-146.mw.lab.eng.bos.redhat.com took [256]ms to reload cache for 1 agents >08:32:52,319 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-85) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:32:53,471 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-3) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >08:32:56,658 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-45) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >08:33:07,312 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-85) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:33:13,718 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-94) Received delete resource response: DeleteResourceResponse[RequestId=10162, Status=Success] >08:33:20,202 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-2) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:33:21,930 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-5) Received create resource response: CreateResourceResponse[RequestId=10033, Status=Success] >08:33:22,365 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-85) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:33:30,435 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-30) Received create resource response: CreateResourceResponse[RequestId=10173, Status=Success] >08:33:37,398 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-85) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:33:50,845 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 10) dhcp180-146.mw.lab.eng.bos.redhat.com took [736]ms to reload cache for 2 agents >08:33:52,416 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-96) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:33:54,807 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-11) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >08:34:04,989 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-25) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=YZW3MuIdz8NBK8Ah9EI4Bx5VuyXcU0b6yyoKP5PUhfkyELpWBQzn/yE8ZwLs2+EGhO4=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:34:07,381 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-11) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:34:10,797 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-58) Received delete resource response: DeleteResourceResponse[RequestId=10221, Status=Success] >08:34:14,042 INFO [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/coregui]] (http-/0.0.0.0:7080-7) org.rhq.coregui.CoreGUI PackageVersionFileUploadServlet: file was uploaded: mock-ejb3.jar >08:34:20,450 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-4) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:34:22,384 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-58) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:34:31,843 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-93) Received create resource response: CreateResourceResponse[RequestId=10202, Status=Success] >08:34:37,515 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-58) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:34:43,444 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-60) Received delete resource response: DeleteResourceResponse[RequestId=10083, Status=Success] >08:34:51,098 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 7) dhcp180-146.mw.lab.eng.bos.redhat.com took [964]ms to reload cache for 2 agents >08:34:52,494 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-107) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:34:56,611 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-73) Received create resource response: CreateResourceResponse[RequestId=10162, Status=Success] >08:34:57,569 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-12) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >08:35:07,490 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-12) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:35:11,606 INFO [org.rhq.enterprise.server.util.concurrent.InventoryReportSerializer] (http-/0.0.0.0:7080-9) tid=3512; agent=dev87-agent-16201: releasing write lock after being locked for millis=22292 >08:35:11,843 INFO [org.rhq.enterprise.server.util.concurrent.AvailabilityReportSerializer] (http-/0.0.0.0:7080-90) tid=4146; agent=dev87-agent-16201: releasing write lock after being locked for millis=19729 >08:35:20,310 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-1) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:35:22,510 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-12) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:35:22,565 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 7) dhcp180-146.mw.lab.eng.bos.redhat.com took [2425]ms to reload cache for 3 agents >08:35:31,104 INFO [org.rhq.enterprise.server.util.concurrent.InventoryReportSerializer] (http-/0.0.0.0:7080-9) tid=3512; agent=dev87-agent-16201: releasing write lock after being locked for millis=12640 >08:35:37,508 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-12) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:35:46,495 INFO [org.rhq.enterprise.server.util.concurrent.InventoryReportSerializer] (http-/0.0.0.0:7080-29) tid=3601; agent=dev07-agent-16226: releasing write lock after being locked for millis=23721 >08:35:46,856 WARN [org.rhq.enterprise.server.discovery.DiscoveryServerServiceImpl] (http-/0.0.0.0:7080-76) Performance: processed AV:[dev07-agent-16226][24][full] - needFull=[false] in (22095)ms >08:35:46,856 INFO [org.rhq.enterprise.server.util.concurrent.AvailabilityReportSerializer] (http-/0.0.0.0:7080-76) tid=3746; agent=dev07-agent-16226: releasing write lock after being locked for millis=22096 >08:35:50,568 INFO [org.rhq.enterprise.server.util.concurrent.AvailabilityReportSerializer] (http-/0.0.0.0:7080-9) tid=3512; agent=dev87-agent-16201: releasing write lock after being locked for millis=7294 >08:35:51,421 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 8) dhcp180-146.mw.lab.eng.bos.redhat.com took [1303]ms to reload cache for 2 agents >08:35:51,629 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-98) Received create resource response: CreateResourceResponse[RequestId=10271, Status=Success] >08:35:52,235 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-108) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >08:35:52,336 INFO [org.rhq.enterprise.server.scheduler.jobs.AsyncResourceDeleteJob] (RHQScheduler_Worker-2) Async resource deletion - 50 successful, 0 failed, took [92172] ms >08:35:52,506 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-90) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:36:07,591 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-7) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:36:09,543 INFO [org.rhq.enterprise.server.util.concurrent.InventoryReportSerializer] (http-/0.0.0.0:7080-29) tid=3601; agent=dev07-agent-16226: releasing write lock after being locked for millis=15650 >08:36:20,568 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-1) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:36:24,262 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-7) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:36:24,569 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 4) dhcp180-146.mw.lab.eng.bos.redhat.com took [4358]ms to reload cache for 2 agents >08:36:29,513 INFO [org.rhq.enterprise.server.util.concurrent.AvailabilityReportSerializer] (http-/0.0.0.0:7080-29) tid=3601; agent=dev07-agent-16226: releasing write lock after being locked for millis=11047 >08:36:37,641 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-7) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:36:52,598 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-12) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:36:55,585 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-56) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >08:36:55,587 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-20) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >08:37:07,593 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-12) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:37:20,351 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-1) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:37:22,582 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-12) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:37:33,670 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-52) Received create resource response: CreateResourceResponse[RequestId=10006, Status=Success] >08:37:37,693 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-12) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:37:43,888 INFO [org.rhq.enterprise.server.util.concurrent.InventoryReportSerializer] (http-/0.0.0.0:7080-52) tid=3722; agent=dspvmg01-agent-16205: releasing write lock after being locked for millis=5952 >08:37:50,582 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 5) dhcp180-146.mw.lab.eng.bos.redhat.com took [472]ms to reload cache for 1 agents >08:37:51,900 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-92) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >08:37:52,546 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-12) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:38:07,516 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-104) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:38:11,502 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-81) Received delete resource response: DeleteResourceResponse[RequestId=10241, Status=Success] >08:38:20,154 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-5) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:38:20,277 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 6) dhcp180-146.mw.lab.eng.bos.redhat.com took [156]ms to reload cache for 1 agents >08:38:22,509 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-104) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:38:37,586 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-64) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:38:52,551 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-7) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:38:52,556 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-37) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >08:39:04,854 INFO [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/coregui]] (http-/0.0.0.0:7080-109) org.rhq.coregui.CoreGUI PackageVersionFileUploadServlet: file was uploaded: mock-war.war >08:39:07,521 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-37) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:39:20,416 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-5) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:39:22,519 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-37) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:39:37,965 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-31) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:39:52,600 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-31) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:39:55,841 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-67) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >08:40:07,566 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-67) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:40:07,799 ERROR [org.rhq.enterprise.server.alert.engine.internal.AlertConditionCacheCoordinator] (EJB default - 10) Could not find agent for scheduleId = 43046 >08:40:08,844 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-20) Received delete resource response: DeleteResourceResponse[RequestId=10231, Status=Success] >08:40:09,192 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-50) Received delete resource response: DeleteResourceResponse[RequestId=10102, Status=Success] >08:40:13,750 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-6) Received create resource response: CreateResourceResponse[RequestId=10282, Status=Success] >08:40:20,189 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-3) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:40:20,450 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 1) dhcp180-146.mw.lab.eng.bos.redhat.com took [344]ms to reload cache for 1 agents >08:40:22,575 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-56) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:40:26,628 INFO [org.rhq.enterprise.server.scheduler.jobs.AsyncResourceDeleteJob] (RHQScheduler_Worker-1) Async resource deletion - 95 successful, 0 failed, took [66502] ms >08:40:37,593 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-56) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:40:52,566 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-56) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:40:56,883 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-35) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >08:41:04,663 INFO [org.rhq.enterprise.server.util.concurrent.InventoryReportSerializer] (http-/0.0.0.0:7080-67) tid=3737; agent=dhcp95-226-agent-16163: releasing write lock after being locked for millis=9871 >08:41:07,613 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-56) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:41:14,173 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-46) Received delete resource response: DeleteResourceResponse[RequestId=10143, Status=Success] >08:41:20,373 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-5) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:41:21,361 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 6) dhcp180-146.mw.lab.eng.bos.redhat.com took [463]ms to reload cache for 1 agents >08:41:21,945 INFO [org.rhq.enterprise.server.util.concurrent.InventoryReportSerializer] (http-/0.0.0.0:7080-67) tid=3737; agent=dhcp95-226-agent-16163: releasing write lock after being locked for millis=11983 >08:41:22,691 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-56) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:41:37,532 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-56) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:41:42,107 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-69) Received delete resource response: DeleteResourceResponse[RequestId=10251, Status=Success] >08:41:50,282 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 7) dhcp180-146.mw.lab.eng.bos.redhat.com took [180]ms to reload cache for 1 agents >08:41:52,622 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-9) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:41:53,984 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-97) Received delete resource response: DeleteResourceResponse[RequestId=10213, Status=Success] >08:41:55,659 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-40) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >08:41:57,802 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-49) Received delete resource response: DeleteResourceResponse[RequestId=10261, Status=Success] >08:42:07,632 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-9) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:42:20,144 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-4) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:42:20,185 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-4) Have not heard from agent [dev114-agent-16230] since [Thu Apr 03 08:36:59 EDT 2014]. Will be backfilled since we suspect it is down >08:42:22,622 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-9) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:42:30,768 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-78) Received create resource response: CreateResourceResponse[RequestId=10163, Status=Success] >08:42:37,629 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-25) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:42:37,699 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-71) Received create resource response: CreateResourceResponse[RequestId=10192, Status=Success] >08:42:50,719 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 4) dhcp180-146.mw.lab.eng.bos.redhat.com took [610]ms to reload cache for 3 agents >08:42:51,632 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-22) Received create resource response: CreateResourceResponse[RequestId=10281, Status=Success] >08:42:52,619 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-25) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:42:58,063 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-74) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >08:42:58,446 INFO [org.rhq.enterprise.server.util.concurrent.InventoryReportSerializer] (http-/0.0.0.0:7080-51) tid=3721; agent=dev07-agent-16226: releasing write lock after being locked for millis=6162 >08:43:07,716 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-74) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:43:20,283 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-5) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:43:21,046 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 1) dhcp180-146.mw.lab.eng.bos.redhat.com took [929]ms to reload cache for 2 agents >08:43:22,627 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-74) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:43:37,639 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-97) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:43:42,243 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-65) Received create resource response: CreateResourceResponse[RequestId=10068, Status=Success] >08:43:50,507 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 7) dhcp180-146.mw.lab.eng.bos.redhat.com took [396]ms to reload cache for 1 agents >08:43:52,839 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-97) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:43:55,556 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-29) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >08:44:07,697 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-97) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:44:20,222 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-1) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:44:22,680 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-12) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:44:34,074 INFO [org.rhq.enterprise.server.scheduler.jobs.AsyncResourceDeleteJob] (RHQScheduler_Worker-4) Async resource deletion - 50 successful, 0 failed, took [14010] ms >08:44:37,687 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-12) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:44:43,125 INFO [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/coregui]] (http-/0.0.0.0:7080-3) org.rhq.coregui.CoreGUI PackageVersionFileUploadServlet: file was uploaded: mock-ejb2.jar >08:44:52,682 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-12) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:44:54,166 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-104) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >08:45:06,742 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-10) Received create resource response: CreateResourceResponse[RequestId=10073, Status=Success] >08:45:07,715 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-12) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:45:19,130 INFO [org.rhq.enterprise.server.util.concurrent.InventoryReportSerializer] (http-/0.0.0.0:7080-101) tid=4329; agent=dev87-agent-16201: releasing write lock after being locked for millis=16030 >08:45:20,330 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-1) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:45:21,249 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 9) dhcp180-146.mw.lab.eng.bos.redhat.com took [1120]ms to reload cache for 2 agents >08:45:24,270 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-62) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:45:30,620 INFO [org.rhq.enterprise.server.util.concurrent.InventoryReportSerializer] (http-/0.0.0.0:7080-31) tid=3603; agent=dev87-agent-16201: releasing write lock after being locked for millis=5602 >08:45:37,693 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-62) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:45:50,250 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 3) dhcp180-146.mw.lab.eng.bos.redhat.com took [138]ms to reload cache for 2 agents >08:45:52,673 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-60) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:45:53,617 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-98) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >08:46:09,613 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-53) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:46:20,176 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-3) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:46:21,668 WARN [org.rhq.helpers.rtfilter.filter.RtFilter] (http-/0.0.0.0:7080-69) Response time log '/qa/hudson_workspace/workspace/jon-32-server-install/dhcp180-146.mw.lab.eng.bos.redhat.com/jon-server-3.2.0.GA/logs/rt/10.16.180.234_coregui_rt.log' has exceeded maximum file size (5242880 bytes) - truncating it... >08:46:22,758 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-53) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:46:37,022 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-22) Received delete resource response: DeleteResourceResponse[RequestId=10023, Status=Success] >08:46:37,790 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-53) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:46:52,747 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-84) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:46:53,597 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-96) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >08:47:00,349 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-60) Received delete resource response: DeleteResourceResponse[RequestId=10072, Status=Success] >08:47:02,549 INFO [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/coregui]] (http-/0.0.0.0:7080-72) org.rhq.coregui.CoreGUI PackageVersionFileUploadServlet: file was uploaded: mock-ejb3.jar >08:47:03,719 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-45) Received create resource response: CreateResourceResponse[RequestId=10133, Status=Success] >08:47:07,788 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-84) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:47:20,358 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-3) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:47:20,382 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 3) dhcp180-146.mw.lab.eng.bos.redhat.com took [243]ms to reload cache for 1 agents >08:47:22,753 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-84) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:47:37,749 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-10) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:47:52,455 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-48) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >08:47:52,803 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-10) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:48:04,849 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-26) Received delete resource response: DeleteResourceResponse[RequestId=10271, Status=Success] >08:48:07,807 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-10) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:48:16,812 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-28) Received delete resource response: DeleteResourceResponse[RequestId=10192, Status=Success] >08:48:20,316 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-5) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:48:21,719 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-63) Received create resource response: CreateResourceResponse[RequestId=10075, Status=Success] >08:48:22,762 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-10) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:48:26,032 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-95) Received delete resource response: DeleteResourceResponse[RequestId=10144, Status=Success] >08:48:37,751 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-10) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:48:50,187 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 5) dhcp180-146.mw.lab.eng.bos.redhat.com took [74]ms to reload cache for 1 agents >08:48:52,754 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-10) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:48:53,209 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-46) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >08:49:07,769 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-10) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:49:19,258 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-40) Received delete resource response: DeleteResourceResponse[RequestId=10281, Status=Success] >08:49:20,283 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-2) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:49:21,939 INFO [org.rhq.enterprise.server.scheduler.jobs.AsyncResourceDeleteJob] (RHQScheduler_Worker-3) Async resource deletion - 2 successful, 0 failed, took [1878] ms >08:49:22,855 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-10) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:49:24,049 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-90) Received create resource response: CreateResourceResponse[RequestId=10034, Status=Success] >08:49:26,220 INFO [org.rhq.enterprise.server.util.concurrent.InventoryReportSerializer] (http-/0.0.0.0:7080-7) tid=3477; agent=dev02-agent-16222: releasing write lock after being locked for millis=10642 >08:49:37,658 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-107) Received delete resource response: DeleteResourceResponse[RequestId=10024, Status=Success] >08:49:37,801 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-78) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:49:50,340 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 4) dhcp180-146.mw.lab.eng.bos.redhat.com took [230]ms to reload cache for 2 agents >08:49:52,361 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-81) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >08:49:52,756 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-78) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:50:02,181 WARN [org.rhq.enterprise.server.measurement.AvailabilityManagerBean] (http-/0.0.0.0:7080-102) Resource [Resource[id=13792, uuid=null, type=<null>, key=null, name=null, parent=<null>]] has multiple availabilities without an endtime [result returns more than one elements] - will attempt to remove the extra ones >AV:[dev02-agent-16222][53][full] >08:50:07,822 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-78) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:50:08,676 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-39) Received create resource response: CreateResourceResponse[RequestId=10074, Status=Success] >08:50:09,238 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-99) Received create resource response: CreateResourceResponse[RequestId=10015, Status=Success] >08:50:20,224 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-1) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:50:20,708 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 7) dhcp180-146.mw.lab.eng.bos.redhat.com took [595]ms to reload cache for 2 agents >08:50:22,840 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-78) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:50:35,766 INFO [org.rhq.enterprise.server.util.concurrent.InventoryReportSerializer] (http-/0.0.0.0:7080-99) tid=4327; agent=dev02-agent-16222: releasing write lock after being locked for millis=12212 >08:50:37,875 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-55) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:50:52,652 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 10) dhcp180-146.mw.lab.eng.bos.redhat.com took [826]ms to reload cache for 2 agents >08:50:52,841 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-55) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:50:53,918 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-47) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >08:50:55,434 INFO [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/coregui]] (http-/0.0.0.0:7080-82) org.rhq.coregui.CoreGUI PackageVersionFileUploadServlet: file was uploaded: mock-war.war >08:51:07,817 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-55) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:51:20,162 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-1) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:51:20,272 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-1) Have not heard from agent [dev101-agent-16177] since [Thu Apr 03 08:45:52 EDT 2014]. Will be backfilled since we suspect it is down >08:51:22,827 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-55) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:51:30,681 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-25) Received create resource response: CreateResourceResponse[RequestId=10046, Status=Success] >08:51:32,892 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-56) Received create resource response: CreateResourceResponse[RequestId=10016, Status=Success] >08:51:37,832 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-84) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:51:50,423 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 9) dhcp180-146.mw.lab.eng.bos.redhat.com took [292]ms to reload cache for 2 agents >08:51:52,816 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-84) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:51:58,610 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-53) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >08:52:07,858 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-53) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:52:20,151 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-4) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:52:22,881 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-53) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:52:23,596 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-2) Received create resource response: CreateResourceResponse[RequestId=10272, Status=Success] >08:52:37,903 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-53) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:52:50,875 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 9) dhcp180-146.mw.lab.eng.bos.redhat.com took [770]ms to reload cache for 2 agents >08:52:52,896 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-45) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:52:57,551 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-80) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >08:53:07,890 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-45) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:53:20,625 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-5) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:53:22,882 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-37) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:53:37,900 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-37) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:53:52,916 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-37) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:54:02,871 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-71) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >08:54:07,974 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-71) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:54:20,397 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-4) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:54:21,025 INFO [org.rhq.enterprise.server.scheduler.jobs.AsyncResourceDeleteJob] (RHQScheduler_Worker-3) Async resource deletion - 1 successful, 0 failed, took [911] ms >08:54:22,939 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-71) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:54:37,963 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-22) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:54:52,928 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-22) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:55:06,405 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-100) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >08:55:07,932 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-22) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:55:12,989 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-9) Received create resource response: CreateResourceResponse[RequestId=10232, Status=Success] >08:55:20,159 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-1) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:55:20,283 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 5) dhcp180-146.mw.lab.eng.bos.redhat.com took [167]ms to reload cache for 1 agents >08:55:22,911 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-104) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:55:26,656 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-82) Received delete resource response: DeleteResourceResponse[RequestId=10103, Status=Success] >08:55:37,998 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-78) Received delete resource response: DeleteResourceResponse[RequestId=10262, Status=Success] >08:55:38,020 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-104) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:55:50,458 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 7) dhcp180-146.mw.lab.eng.bos.redhat.com took [351]ms to reload cache for 1 agents >08:55:52,926 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-6) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:55:53,703 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-62) Received delete resource response: DeleteResourceResponse[RequestId=10214, Status=Success] >08:56:00,935 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-60) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >08:56:08,009 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-60) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:56:17,533 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-85) Received create resource response: CreateResourceResponse[RequestId=10291, Status=Success] >08:56:20,243 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-3) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:56:23,654 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-52) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:56:38,043 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-52) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:56:50,486 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 1) dhcp180-146.mw.lab.eng.bos.redhat.com took [377]ms to reload cache for 1 agents >08:56:52,972 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-22) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:56:54,208 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-74) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >08:57:07,985 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-22) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:57:13,001 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-49) Received delete resource response: DeleteResourceResponse[RequestId=10222, Status=Success] >08:57:20,152 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-2) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:57:22,977 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-101) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:57:38,032 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-101) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:57:53,039 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-101) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:57:54,212 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-13) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >08:58:08,030 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-101) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:58:09,045 INFO [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/coregui]] (http-/0.0.0.0:7080-61) org.rhq.coregui.CoreGUI PackageVersionFileUploadServlet: file was uploaded: mock-ejb2.jar >08:58:15,640 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-81) Received create resource response: CreateResourceResponse[RequestId=10026, Status=Success] >08:58:20,137 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-2) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:58:20,265 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 10) dhcp180-146.mw.lab.eng.bos.redhat.com took [158]ms to reload cache for 1 agents >08:58:23,126 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-82) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:58:33,823 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-69) Received create resource response: CreateResourceResponse[RequestId=10054, Status=Success] >08:58:34,232 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-40) Received create resource response: CreateResourceResponse[RequestId=10183, Status=Success] >08:58:38,054 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-82) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:58:47,741 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-57) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >08:58:50,571 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 7) dhcp180-146.mw.lab.eng.bos.redhat.com took [419]ms to reload cache for 2 agents >08:58:53,048 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-57) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:59:08,037 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-41) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:59:20,127 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-2) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >08:59:20,530 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 9) dhcp180-146.mw.lab.eng.bos.redhat.com took [183]ms to reload cache for 1 agents >08:59:23,031 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-41) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:59:35,079 INFO [org.rhq.enterprise.server.scheduler.jobs.AsyncResourceDeleteJob] (RHQScheduler_Worker-5) Async resource deletion - 48 successful, 0 failed, took [15031] ms >08:59:38,105 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-41) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >08:59:49,134 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-4) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >08:59:53,134 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-41) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:00:00,044 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-2) Data Purge Job STARTING >09:00:00,049 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-2) Measurement data compression starting at Thu Apr 03 09:00:00 EDT 2014 >09:00:08,260 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-41) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:00:14,130 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-40) Received delete resource response: DeleteResourceResponse[RequestId=10025, Status=Success] >09:00:19,561 INFO [org.rhq.server.metrics.MetricsServer] (RHQScheduler_Worker-2) Finished computing aggregates for table [raw_metrics]19512 ms >09:00:20,269 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-1) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:00:23,692 INFO [org.rhq.server.metrics.MetricsServer] (RHQScheduler_Worker-2) Finished computing aggregates for table [one_hour_metrics] 0 ms >09:00:23,692 INFO [org.rhq.server.metrics.MetricsServer] (RHQScheduler_Worker-2) Finished computing aggregates for table [six_hour_metrics] 0 ms >09:00:23,693 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-2) Measurement data compression completed in [23644]ms >09:00:23,693 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-2) Measurement calltime data purge starting at Thu Apr 03 09:00:23 EDT 2014 >09:00:23,693 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-2) Purging calltime data that is older than Mon Mar 03 08:00:23 EST 2014 >09:00:23,696 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-2) Calltime purged [0] - completed in [3]ms >09:00:23,696 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-2) Event data purge starting at Thu Apr 03 09:00:23 EDT 2014 >09:00:23,696 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-2) Purging event data older than Thu Mar 20 09:00:23 EDT 2014 >09:00:23,786 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-2) Event data purged [0] - completed in [90]ms >09:00:23,786 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-2) Alert data purge starting at Thu Apr 03 09:00:23 EDT 2014 >09:00:23,786 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-2) Purging alert data older than Mon Mar 03 08:00:23 EST 2014 >09:00:23,846 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-88) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:00:23,995 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-2) Alert data purged [0] - completed in [209]ms >09:00:23,996 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-2) Alert definition unused purge starting at Thu Apr 03 09:00:23 EDT 2014 >09:00:24,106 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-2) Alert definitions purged [0] - completed in [110]ms >09:00:24,106 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-2) Alert condition orphan purge starting at Thu Apr 03 09:00:24 EDT 2014 >09:00:24,201 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-2) Purged [0] orphan alert conditions - completed in [95]ms >09:00:24,201 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-2) Alert notification orphan purge starting at Thu Apr 03 09:00:24 EDT 2014 >09:00:24,266 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-2) Purged [0] orphan alert notifications - completed in [65]ms >09:00:24,267 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-2) Trait data purge starting at Thu Apr 03 09:00:24 EDT 2014 >09:00:24,267 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-2) Purging traits that are older than Wed Apr 03 09:00:24 EDT 2013 >09:00:24,312 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-2) Traits data purged [0] - completed in [45]ms >09:00:24,312 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-2) Availability data purge starting at Thu Apr 03 09:00:24 EDT 2014 >09:00:24,312 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-2) Purging availablities that are older than Wed Apr 03 09:00:24 EDT 2013 >09:00:24,386 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-2) Availability data purged [0] - completed in [74]ms >09:00:24,386 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-2) Drift file orphan purge starting at Thu Apr 03 09:00:24 EDT 2014 >09:00:24,386 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-2) Purging orphaned drift files older than Mon Mar 03 08:00:24 EST 2014 >09:00:24,509 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-2) Purged [0] orphaned drift files - completed in [123]ms >09:00:24,510 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-2) Database maintenance starting at Thu Apr 03 09:00:24 EDT 2014 >09:00:24,510 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-2) Performing hourly database maintenance >09:00:30,132 ERROR [org.jboss.as.ejb3.invocation] (http-/0.0.0.0:7080-26) JBAS014134: EJB Invocation failed on component SystemManagerBean for method public abstract org.rhq.core.domain.common.composite.SystemSettings org.rhq.enterprise.server.system.SystemManagerLocal.getUnmaskedSystemSettings(boolean): javax.ejb.ConcurrentAccessTimeoutException: JBAS014373: EJB 3.1 PFD2 4.8.5.5.1 concurrent access timeout on org.jboss.invocation.InterceptorContext$Invocation@19b208b5 - could not obtain lock within 5000MILLISECONDS > at org.jboss.as.ejb3.concurrency.ContainerManagedConcurrencyInterceptor.processInvocation(ContainerManagedConcurrencyInterceptor.java:100) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.system.SystemManagerLocal$$$view165.getUnmaskedSystemSettings(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.SystemGWTServiceImpl.getSessionTimeout(SystemGWTServiceImpl.java:77) > at sun.reflect.GeneratedMethodAccessor560.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] > >09:00:30,140 ERROR [org.jboss.as.ejb3.invocation] (http-/0.0.0.0:7080-74) JBAS014134: EJB Invocation failed on component SystemManagerBean for method public abstract org.rhq.core.domain.common.ProductInfo org.rhq.enterprise.server.system.SystemManagerRemote.getProductInfo(org.rhq.core.domain.auth.Subject): javax.ejb.ConcurrentAccessTimeoutException: JBAS014373: EJB 3.1 PFD2 4.8.5.5.1 concurrent access timeout on org.jboss.invocation.InterceptorContext$Invocation@79cb25f5 - could not obtain lock within 5000MILLISECONDS > at org.jboss.as.ejb3.concurrency.ContainerManagedConcurrencyInterceptor.processInvocation(ContainerManagedConcurrencyInterceptor.java:100) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.system.SystemManagerLocal$$$view165.getProductInfo(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.SystemGWTServiceImpl.getProductInfo(SystemGWTServiceImpl.java:59) > at sun.reflect.GeneratedMethodAccessor561.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] > >09:00:30,317 WARN [org.rhq.coregui.server.gwt.SystemGWTServiceImpl] (http-/0.0.0.0:7080-26) Sending exception to client: [1396530030317] : javax.ejb.ConcurrentAccessTimeoutException: JBAS014373: EJB 3.1 PFD2 4.8.5.5.1 concurrent access timeout on org.jboss.invocation.InterceptorContext$Invocation@19b208b5 - could not obtain lock within 5000MILLISECONDS > at org.jboss.as.ejb3.concurrency.ContainerManagedConcurrencyInterceptor.processInvocation(ContainerManagedConcurrencyInterceptor.java:100) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.system.SystemManagerLocal$$$view165.getUnmaskedSystemSettings(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.SystemGWTServiceImpl.getSessionTimeout(SystemGWTServiceImpl.java:77) [classes:] > at sun.reflect.GeneratedMethodAccessor560.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) [gwt-servlet-2.5.0.jar:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) [classes:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) [classes:] > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) [jboss-as-web-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] > >09:00:30,368 WARN [org.rhq.coregui.server.gwt.SystemGWTServiceImpl] (http-/0.0.0.0:7080-74) Sending exception to client: [1396530030368] : javax.ejb.ConcurrentAccessTimeoutException: JBAS014373: EJB 3.1 PFD2 4.8.5.5.1 concurrent access timeout on org.jboss.invocation.InterceptorContext$Invocation@79cb25f5 - could not obtain lock within 5000MILLISECONDS > at org.jboss.as.ejb3.concurrency.ContainerManagedConcurrencyInterceptor.processInvocation(ContainerManagedConcurrencyInterceptor.java:100) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.system.SystemManagerLocal$$$view165.getProductInfo(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.SystemGWTServiceImpl.getProductInfo(SystemGWTServiceImpl.java:59) [classes:] > at sun.reflect.GeneratedMethodAccessor561.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) [gwt-servlet-2.5.0.jar:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) [classes:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) [classes:] > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) [jboss-as-web-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] > >09:00:33,196 INFO [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/coregui]] (http-/0.0.0.0:7080-99) org.rhq.coregui.CoreGUI PackageVersionFileUploadServlet: file was uploaded: mock-ejb3.jar >09:00:38,340 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-17) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:00:42,824 ERROR [org.jboss.as.ejb3.invocation] (http-/0.0.0.0:7080-28) JBAS014134: EJB Invocation failed on component SystemManagerBean for method public abstract org.rhq.core.domain.common.ProductInfo org.rhq.enterprise.server.system.SystemManagerRemote.getProductInfo(org.rhq.core.domain.auth.Subject): javax.ejb.ConcurrentAccessTimeoutException: JBAS014373: EJB 3.1 PFD2 4.8.5.5.1 concurrent access timeout on org.jboss.invocation.InterceptorContext$Invocation@795d5bf3 - could not obtain lock within 5000MILLISECONDS > at org.jboss.as.ejb3.concurrency.ContainerManagedConcurrencyInterceptor.processInvocation(ContainerManagedConcurrencyInterceptor.java:100) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.system.SystemManagerLocal$$$view165.getProductInfo(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.SystemGWTServiceImpl.getProductInfo(SystemGWTServiceImpl.java:59) > at sun.reflect.GeneratedMethodAccessor561.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] > >09:00:42,872 ERROR [org.jboss.as.ejb3.invocation] (http-/0.0.0.0:7080-104) JBAS014134: EJB Invocation failed on component SystemManagerBean for method public abstract org.rhq.core.domain.common.composite.SystemSettings org.rhq.enterprise.server.system.SystemManagerLocal.getUnmaskedSystemSettings(boolean): javax.ejb.ConcurrentAccessTimeoutException: JBAS014373: EJB 3.1 PFD2 4.8.5.5.1 concurrent access timeout on org.jboss.invocation.InterceptorContext$Invocation@26f3f514 - could not obtain lock within 5000MILLISECONDS > at org.jboss.as.ejb3.concurrency.ContainerManagedConcurrencyInterceptor.processInvocation(ContainerManagedConcurrencyInterceptor.java:100) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.system.SystemManagerLocal$$$view165.getUnmaskedSystemSettings(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.SystemGWTServiceImpl.getSessionTimeout(SystemGWTServiceImpl.java:77) > at sun.reflect.GeneratedMethodAccessor560.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] > >09:00:42,979 WARN [org.rhq.coregui.server.gwt.SystemGWTServiceImpl] (http-/0.0.0.0:7080-28) Sending exception to client: [1396530042979] : javax.ejb.ConcurrentAccessTimeoutException: JBAS014373: EJB 3.1 PFD2 4.8.5.5.1 concurrent access timeout on org.jboss.invocation.InterceptorContext$Invocation@795d5bf3 - could not obtain lock within 5000MILLISECONDS > at org.jboss.as.ejb3.concurrency.ContainerManagedConcurrencyInterceptor.processInvocation(ContainerManagedConcurrencyInterceptor.java:100) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.system.SystemManagerLocal$$$view165.getProductInfo(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.SystemGWTServiceImpl.getProductInfo(SystemGWTServiceImpl.java:59) [classes:] > at sun.reflect.GeneratedMethodAccessor561.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) [gwt-servlet-2.5.0.jar:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) [classes:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) [classes:] > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) [jboss-as-web-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] > >09:00:43,033 WARN [org.rhq.coregui.server.gwt.SystemGWTServiceImpl] (http-/0.0.0.0:7080-104) Sending exception to client: [1396530043033] : javax.ejb.ConcurrentAccessTimeoutException: JBAS014373: EJB 3.1 PFD2 4.8.5.5.1 concurrent access timeout on org.jboss.invocation.InterceptorContext$Invocation@26f3f514 - could not obtain lock within 5000MILLISECONDS > at org.jboss.as.ejb3.concurrency.ContainerManagedConcurrencyInterceptor.processInvocation(ContainerManagedConcurrencyInterceptor.java:100) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.system.SystemManagerLocal$$$view165.getUnmaskedSystemSettings(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.SystemGWTServiceImpl.getSessionTimeout(SystemGWTServiceImpl.java:77) [classes:] > at sun.reflect.GeneratedMethodAccessor560.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) [gwt-servlet-2.5.0.jar:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) [classes:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) [classes:] > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) [jboss-as-web-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] > >09:00:47,496 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-33) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:00:48,209 ERROR [org.jboss.as.ejb3.invocation] (http-/0.0.0.0:7080-55) JBAS014134: EJB Invocation failed on component SystemManagerBean for method public abstract org.rhq.core.domain.common.composite.SystemSettings org.rhq.enterprise.server.system.SystemManagerLocal.getUnmaskedSystemSettings(boolean): javax.ejb.ConcurrentAccessTimeoutException: JBAS014373: EJB 3.1 PFD2 4.8.5.5.1 concurrent access timeout on org.jboss.invocation.InterceptorContext$Invocation@e40bf32 - could not obtain lock within 5000MILLISECONDS > at org.jboss.as.ejb3.concurrency.ContainerManagedConcurrencyInterceptor.processInvocation(ContainerManagedConcurrencyInterceptor.java:100) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.system.SystemManagerLocal$$$view165.getUnmaskedSystemSettings(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.SystemGWTServiceImpl.getSessionTimeout(SystemGWTServiceImpl.java:77) > at sun.reflect.GeneratedMethodAccessor560.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] > >09:00:48,212 ERROR [org.jboss.as.ejb3.invocation] (http-/0.0.0.0:7080-49) JBAS014134: EJB Invocation failed on component SystemManagerBean for method public abstract org.rhq.core.domain.common.ProductInfo org.rhq.enterprise.server.system.SystemManagerRemote.getProductInfo(org.rhq.core.domain.auth.Subject): javax.ejb.ConcurrentAccessTimeoutException: JBAS014373: EJB 3.1 PFD2 4.8.5.5.1 concurrent access timeout on org.jboss.invocation.InterceptorContext$Invocation@3cc872ed - could not obtain lock within 5000MILLISECONDS > at org.jboss.as.ejb3.concurrency.ContainerManagedConcurrencyInterceptor.processInvocation(ContainerManagedConcurrencyInterceptor.java:100) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.system.SystemManagerLocal$$$view165.getProductInfo(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.SystemGWTServiceImpl.getProductInfo(SystemGWTServiceImpl.java:59) > at sun.reflect.GeneratedMethodAccessor561.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] > >09:00:48,348 WARN [org.rhq.coregui.server.gwt.SystemGWTServiceImpl] (http-/0.0.0.0:7080-49) Sending exception to client: [1396530048348] : javax.ejb.ConcurrentAccessTimeoutException: JBAS014373: EJB 3.1 PFD2 4.8.5.5.1 concurrent access timeout on org.jboss.invocation.InterceptorContext$Invocation@3cc872ed - could not obtain lock within 5000MILLISECONDS > at org.jboss.as.ejb3.concurrency.ContainerManagedConcurrencyInterceptor.processInvocation(ContainerManagedConcurrencyInterceptor.java:100) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.system.SystemManagerLocal$$$view165.getProductInfo(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.SystemGWTServiceImpl.getProductInfo(SystemGWTServiceImpl.java:59) [classes:] > at sun.reflect.GeneratedMethodAccessor561.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) [gwt-servlet-2.5.0.jar:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) [classes:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) [classes:] > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) [jboss-as-web-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] > >09:00:48,345 WARN [org.rhq.coregui.server.gwt.SystemGWTServiceImpl] (http-/0.0.0.0:7080-55) Sending exception to client: [1396530048345] : javax.ejb.ConcurrentAccessTimeoutException: JBAS014373: EJB 3.1 PFD2 4.8.5.5.1 concurrent access timeout on org.jboss.invocation.InterceptorContext$Invocation@e40bf32 - could not obtain lock within 5000MILLISECONDS > at org.jboss.as.ejb3.concurrency.ContainerManagedConcurrencyInterceptor.processInvocation(ContainerManagedConcurrencyInterceptor.java:100) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.system.SystemManagerLocal$$$view165.getUnmaskedSystemSettings(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.coregui.server.gwt.SystemGWTServiceImpl.getSessionTimeout(SystemGWTServiceImpl.java:77) [classes:] > at sun.reflect.GeneratedMethodAccessor560.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:561) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:208) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:248) [gwt-servlet-2.5.0.jar:] > at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62) [gwt-servlet-2.5.0.jar:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at org.rhq.coregui.server.gwt.AbstractGWTServiceImpl.service(AbstractGWTServiceImpl.java:88) [classes:] > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1] > at sun.reflect.GeneratedMethodAccessor175.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:155) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:288) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.helpers.rtfilter.filter.RtFilter.doFilter(RtFilter.java:125) [rhq-rtfilter-4.9.0.JON320GA.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.rhq.coregui.server.filter.CacheControlFilter.doFilter(CacheControlFilter.java:70) [classes:] > at sun.reflect.GeneratedMethodAccessor320.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:263) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:261) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at javax.security.auth.Subject.doAsPrivileged(Subject.java:517) [rt.jar:1.6.0_45] > at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:236) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain.access$000(ApplicationFilterChain.java:59) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.ApplicationFilterChain$1.run(ApplicationFilterChain.java:197) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.6.0_45] > at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) [jboss-as-web-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1] > at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45] > >09:00:49,702 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-2) Database maintenance completed in [25192]ms >09:00:49,703 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-2) Auto-calculation of baselines starting at Thu Apr 03 09:00:49 EDT 2014 >09:00:49,709 INFO [org.rhq.enterprise.server.measurement.MeasurementBaselineManagerBean] (RHQScheduler_Worker-2) Calculating auto baselines >09:00:49,709 INFO [org.rhq.enterprise.server.measurement.MeasurementBaselineManagerBean] (RHQScheduler_Worker-2) Deleting baselines computations older than Mon Mar 31 09:00:49 EDT 2014 >09:00:49,709 INFO [org.rhq.enterprise.server.measurement.MeasurementBaselineManagerBean] (RHQScheduler_Worker-2) Inserting new baselines using last 7 days of 1H data >09:00:49,724 INFO [org.rhq.enterprise.server.measurement.MeasurementBaselineManagerBean] (RHQScheduler_Worker-2) Removed [0] old baselines - they will now be recalculated (15)ms >09:00:53,155 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-33) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:01:03,554 INFO [org.rhq.enterprise.server.measurement.MeasurementBaselineManagerBean] (RHQScheduler_Worker-2) Calculated and inserted [5605] new baselines. (13830)ms >09:01:03,754 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-2) Auto-calculation of baselines completed in [14051]ms >09:01:03,820 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-2) Auto-calculation of OOBs starting >09:01:03,918 INFO [org.rhq.enterprise.server.measurement.MeasurementOOBManagerBean] (RHQScheduler_Worker-2) Removed [0] outdated OOBs >09:01:04,002 INFO [org.rhq.enterprise.server.measurement.MeasurementOOBManagerBean] (RHQScheduler_Worker-2) Computing OOBs >09:01:08,263 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-7) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:01:20,694 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-5) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:01:22,519 INFO [org.rhq.enterprise.server.util.concurrent.InventoryReportSerializer] (http-/0.0.0.0:7080-75) tid=3745; agent=dev07-agent-16226: releasing write lock after being locked for millis=21348 >09:01:22,772 INFO [org.rhq.enterprise.server.util.concurrent.AvailabilityReportSerializer] (http-/0.0.0.0:7080-94) tid=4150; agent=dev07-agent-16226: releasing write lock after being locked for millis=12963 >09:01:24,882 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-63) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:01:38,253 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-63) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:01:39,498 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 9) dhcp180-146.mw.lab.eng.bos.redhat.com took [17254]ms to reload cache for 25 agents >09:01:47,309 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-51) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:01:48,263 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-106) Received delete resource response: DeleteResourceResponse[RequestId=10014, Status=Success] >09:01:48,416 INFO [org.rhq.enterprise.server.util.concurrent.InventoryReportSerializer] (http-/0.0.0.0:7080-75) tid=3745; agent=dev07-agent-16226: releasing write lock after being locked for millis=17907 >09:01:50,633 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 7) dhcp180-146.mw.lab.eng.bos.redhat.com took [523]ms to reload cache for 1 agents >09:01:52,039 INFO [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/coregui]] (http-/0.0.0.0:7080-40) org.rhq.coregui.CoreGUI PackageVersionFileUploadServlet: file was uploaded: mock-ejb2.jar >09:01:53,261 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-51) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:02:15,605 INFO [org.rhq.enterprise.server.measurement.MeasurementOOBManagerBean] (RHQScheduler_Worker-2) Finished calculating 196 OOBs in 71602 ms >09:02:15,606 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-2) Auto-calculation of OOBs completed in [71786]ms >09:02:15,606 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-2) Data Purge Job FINISHED [135562]ms >09:02:18,433 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-14) Received create resource response: CreateResourceResponse[RequestId=10193, Status=Invalid Artifact] >09:02:20,243 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-2) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:02:25,291 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-94) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:02:40,279 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-58) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:02:47,477 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-65) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:02:55,241 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-65) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:03:10,178 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-41) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:03:16,902 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-35) Received delete resource response: DeleteResourceResponse[RequestId=10063, Status=Success] >09:03:20,344 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-4) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:03:22,993 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-82) Received delete resource response: DeleteResourceResponse[RequestId=10215, Status=Success] >09:03:25,193 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-41) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:03:39,547 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-16) Received create resource response: CreateResourceResponse[RequestId=10301, Status=Success] >09:03:40,237 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-41) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:03:41,280 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-65) Received create resource response: CreateResourceResponse[RequestId=10242, Status=Success] >09:03:47,463 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-78) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:03:50,330 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 5) dhcp180-146.mw.lab.eng.bos.redhat.com took [219]ms to reload cache for 2 agents >09:03:55,198 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-78) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:04:10,261 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-91) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:04:14,766 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-104) Received create resource response: CreateResourceResponse[RequestId=10027, Status=Success] >09:04:20,368 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-1) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:04:21,121 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 3) dhcp180-146.mw.lab.eng.bos.redhat.com took [648]ms to reload cache for 2 agents >09:04:21,815 INFO [org.rhq.enterprise.server.scheduler.jobs.AsyncResourceDeleteJob] (RHQScheduler_Worker-5) Async resource deletion - 2 successful, 0 failed, took [1733] ms >09:04:28,611 INFO [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/coregui]] (http-/0.0.0.0:7080-65) org.rhq.coregui.CoreGUI PackageVersionFileUploadServlet: file was uploaded: mock-war.war >09:04:36,893 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-71) Received delete resource response: DeleteResourceResponse[RequestId=10084, Status=Success] >09:04:46,319 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-6) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:04:47,625 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-89) Received create resource response: CreateResourceResponse[RequestId=10076, Status=Success] >09:05:20,421 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 1) dhcp180-146.mw.lab.eng.bos.redhat.com took [308]ms to reload cache for 1 agents >09:05:30,062 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-101) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:05:48,752 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-57) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:06:04,541 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-82) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:06:07,848 INFO [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/coregui]] (http-/0.0.0.0:7080-41) org.rhq.coregui.CoreGUI PackageVersionFileUploadServlet: file was uploaded: mock-ejb2.jar >09:06:20,133 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-1) Have not heard from agent [dev06-agent-16210] since [Thu Apr 03 09:00:37 EDT 2014]. Will be backfilled since we suspect it is down >09:06:20,444 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-1) Have not heard from agent [dhcp180-216-agent-16163] since [Thu Apr 03 09:01:02 EDT 2014]. Will be backfilled since we suspect it is down >09:06:20,589 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-1) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:06:25,351 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-2) Received delete resource response: DeleteResourceResponse[RequestId=10291, Status=Success] >09:06:30,472 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-24) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:06:33,091 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-97) Received create resource response: CreateResourceResponse[RequestId=10082, Status=Success] >09:06:33,166 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-58) Received create resource response: CreateResourceResponse[RequestId=10223, Status=Success] >09:06:45,781 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-96) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:06:50,326 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 3) dhcp180-146.mw.lab.eng.bos.redhat.com took [210]ms to reload cache for 2 agents >09:06:51,722 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-26) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:07:01,076 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-26) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:07:16,087 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-46) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:07:16,430 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-47) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:07:20,125 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-1) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:07:28,174 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-78) Received create resource response: CreateResourceResponse[RequestId=10123, Status=Success] >09:07:31,734 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-24) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:07:47,106 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-29) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:07:51,601 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 8) dhcp180-146.mw.lab.eng.bos.redhat.com took [701]ms to reload cache for 1 agents >09:08:02,461 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-33) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:08:14,816 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-55) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:08:17,712 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-33) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:08:20,135 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-4) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:08:22,910 INFO [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/coregui]] (http-/0.0.0.0:7080-65) org.rhq.coregui.CoreGUI PackageVersionFileUploadServlet: file was uploaded: mock-ejb3.jar >09:08:33,071 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-86) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:08:44,195 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-73) Received create resource response: CreateResourceResponse[RequestId=10222, Status=Success] >09:08:48,354 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-86) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:08:50,186 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 5) dhcp180-146.mw.lab.eng.bos.redhat.com took [72]ms to reload cache for 1 agents >09:09:01,877 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-31) Received delete resource response: DeleteResourceResponse[RequestId=10301, Status=Success] >09:09:03,756 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-46) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:09:15,766 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-107) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:09:19,179 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-84) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:09:20,238 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-1) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:09:20,865 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 4) dhcp180-146.mw.lab.eng.bos.redhat.com took [252]ms to reload cache for 1 agents >09:09:20,956 INFO [org.rhq.enterprise.server.scheduler.jobs.AsyncResourceDeleteJob] (RHQScheduler_Worker-5) Async resource deletion - 1 successful, 0 failed, took [912] ms >09:09:34,515 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-38) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:09:41,448 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-86) Received delete resource response: DeleteResourceResponse[RequestId=10321, Status=Success] >09:09:49,862 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-104) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:10:07,435 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-50) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:10:20,123 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-2) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:10:20,495 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-50) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:10:35,783 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-74) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:10:40,934 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-84) Received delete resource response: DeleteResourceResponse[RequestId=10064, Status=Success] >09:10:41,727 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-33) Received delete resource response: DeleteResourceResponse[RequestId=10331, Status=Success] >09:10:51,105 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-79) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:11:06,614 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-80) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:11:17,068 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (RHQScheduler_Worker-3) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[invokeOperation], targetInterfaceName=org.rhq.core.clientapi.agent.operation.OperationAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:11:17,076 ERROR [org.rhq.enterprise.server.operation.ResourceOperationJob] (RHQScheduler_Worker-3) Failed to execute scheduled operation [ResourceOperationSchedule: resource=[Resource[id=11748, uuid=16ade636-1e45-4fb9-8b27-1bbf2dded6db, type={JBossAS5}Script, key=twiddle.sh, name=twiddle.sh]],job-name=[rhq-resource-11748--1319569547-1396530368382], job-group=[rhq-resource-11748], operation-name=[execute], subject=[Subject[id=2,name=rhqadmin]], description=[]]: org.rhq.enterprise.server.authz.PermissionException: The session ID for user [rhqadmin] is invalid!: invocation: method=public org.rhq.core.domain.operation.OperationHistory org.rhq.enterprise.server.operation.OperationManagerBean.updateOperationHistory(org.rhq.core.domain.auth.Subject,org.rhq.core.domain.operation.OperationHistory),context-data={} > at org.rhq.enterprise.server.authz.RequiredPermissionsInterceptor.buildPermissionException(RequiredPermissionsInterceptor.java:164) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.enterprise.server.authz.RequiredPermissionsInterceptor.buildPermissionException(RequiredPermissionsInterceptor.java:160) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.enterprise.server.authz.RequiredPermissionsInterceptor.checkRequiredPermissions(RequiredPermissionsInterceptor.java:100) [rhq-server.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor91.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.jpa.interceptor.SBInvocationInterceptor.processInvocation(SBInvocationInterceptor.java:47) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.pool.PooledInstanceInterceptor.processInvocation(PooledInstanceInterceptor.java:51) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.operation.OperationManagerLocal$$$view149.updateOperationHistory(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.enterprise.server.operation.ResourceOperationJob.invokeOperationOnResource(ResourceOperationJob.java:214) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.enterprise.server.operation.ResourceOperationJob.execute(ResourceOperationJob.java:113) [rhq-server.jar:4.9.0.JON320GA] > at org.quartz.core.JobRunShell.run(JobRunShell.java:202) [quartz-1.6.5.jar:1.6.5] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:525) [quartz-1.6.5.jar:1.6.5] > >09:11:17,190 INFO [org.quartz.core.JobRunShell] (RHQScheduler_Worker-3) Job rhq-resource-11748.rhq-resource-11748--1319569547-1396530368382 threw a JobExecutionException: : org.quartz.JobExecutionException: Failed to execute scheduled operation [ResourceOperationSchedule: resource=[Resource[id=11748, uuid=16ade636-1e45-4fb9-8b27-1bbf2dded6db, type={JBossAS5}Script, key=twiddle.sh, name=twiddle.sh]],job-name=[rhq-resource-11748--1319569547-1396530368382], job-group=[rhq-resource-11748], operation-name=[execute], subject=[Subject[id=2,name=rhqadmin]], description=[]] [See nested exception: org.rhq.enterprise.server.authz.PermissionException: The session ID for user [rhqadmin] is invalid!: invocation: method=public org.rhq.core.domain.operation.OperationHistory org.rhq.enterprise.server.operation.OperationManagerBean.updateOperationHistory(org.rhq.core.domain.auth.Subject,org.rhq.core.domain.operation.OperationHistory),context-data={}] > at org.rhq.enterprise.server.operation.ResourceOperationJob.execute(ResourceOperationJob.java:131) [rhq-server.jar:4.9.0.JON320GA] > at org.quartz.core.JobRunShell.run(JobRunShell.java:202) [quartz-1.6.5.jar:1.6.5] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:525) [quartz-1.6.5.jar:1.6.5] >Caused by: org.rhq.enterprise.server.authz.PermissionException: The session ID for user [rhqadmin] is invalid!: invocation: method=public org.rhq.core.domain.operation.OperationHistory org.rhq.enterprise.server.operation.OperationManagerBean.updateOperationHistory(org.rhq.core.domain.auth.Subject,org.rhq.core.domain.operation.OperationHistory),context-data={} > at org.rhq.enterprise.server.authz.RequiredPermissionsInterceptor.buildPermissionException(RequiredPermissionsInterceptor.java:164) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.enterprise.server.authz.RequiredPermissionsInterceptor.buildPermissionException(RequiredPermissionsInterceptor.java:160) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.enterprise.server.authz.RequiredPermissionsInterceptor.checkRequiredPermissions(RequiredPermissionsInterceptor.java:100) [rhq-server.jar:4.9.0.JON320GA] > at sun.reflect.GeneratedMethodAccessor91.invoke(Unknown Source) [:1.6.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45] > at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45] > at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.jpa.interceptor.SBInvocationInterceptor.processInvocation(SBInvocationInterceptor.java:47) [jboss-as-jpa-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.pool.PooledInstanceInterceptor.processInvocation(PooledInstanceInterceptor.java:51) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:248) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:315) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:214) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.2.1.Final-redhat-10.jar:7.2.1.Final-redhat-10] > at org.rhq.enterprise.server.operation.OperationManagerLocal$$$view149.updateOperationHistory(Unknown Source) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.enterprise.server.operation.ResourceOperationJob.invokeOperationOnResource(ResourceOperationJob.java:214) [rhq-server.jar:4.9.0.JON320GA] > at org.rhq.enterprise.server.operation.ResourceOperationJob.execute(ResourceOperationJob.java:113) [rhq-server.jar:4.9.0.JON320GA] > ... 2 more > >09:11:17,791 INFO [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/coregui]] (http-/0.0.0.0:7080-84) org.rhq.coregui.CoreGUI PackageVersionFileUploadServlet: file was uploaded: mock-ejb3.jar >09:11:20,126 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-1) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:11:22,006 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-43) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:11:31,239 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-96) Received create resource response: CreateResourceResponse[RequestId=10311, Status=Success] >09:11:37,311 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-73) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:11:45,822 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-97) Received create resource response: CreateResourceResponse[RequestId=10321, Status=Success] >09:11:50,677 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 3) dhcp180-146.mw.lab.eng.bos.redhat.com took [409]ms to reload cache for 1 agents >09:11:52,581 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-73) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:12:07,885 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-73) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:12:20,167 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-5) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:12:20,422 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 8) dhcp180-146.mw.lab.eng.bos.redhat.com took [311]ms to reload cache for 1 agents >09:12:23,183 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-55) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:12:30,582 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-49) Received create resource response: CreateResourceResponse[RequestId=10331, Status=Success] >09:12:32,685 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-58) Received create resource response: CreateResourceResponse[RequestId=10007, Status=Success] >09:12:38,675 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-55) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:12:45,377 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-16) Received create resource response: CreateResourceResponse[RequestId=10174, Status=Success] >09:12:45,757 INFO [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/coregui]] (http-/0.0.0.0:7080-17) org.rhq.coregui.CoreGUI PackageVersionFileUploadServlet: file was uploaded: mock-war.war >09:12:50,893 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 5) dhcp180-146.mw.lab.eng.bos.redhat.com took [784]ms to reload cache for 3 agents >09:12:54,013 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-52) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:13:09,375 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-33) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:13:19,796 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-44) Received create resource response: CreateResourceResponse[RequestId=10333, Status=Success] >09:13:20,116 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-3) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:13:24,732 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-35) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:13:35,724 INFO [org.rhq.enterprise.server.core.CoreServerServiceImpl] (http-/0.0.0.0:7080-6) Got agent registration request for new agent: dev20-01-agent-16219[10.16.92.19:16219][4.9.0.JON320GA(734bd56)] >09:13:36,212 INFO [org.rhq.enterprise.server.core.CoreServerServiceImpl] (http-/0.0.0.0:7080-19) Agent [dev20-01-agent-16219][4.9.0.JON320GA(734bd56)] would like to connect to this server >09:13:36,682 INFO [org.rhq.enterprise.server.core.CoreServerServiceImpl] (http-/0.0.0.0:7080-19) Agent [dev20-01-agent-16219] has connected to this server at Thu Apr 03 09:13:36 EDT 2014 >09:13:41,215 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-59) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:13:41,244 INFO [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/coregui]] (http-/0.0.0.0:7080-47) org.rhq.coregui.CoreGUI PackageVersionFileUploadServlet: file was uploaded: mock-war.war >09:13:42,620 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-45) Received create resource response: CreateResourceResponse[RequestId=10028, Status=Success] >09:13:45,917 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-87) Received delete resource response: DeleteResourceResponse[RequestId=10311, Status=Success] >09:13:50,376 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 3) dhcp180-146.mw.lab.eng.bos.redhat.com took [222]ms to reload cache for 2 agents >09:13:55,432 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-59) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:14:10,744 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-65) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:14:20,150 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-4) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:14:20,604 INFO [org.rhq.enterprise.server.scheduler.jobs.AsyncResourceDeleteJob] (RHQScheduler_Worker-1) Async resource deletion - 1 successful, 0 failed, took [587] ms >09:14:26,228 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-43) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:14:41,528 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-79) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:14:55,029 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-48) Received delete resource response: DeleteResourceResponse[RequestId=10145, Status=Success] >09:14:56,894 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-79) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:14:56,962 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-52) Received create resource response: CreateResourceResponse[RequestId=10252, Status=Success] >09:15:11,627 INFO [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/coregui]] (http-/0.0.0.0:7080-50) org.rhq.coregui.CoreGUI PackageVersionFileUploadServlet: file was uploaded: mock-ejb2.jar >09:15:12,201 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-80) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:15:20,123 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-5) Have not heard from agent [dev98-agent-16216] since [Thu Apr 03 09:10:11 EDT 2014]. Will be backfilled since we suspect it is down >09:15:20,229 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-5) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:15:20,233 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 9) dhcp180-146.mw.lab.eng.bos.redhat.com took [110]ms to reload cache for 1 agents >09:15:27,526 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-63) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:15:39,822 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-51) Received create resource response: CreateResourceResponse[RequestId=10095, Status=Success] >09:15:43,024 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-42) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:15:50,256 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 6) dhcp180-146.mw.lab.eng.bos.redhat.com took [148]ms to reload cache for 1 agents >09:15:58,436 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-65) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:16:13,733 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-53) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:16:14,511 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-42) Received delete resource response: DeleteResourceResponse[RequestId=10216, Status=Success] >09:16:20,132 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-2) Have not heard from agent [dev20-01-agent-16218] since [Thu Apr 03 09:10:50 EDT 2014]. Will be backfilled since we suspect it is down >09:16:20,619 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-2) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:16:21,161 INFO [org.rhq.enterprise.server.util.concurrent.InventoryReportSerializer] (http-/0.0.0.0:7080-16) tid=3567; agent=dev20-01-agent-16219: releasing write lock after being locked for millis=6026 >09:16:21,408 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 5) dhcp180-146.mw.lab.eng.bos.redhat.com took [238]ms to reload cache for 2 agents >09:16:29,023 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-53) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:16:29,972 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-32) Received create resource response: CreateResourceResponse[RequestId=10332, Status=Success] >09:16:44,312 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-1) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:16:50,366 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 7) dhcp180-146.mw.lab.eng.bos.redhat.com took [258]ms to reload cache for 2 agents >09:16:53,682 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-90) Received create resource response: CreateResourceResponse[RequestId=10164, Status=Success] >09:16:59,641 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-62) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:17:11,389 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-7) Received delete resource response: DeleteResourceResponse[RequestId=10341, Status=Success] >09:17:14,995 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-59) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:17:17,640 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-44) Received delete resource response: DeleteResourceResponse[RequestId=10252, Status=Success] >09:17:20,119 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-4) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:17:20,165 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-4) Have not heard from agent [dev96-agent-16179] since [Thu Apr 03 09:11:47 EDT 2014]. Will be backfilled since we suspect it is down >09:17:20,223 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 1) dhcp180-146.mw.lab.eng.bos.redhat.com took [104]ms to reload cache for 1 agents >09:17:30,391 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-80) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:17:31,468 INFO [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/coregui]] (http-/0.0.0.0:7080-24) org.rhq.coregui.CoreGUI PackageVersionFileUploadServlet: file was uploaded: mock-ejb3.jar >09:17:45,618 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-13) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:18:00,993 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-59) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:18:16,351 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-59) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:18:20,128 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-3) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:18:31,702 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-74) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:18:38,538 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-16) Received delete resource response: DeleteResourceResponse[RequestId=10282, Status=Success] >09:18:47,001 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-59) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:19:02,268 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-59) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:19:07,596 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-85) Received delete resource response: DeleteResourceResponse[RequestId=10183, Status=Success] >09:19:09,991 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-82) Received create resource response: CreateResourceResponse[RequestId=10213, Status=Success] >09:19:17,667 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-55) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:19:20,176 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-5) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:19:20,636 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 1) dhcp180-146.mw.lab.eng.bos.redhat.com took [215]ms to reload cache for 1 agents >09:19:21,544 INFO [org.rhq.enterprise.server.scheduler.jobs.AsyncResourceDeleteJob] (RHQScheduler_Worker-4) Async resource deletion - 3 successful, 0 failed, took [1531] ms >09:19:34,316 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-85) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:19:48,243 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-85) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:19:50,232 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 10) dhcp180-146.mw.lab.eng.bos.redhat.com took [91]ms to reload cache for 1 agents >09:20:03,702 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-21) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:20:05,856 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-35) Received create resource response: CreateResourceResponse[RequestId=10070, Status=Success] >09:20:18,997 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-29) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:20:20,157 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-1) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:20:20,342 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 6) dhcp180-146.mw.lab.eng.bos.redhat.com took [217]ms to reload cache for 1 agents >09:20:33,755 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-82) Received create resource response: CreateResourceResponse[RequestId=10055, Status=Success] >09:20:34,326 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-29) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:20:49,442 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-62) Received create resource response: CreateResourceResponse[RequestId=10069, Status=Success] >09:20:49,597 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-29) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:20:50,286 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 10) dhcp180-146.mw.lab.eng.bos.redhat.com took [174]ms to reload cache for 1 agents >09:21:04,911 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-29) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:21:20,130 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-3) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:21:20,274 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 7) dhcp180-146.mw.lab.eng.bos.redhat.com took [143]ms to reload cache for 1 agents >09:21:20,278 INFO [org.rhq.enterprise.server.operation.OperationManagerBean] (RHQScheduler_Worker-1) Operation execution seems to have been orphaned - timing it out: ResourceOperationHistory: resource=[Resource[id=11748, uuid=16ade636-1e45-4fb9-8b27-1bbf2dded6db, type={JBossAS5}Script, key=twiddle.sh, name=twiddle.sh, parent=EAP 10.16.92.108:1099 default]], group-history=[null], id=[10285], job-name=[rhq-resource-11748--1319569547-1396530368382], job-group=[rhq-resource-11748], status=[In Progress], subject-name=[rhqadmin], ctime=[Thu Apr 03 09:06:08 EDT 2014], mtime=[Thu Apr 03 09:06:08 EDT 2014], duration-millis=[911509], error-message=[null] >09:21:20,382 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-29) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:21:31,409 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-3) Received delete resource response: DeleteResourceResponse[RequestId=10065, Status=Success] >09:22:01,941 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-1) Received delete resource response: DeleteResourceResponse[RequestId=10104, Status=Success] >09:22:06,064 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-42) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:22:12,202 INFO [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/coregui]] (http-/0.0.0.0:7080-48) org.rhq.coregui.CoreGUI PackageVersionFileUploadServlet: file was uploaded: mock-war.war >09:22:20,117 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-3) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:22:29,047 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-78) Received delete resource response: DeleteResourceResponse[RequestId=10105, Status=Success] >09:22:41,745 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-107) Received create resource response: CreateResourceResponse[RequestId=10243, Status=Success] >09:22:50,577 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 1) dhcp180-146.mw.lab.eng.bos.redhat.com took [465]ms to reload cache for 1 agents >09:23:11,775 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-42) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:23:20,128 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-3) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:23:26,796 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-42) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:23:41,785 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-42) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:23:44,102 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-47) Received delete resource response: DeleteResourceResponse[RequestId=10242, Status=Success] >09:23:49,131 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-21) Received delete resource response: DeleteResourceResponse[RequestId=10005, Status=Success] >09:23:56,781 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-9) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:24:11,771 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-9) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:24:20,159 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-3) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:24:20,845 INFO [org.rhq.enterprise.server.scheduler.jobs.AsyncResourceDeleteJob] (RHQScheduler_Worker-1) Async resource deletion - 4 successful, 0 failed, took [824] ms >09:24:26,788 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-9) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:24:31,632 WARN [org.rhq.helpers.rtfilter.filter.RtFilter] (http-/0.0.0.0:7080-49) Response time log '/qa/hudson_workspace/workspace/jon-32-server-install/dhcp180-146.mw.lab.eng.bos.redhat.com/jon-server-3.2.0.GA/logs/rt/10.16.180.234_coregui_rt.log' has exceeded maximum file size (5242880 bytes) - truncating it... >09:24:41,791 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-9) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:24:56,772 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-9) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:24:57,559 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-79) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=bCVEQpzRqk7UZ3APJ5ZaW5wm4EgvglQQBuu9z9KFdlAMx9UIROk9fRWkhW+7FInWtgc=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:25:12,496 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-106) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:25:20,140 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-1) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:25:26,853 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-106) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:25:41,856 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-16) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:25:56,825 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-16) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:25:58,668 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-31) Received create resource response: CreateResourceResponse[RequestId=10203, Status=Success] >09:26:03,133 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-63) Received create resource response: CreateResourceResponse[RequestId=10029, Status=Success] >09:26:11,912 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-16) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:26:20,139 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-1) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:26:20,478 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 6) dhcp180-146.mw.lab.eng.bos.redhat.com took [362]ms to reload cache for 2 agents >09:26:26,843 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-62) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:26:41,820 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-16) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:26:52,066 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-24) Received delete resource response: DeleteResourceResponse[RequestId=10044, Status=Success] >09:26:56,821 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-16) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:27:11,866 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-16) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:27:20,207 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-3) Have not heard from agent [dev104-agent-16291] since [Thu Apr 03 09:21:33 EDT 2014]. Will be backfilled since we suspect it is down >09:27:20,599 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-3) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:27:26,920 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-16) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:27:41,937 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-16) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:27:50,216 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 5) dhcp180-146.mw.lab.eng.bos.redhat.com took [104]ms to reload cache for 1 agents >09:27:56,864 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-50) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:28:01,177 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-38) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:28:07,649 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (RHQScheduler_Worker-4) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=2000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:28:07,649 ERROR [org.rhq.enterprise.server.measurement.MeasurementScheduleManagerBean] (RHQScheduler_Worker-4) Could not send measurement schedule updates to agent[id=10041], marking resources for update; agent ping failed >09:28:14,779 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-52) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:28:20,185 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-1) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:28:23,697 INFO [org.rhq.enterprise.server.util.concurrent.InventoryReportSerializer] (http-/0.0.0.0:7080-78) tid=3948; agent=dev20-01-agent-16219: releasing write lock after being locked for millis=7363 >09:28:23,788 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 5) dhcp180-146.mw.lab.eng.bos.redhat.com took [122]ms to reload cache for 1 agents >09:28:25,381 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-29) Received create resource response: CreateResourceResponse[RequestId=10153, Status=Failure] >09:28:27,990 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-98) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:28:43,049 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-98) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:28:50,401 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 10) dhcp180-146.mw.lab.eng.bos.redhat.com took [292]ms to reload cache for 1 agents >09:28:58,049 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-98) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:29:01,150 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-58) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:29:13,027 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-58) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:29:20,377 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-4) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:29:20,513 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-4) Have not heard from agent [dev103-agent-16199] since [Thu Apr 03 09:23:38 EDT 2014]. Will be backfilled since we suspect it is down >09:29:22,414 INFO [org.rhq.enterprise.server.scheduler.jobs.AsyncResourceDeleteJob] (RHQScheduler_Worker-3) Async resource deletion - 2 successful, 0 failed, took [2306] ms >09:29:30,117 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-56) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:29:43,111 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-56) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:29:58,113 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-98) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:30:01,114 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-73) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:30:13,125 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-98) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:30:20,125 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-2) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:30:28,121 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-98) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:30:34,237 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-91) Received delete resource response: DeleteResourceResponse[RequestId=10243, Status=Success] >09:30:43,110 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-98) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:30:58,154 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-98) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:31:00,775 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-106) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:31:13,157 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-98) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:31:20,037 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-12) Received delete resource response: DeleteResourceResponse[RequestId=10312, Status=Success] >09:31:20,219 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-5) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:31:28,170 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-95) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:31:43,197 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-26) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:31:58,200 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-62) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:31:59,893 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-21) Received delete resource response: DeleteResourceResponse[RequestId=10351, Status=Success] >09:32:03,414 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-65) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:32:04,842 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-53) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:32:13,153 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-53) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:32:20,129 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-2) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:32:28,272 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-91) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:32:30,152 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-103) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=bCVEQpzRqk7UZ3APJ5ZaW5wm4EgvglQQBuu9z9KFdlAMx9UIROk9fRWkhW+7FInWtgc=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:32:33,615 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (RHQScheduler_Worker-3) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=2000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:32:33,616 ERROR [org.rhq.enterprise.server.measurement.MeasurementScheduleManagerBean] (RHQScheduler_Worker-3) Could not send measurement schedule updates to agent[id=10041], marking resources for update; agent ping failed >09:32:43,432 INFO [org.rhq.enterprise.server.util.concurrent.InventoryReportSerializer] (http-/0.0.0.0:7080-92) tid=4148; agent=dev20-01-agent-16219: releasing write lock after being locked for millis=6437 >09:32:44,404 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-55) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:32:46,968 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-34) Received create resource response: CreateResourceResponse[RequestId=10194, Status=Success] >09:32:51,229 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 2) dhcp180-146.mw.lab.eng.bos.redhat.com took [1117]ms to reload cache for 2 agents >09:32:52,154 INFO [org.rhq.enterprise.server.util.concurrent.InventoryReportSerializer] (http-/0.0.0.0:7080-40) tid=3667; agent=dev20-01-agent-16219: releasing write lock after being locked for millis=6431 >09:32:53,375 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-52) Received delete resource response: DeleteResourceResponse[RequestId=10322, Status=Success] >09:32:59,399 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-21) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:33:14,350 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-21) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:33:19,022 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-64) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:33:20,144 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-3) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:33:20,331 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 6) dhcp180-146.mw.lab.eng.bos.redhat.com took [211]ms to reload cache for 1 agents >09:33:29,359 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-64) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:33:44,360 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-33) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:33:48,365 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-71) Received create resource response: CreateResourceResponse[RequestId=10302, Status=Success] >09:33:49,422 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-87) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=YZW3MuIdz8NBK8Ah9EI4Bx5VuyXcU0b6yyoKP5PUhfkyELpWBQzn/yE8ZwLs2+EGhO4=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:33:59,388 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-21) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:34:14,374 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-21) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:34:20,161 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-2) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:34:20,481 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 6) dhcp180-146.mw.lab.eng.bos.redhat.com took [119]ms to reload cache for 1 agents >09:34:23,999 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-7) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:34:29,395 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-7) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:34:31,442 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-64) Received delete resource response: DeleteResourceResponse[RequestId=10006, Status=Success] >09:34:33,432 INFO [org.rhq.enterprise.server.scheduler.jobs.AsyncResourceDeleteJob] (RHQScheduler_Worker-1) Async resource deletion - 49 successful, 0 failed, took [13377] ms >09:34:44,412 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-7) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:34:59,365 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-7) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:35:03,013 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-13) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:35:15,758 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-13) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:35:20,282 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-1) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:35:29,461 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-38) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:35:44,410 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-38) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:35:57,846 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-29) Received delete resource response: DeleteResourceResponse[RequestId=10095, Status=Success] >09:35:59,437 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-38) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:36:03,111 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-47) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:36:10,372 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-101) Received create resource response: CreateResourceResponse[RequestId=10035, Status=Success] >09:36:14,480 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-38) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:36:20,343 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-1) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:36:20,590 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 5) dhcp180-146.mw.lab.eng.bos.redhat.com took [469]ms to reload cache for 1 agents >09:36:26,261 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-9) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:36:29,412 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-89) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:36:44,421 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-89) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:36:59,520 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-89) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:37:14,687 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-89) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:37:20,335 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-3) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:37:21,440 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-82) Received delete resource response: DeleteResourceResponse[RequestId=10113, Status=Success] >09:37:29,424 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-89) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:37:34,497 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-9) Received create resource response: CreateResourceResponse[RequestId=10244, Status=Failure] >09:37:43,071 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-23) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:37:44,461 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-39) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:37:59,479 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-62) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:38:12,600 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-89) Received delete resource response: DeleteResourceResponse[RequestId=10152, Status=Success] >09:38:14,510 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-62) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:38:20,127 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-2) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:38:20,147 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-2) Have not heard from agent [dhcp180-124-agent-16163] since [Thu Apr 03 09:32:53 EDT 2014]. Will be backfilled since we suspect it is down >09:38:29,453 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-16) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:38:33,201 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-87) Received delete resource response: DeleteResourceResponse[RequestId=10313, Status=Success] >09:38:44,479 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-16) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:38:45,048 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-43) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:38:54,396 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-79) Received create resource response: CreateResourceResponse[RequestId=10334, Status=Success] >09:38:59,457 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-16) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:39:03,523 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-81) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:39:14,496 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-81) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:39:20,267 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-5) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:39:20,727 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 6) dhcp180-146.mw.lab.eng.bos.redhat.com took [363]ms to reload cache for 1 agents >09:39:21,309 INFO [org.rhq.enterprise.server.scheduler.jobs.AsyncResourceDeleteJob] (RHQScheduler_Worker-4) Async resource deletion - 1 successful, 0 failed, took [1230] ms >09:39:30,701 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-87) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:39:44,461 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-87) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:39:59,528 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-95) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:40:03,291 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-41) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:40:14,535 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-95) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:40:20,130 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-1) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:40:29,181 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-107) Received create resource response: CreateResourceResponse[RequestId=10341, Status=Success] >09:40:29,525 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-95) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:40:44,536 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-59) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:40:50,236 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 10) dhcp180-146.mw.lab.eng.bos.redhat.com took [126]ms to reload cache for 1 agents >09:40:59,544 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-59) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:41:04,069 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-48) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:41:14,516 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-48) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:41:20,128 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-5) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:41:29,553 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-23) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:41:30,733 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-92) Received create resource response: CreateResourceResponse[RequestId=10008, Status=Success] >09:41:31,635 ERROR [org.rhq.enterprise.server.alert.engine.internal.AlertConditionCacheCoordinator] (EJB default - 2) Could not find agent for scheduleId = 46880 >09:41:44,514 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-23) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:41:50,328 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 4) dhcp180-146.mw.lab.eng.bos.redhat.com took [217]ms to reload cache for 1 agents >09:41:59,334 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-13) Received create resource response: CreateResourceResponse[RequestId=10124, Status=Success] >09:41:59,709 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-23) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:42:04,490 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-42) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:42:14,652 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-23) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:42:21,454 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-4) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:42:21,681 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 7) dhcp180-146.mw.lab.eng.bos.redhat.com took [377]ms to reload cache for 1 agents >09:42:29,629 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-23) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:42:44,622 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-59) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:42:50,909 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-103) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:42:59,645 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-103) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:42:59,845 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-88) Received create resource response: CreateResourceResponse[RequestId=10303, Status=Success] >09:43:03,422 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-36) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:43:14,624 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-103) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:43:20,170 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-4) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:43:20,319 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 6) dhcp180-146.mw.lab.eng.bos.redhat.com took [216]ms to reload cache for 1 agents >09:43:29,635 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-103) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:43:44,628 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-58) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:43:59,694 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-58) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:44:03,849 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-43) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:44:04,696 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-64) Received create resource response: CreateResourceResponse[RequestId=10036, Status=Success] >09:44:14,711 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-58) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:44:20,210 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-2) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:44:20,698 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 8) dhcp180-146.mw.lab.eng.bos.redhat.com took [327]ms to reload cache for 1 agents >09:44:29,691 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-58) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:44:36,256 INFO [org.rhq.enterprise.server.scheduler.jobs.AsyncResourceDeleteJob] (RHQScheduler_Worker-4) Async resource deletion - 47 successful, 0 failed, took [16195] ms >09:44:44,683 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-30) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:44:59,741 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-47) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:45:03,533 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-53) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:45:16,011 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-47) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:45:17,071 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-59) Received create resource response: CreateResourceResponse[RequestId=10204, Status=Success] >09:45:18,824 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-21) Received create resource response: CreateResourceResponse[RequestId=10047, Status=Success] >09:45:20,170 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-2) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:45:29,686 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-21) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:45:44,775 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-21) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:45:50,381 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 3) dhcp180-146.mw.lab.eng.bos.redhat.com took [279]ms to reload cache for 2 agents >09:45:59,741 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-21) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:46:03,534 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-107) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:46:14,884 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-48) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:46:20,167 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-1) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:46:20,512 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 5) dhcp180-146.mw.lab.eng.bos.redhat.com took [380]ms to reload cache for 1 agents >09:46:29,783 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-48) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:46:44,776 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-48) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:46:50,365 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 7) dhcp180-146.mw.lab.eng.bos.redhat.com took [255]ms to reload cache for 1 agents >09:46:51,172 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-41) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:46:58,055 INFO [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/coregui]] (http-/0.0.0.0:7080-90) org.rhq.coregui.CoreGUI PackageVersionFileUploadServlet: file was uploaded: mock-ejb2.jar >09:46:59,747 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-41) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:47:03,643 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-62) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:47:14,756 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-62) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:47:17,361 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-101) Received create resource response: CreateResourceResponse[RequestId=10102, Status=Success] >09:47:19,606 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-44) Received create resource response: CreateResourceResponse[RequestId=10245, Status=Failure] >09:47:20,143 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-3) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:47:20,247 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 6) dhcp180-146.mw.lab.eng.bos.redhat.com took [123]ms to reload cache for 1 agents >09:47:29,745 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-71) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:47:35,848 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-7) Received create resource response: CreateResourceResponse[RequestId=10165, Status=Success] >09:47:44,763 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-71) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:47:50,185 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 6) dhcp180-146.mw.lab.eng.bos.redhat.com took [71]ms to reload cache for 1 agents >09:47:53,715 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-92) Received create resource response: CreateResourceResponse[RequestId=10253, Status=Success] >09:47:59,842 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-71) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:48:03,792 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-31) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:48:14,844 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-31) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:48:20,138 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-1) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:48:20,249 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 3) dhcp180-146.mw.lab.eng.bos.redhat.com took [134]ms to reload cache for 1 agents >09:48:29,802 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-71) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:48:44,850 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-71) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:48:59,405 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-12) Received create resource response: CreateResourceResponse[RequestId=10103, Status=Success] >09:48:59,833 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-82) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:49:03,657 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-52) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:49:14,797 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-82) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:49:20,215 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-4) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:49:20,594 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 10) dhcp180-146.mw.lab.eng.bos.redhat.com took [232]ms to reload cache for 1 agents >09:49:29,819 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-82) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:49:44,792 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-87) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:49:59,848 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-87) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:50:04,011 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-51) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:50:12,321 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-65) Received delete resource response: DeleteResourceResponse[RequestId=10302, Status=Success] >09:50:14,862 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-87) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:50:17,925 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-90) Received create resource response: CreateResourceResponse[RequestId=10096, Status=Success] >09:50:20,154 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-1) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:50:20,252 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 10) dhcp180-146.mw.lab.eng.bos.redhat.com took [133]ms to reload cache for 1 agents >09:50:29,842 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-87) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:50:44,894 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-53) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:50:59,856 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-98) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:51:03,830 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-30) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:51:14,917 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-98) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:51:20,171 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-1) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:51:29,839 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-103) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:51:46,478 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-49) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:52:00,006 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-49) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:52:05,185 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-47) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:52:14,914 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-47) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:52:20,147 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-5) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:52:25,365 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-33) Received delete resource response: DeleteResourceResponse[RequestId=10106, Status=Success] >09:52:29,915 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-17) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:52:33,947 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-12) Received delete resource response: DeleteResourceResponse[RequestId=10232, Status=Success] >09:52:44,891 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-17) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:52:59,895 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-17) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:53:04,716 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-34) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:53:11,932 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-46) Received delete resource response: DeleteResourceResponse[RequestId=10263, Status=Success] >09:53:14,926 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-17) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:53:20,129 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-4) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:53:29,910 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-17) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:53:44,890 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-7) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:53:53,308 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-33) Received delete resource response: DeleteResourceResponse[RequestId=10134, Status=Success] >09:53:59,977 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-7) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:54:04,300 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-48) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:54:14,980 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-7) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:54:20,183 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-2) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:54:29,950 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-7) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:54:44,946 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-7) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:54:59,998 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-7) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:55:01,316 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-21) Received create resource response: CreateResourceResponse[RequestId=10175, Status=Success] >09:55:03,638 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-102) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:55:06,148 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-82) Received delete resource response: DeleteResourceResponse[RequestId=10026, Status=Success] >09:55:14,972 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-7) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:55:20,135 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-4) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:55:20,316 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 4) dhcp180-146.mw.lab.eng.bos.redhat.com took [211]ms to reload cache for 1 agents >09:55:29,955 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-7) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:55:44,947 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-20) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:55:56,602 INFO [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/coregui]] (http-/0.0.0.0:7080-49) org.rhq.coregui.CoreGUI PackageVersionFileUploadServlet: file was uploaded: mock-ejb3.jar >09:56:00,011 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-20) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:56:04,045 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-16) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:56:14,996 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-20) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:56:17,723 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-1) Received create resource response: CreateResourceResponse[RequestId=10077, Status=Failure] >09:56:20,139 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-2) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:56:30,005 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-20) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:56:45,000 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-20) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:57:00,009 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-64) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:57:04,488 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-71) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:57:12,973 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-24) Received create resource response: CreateResourceResponse[RequestId=10078, Status=Success] >09:57:15,015 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-64) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:57:20,162 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-5) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:57:20,301 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 3) dhcp180-146.mw.lab.eng.bos.redhat.com took [183]ms to reload cache for 1 agents >09:57:28,114 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-95) Received create resource response: CreateResourceResponse[RequestId=10283, Status=Success] >09:57:30,000 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-20) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:57:45,010 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-63) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:57:50,301 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 9) dhcp180-146.mw.lab.eng.bos.redhat.com took [195]ms to reload cache for 1 agents >09:58:00,084 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-63) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:58:03,694 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-13) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:58:15,097 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-63) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:58:20,163 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-1) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:58:20,219 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-1) Have not heard from agent [dev59-agent-16200] since [Thu Apr 03 09:52:38 EDT 2014]. Will be backfilled since we suspect it is down >09:58:30,142 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-63) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:58:45,082 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-1) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:59:00,120 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-1) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:59:04,464 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-55) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >09:59:04,963 INFO [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/coregui]] (http-/0.0.0.0:7080-62) org.rhq.coregui.CoreGUI PackageVersionFileUploadServlet: file was uploaded: mock-ejb2.jar >09:59:15,109 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-1) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:59:20,244 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-4) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >09:59:27,813 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-38) Received create resource response: CreateResourceResponse[RequestId=10184, Status=Success] >09:59:30,111 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-1) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:59:35,619 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-32) Received delete resource response: DeleteResourceResponse[RequestId=10007, Status=Success] >09:59:45,112 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-107) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >09:59:50,245 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 4) dhcp180-146.mw.lab.eng.bos.redhat.com took [132]ms to reload cache for 1 agents >10:00:00,026 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-5) Data Purge Job STARTING >10:00:00,035 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-5) Measurement data compression starting at Thu Apr 03 10:00:00 EDT 2014 >10:00:00,104 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-107) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:00:04,678 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-19) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >10:00:06,298 INFO [org.rhq.server.metrics.MetricsServer] (RHQScheduler_Worker-5) Finished computing aggregates for table [raw_metrics]6263 ms >10:00:07,619 INFO [org.rhq.server.metrics.MetricsServer] (RHQScheduler_Worker-5) Finished computing aggregates for table [one_hour_metrics] 0 ms >10:00:07,619 INFO [org.rhq.server.metrics.MetricsServer] (RHQScheduler_Worker-5) Finished computing aggregates for table [six_hour_metrics] 0 ms >10:00:07,619 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-5) Measurement data compression completed in [7584]ms >10:00:07,620 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-5) Measurement calltime data purge starting at Thu Apr 03 10:00:07 EDT 2014 >10:00:07,620 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-5) Purging calltime data that is older than Mon Mar 03 09:00:07 EST 2014 >10:00:07,627 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-5) Calltime purged [0] - completed in [7]ms >10:00:07,627 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-5) Event data purge starting at Thu Apr 03 10:00:07 EDT 2014 >10:00:07,627 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-5) Purging event data older than Thu Mar 20 10:00:07 EDT 2014 >10:00:07,675 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-5) Event data purged [0] - completed in [48]ms >10:00:07,676 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-5) Alert data purge starting at Thu Apr 03 10:00:07 EDT 2014 >10:00:07,676 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-5) Purging alert data older than Mon Mar 03 09:00:07 EST 2014 >10:00:07,811 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-5) Alert data purged [0] - completed in [135]ms >10:00:07,812 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-5) Alert definition unused purge starting at Thu Apr 03 10:00:07 EDT 2014 >10:00:07,850 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-5) Alert definitions purged [0] - completed in [39]ms >10:00:07,850 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-5) Alert condition orphan purge starting at Thu Apr 03 10:00:07 EDT 2014 >10:00:07,889 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-5) Purged [0] orphan alert conditions - completed in [39]ms >10:00:07,889 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-5) Alert notification orphan purge starting at Thu Apr 03 10:00:07 EDT 2014 >10:00:07,910 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-5) Purged [0] orphan alert notifications - completed in [21]ms >10:00:07,910 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-5) Trait data purge starting at Thu Apr 03 10:00:07 EDT 2014 >10:00:07,910 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-5) Purging traits that are older than Wed Apr 03 10:00:07 EDT 2013 >10:00:07,933 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-5) Traits data purged [0] - completed in [23]ms >10:00:07,933 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-5) Availability data purge starting at Thu Apr 03 10:00:07 EDT 2014 >10:00:07,933 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-5) Purging availablities that are older than Wed Apr 03 10:00:07 EDT 2013 >10:00:07,957 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-5) Availability data purged [0] - completed in [24]ms >10:00:07,958 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-5) Drift file orphan purge starting at Thu Apr 03 10:00:07 EDT 2014 >10:00:07,958 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-5) Purging orphaned drift files older than Mon Mar 03 09:00:07 EST 2014 >10:00:08,003 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-5) Purged [0] orphaned drift files - completed in [45]ms >10:00:08,018 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-5) Database maintenance starting at Thu Apr 03 10:00:08 EDT 2014 >10:00:08,018 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-5) Performing hourly database maintenance >10:00:15,247 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-19) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:00:17,028 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-5) Database maintenance completed in [9010]ms >10:00:17,028 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-5) Auto-calculation of baselines starting at Thu Apr 03 10:00:17 EDT 2014 >10:00:17,034 INFO [org.rhq.enterprise.server.measurement.MeasurementBaselineManagerBean] (RHQScheduler_Worker-5) Calculating auto baselines >10:00:17,034 INFO [org.rhq.enterprise.server.measurement.MeasurementBaselineManagerBean] (RHQScheduler_Worker-5) Deleting baselines computations older than Mon Mar 31 10:00:17 EDT 2014 >10:00:17,034 INFO [org.rhq.enterprise.server.measurement.MeasurementBaselineManagerBean] (RHQScheduler_Worker-5) Inserting new baselines using last 7 days of 1H data >10:00:17,041 INFO [org.rhq.enterprise.server.measurement.MeasurementBaselineManagerBean] (RHQScheduler_Worker-5) Removed [0] old baselines - they will now be recalculated (6)ms >10:00:20,266 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-1) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >10:00:30,240 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-19) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:00:33,040 INFO [org.rhq.enterprise.server.measurement.MeasurementBaselineManagerBean] (RHQScheduler_Worker-5) Calculated and inserted [5847] new baselines. (15998)ms >10:00:33,182 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-5) Auto-calculation of baselines completed in [16154]ms >10:00:33,183 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-5) Auto-calculation of OOBs starting >10:00:33,252 INFO [org.rhq.enterprise.server.measurement.MeasurementOOBManagerBean] (RHQScheduler_Worker-5) Removed [0] outdated OOBs >10:00:33,412 INFO [org.rhq.enterprise.server.measurement.MeasurementOOBManagerBean] (RHQScheduler_Worker-5) Computing OOBs >10:00:45,234 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-19) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:01:00,281 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-95) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:01:02,979 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 8) dhcp180-146.mw.lab.eng.bos.redhat.com took [12753]ms to reload cache for 26 agents >10:01:04,194 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-73) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >10:01:15,212 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-95) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:01:16,261 INFO [org.rhq.enterprise.server.measurement.MeasurementOOBManagerBean] (RHQScheduler_Worker-5) Finished calculating 235 OOBs in 42848 ms >10:01:16,262 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-5) Auto-calculation of OOBs completed in [43079]ms >10:01:16,262 INFO [org.rhq.enterprise.server.scheduler.jobs.DataPurgeJob] (RHQScheduler_Worker-5) Data Purge Job FINISHED [76236]ms >10:01:20,127 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-1) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >10:01:20,462 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-33) Received delete resource response: DeleteResourceResponse[RequestId=10202, Status=Success] >10:01:24,299 INFO [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/coregui]] (http-/0.0.0.0:7080-30) org.rhq.coregui.CoreGUI PackageVersionFileUploadServlet: file was uploaded: mock-ejb3.jar >10:01:30,161 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-95) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:01:45,187 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-41) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:02:00,224 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-41) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:02:03,973 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-82) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >10:02:15,190 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-41) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:02:20,152 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-3) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >10:02:30,216 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-41) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:02:38,644 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-16) Received delete resource response: DeleteResourceResponse[RequestId=10052, Status=Success] >10:02:45,168 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-87) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:03:00,172 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-87) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:03:04,401 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-29) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >10:03:15,168 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-87) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:03:20,167 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-5) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >10:03:30,216 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-89) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:03:46,882 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-19) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:04:00,193 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-19) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:04:04,443 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-98) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >10:04:04,514 INFO [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/coregui]] (http-/0.0.0.0:7080-73) org.rhq.coregui.CoreGUI PackageVersionFileUploadServlet: file was uploaded: mock-war.war >10:04:10,548 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-16) Received create resource response: CreateResourceResponse[RequestId=10351, Status=Success] >10:04:15,217 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-98) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:04:15,792 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-95) Received delete resource response: DeleteResourceResponse[RequestId=10163, Status=Success] >10:04:20,166 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-2) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >10:04:20,460 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 9) dhcp180-146.mw.lab.eng.bos.redhat.com took [83]ms to reload cache for 1 agents >10:04:25,736 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-101) Received create resource response: CreateResourceResponse[RequestId=10292, Status=Failure] >10:04:26,284 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-49) Received delete resource response: DeleteResourceResponse[RequestId=10283, Status=Success] >10:04:30,263 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-46) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:04:45,238 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-46) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:05:00,224 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-46) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:05:04,156 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-81) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >10:05:10,371 INFO [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/coregui]] (http-/0.0.0.0:7080-74) org.rhq.coregui.CoreGUI PackageVersionFileUploadServlet: file was uploaded: mock-war.war >10:05:15,228 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-46) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:05:20,127 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-2) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >10:05:30,231 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-46) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:05:45,220 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-101) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:06:00,233 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-101) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:06:04,910 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-3) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >10:06:15,289 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-3) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:06:20,246 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-1) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >10:06:30,287 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-3) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:06:45,318 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-3) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:07:00,291 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-3) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:07:04,467 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-56) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >10:07:12,053 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-33) Received delete resource response: DeleteResourceResponse[RequestId=10361, Status=Success] >10:07:15,271 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-56) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:07:20,136 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-5) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >10:07:25,127 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-92) Received create resource response: CreateResourceResponse[RequestId=10233, Status=Success] >10:07:26,245 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-63) Received create resource response: CreateResourceResponse[RequestId=10246, Status=Success] >10:07:30,278 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-20) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:07:44,379 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-33) Received delete resource response: DeleteResourceResponse[RequestId=10362, Status=Success] >10:07:45,286 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-20) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:07:50,258 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 2) dhcp180-146.mw.lab.eng.bos.redhat.com took [136]ms to reload cache for 2 agents >10:08:00,285 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-20) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:08:04,002 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-59) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >10:08:15,373 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-59) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:08:20,140 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-2) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >10:08:30,333 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-42) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:08:45,342 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-41) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:08:51,759 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-23) Received create resource response: CreateResourceResponse[RequestId=10166, Status=Success] >10:08:56,260 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-41) Received delete resource response: DeleteResourceResponse[RequestId=10085, Status=Success] >10:09:00,337 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-42) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:09:04,278 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-71) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >10:09:15,342 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-42) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:09:20,159 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-3) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >10:09:20,419 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 6) dhcp180-146.mw.lab.eng.bos.redhat.com took [57]ms to reload cache for 1 agents >10:09:30,335 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-42) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:09:45,339 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-92) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:10:00,351 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-92) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:10:04,078 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-95) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >10:10:15,398 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-95) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:10:20,128 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-5) Have not heard from agent [dev117-agent-16219] since [Thu Apr 03 10:04:44 EDT 2014]. Will be backfilled since we suspect it is down >10:10:20,367 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-5) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >10:10:30,407 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-95) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:10:45,389 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-95) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:11:00,391 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-95) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:11:04,148 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-45) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >10:11:15,398 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-45) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:11:20,126 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-5) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >10:11:30,388 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-19) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:11:45,389 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-19) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:12:00,400 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-19) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:12:04,316 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-24) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >10:12:20,140 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-4) Have not heard from agent [soa3-agent-16226] since [Thu Apr 03 10:07:14 EDT 2014]. Will be backfilled since we suspect it is down >10:12:20,316 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-4) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >10:12:28,230 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-51) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:12:43,193 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-23) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:12:58,204 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-23) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:13:04,175 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-32) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >10:13:13,199 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-32) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:13:14,762 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-29) Received create resource response: CreateResourceResponse[RequestId=10195, Status=Success] >10:13:20,152 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-4) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >10:13:20,219 INFO [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 8) dhcp180-146.mw.lab.eng.bos.redhat.com took [107]ms to reload cache for 1 agents >10:13:28,197 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-97) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:13:33,500 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-32) Received delete resource response: DeleteResourceResponse[RequestId=10153, Status=Success] >10:13:43,202 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-97) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:13:50,990 INFO [org.rhq.enterprise.server.resource.ResourceFactoryServerServiceImpl] (http-/0.0.0.0:7080-20) Received create resource response: CreateResourceResponse[RequestId=10361, Status=Success] >10:13:58,187 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-97) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:14:04,189 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-88) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[getRealTimeMeasurementValue], targetInterfaceName=org.rhq.core.clientapi.agent.measurement.MeasurementAgentService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] -> java.lang.reflect.InvocationTargetException:null -> java.net.SocketException:Connection reset. Cause: org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [socket://10.16.92.107:16223/?backlog=200&clientMaxPoolSize=304&enableTcpNoDelay=true&maxPoolSize=303&numAcceptThreads=1&rhq.communications.connector.rhqtype=agent&socketTimeout=60000] >10:14:13,205 ERROR [org.rhq.enterprise.communications.command.client.ClientCommandSenderTask] (http-/0.0.0.0:7080-88) {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.security-token=Z0JYTkiCyerH5+ROt2s6qUduy2cWGcWe1GSdynl+sz5tRqN/ZYIdVoXy2RR7fWOgyt8=, rhq.timeout=5000, rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[ping], targetInterfaceName=org.rhq.enterprise.communications.Ping}]]. Cause: java.util.concurrent.TimeoutException:null. Cause: java.util.concurrent.TimeoutException >10:14:20,165 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-5) Have not heard from agent [dev108-agent-16223] since [Thu Apr 03 07:40:58 EDT 2014]. Will be backfilled since we suspect it is down >10:14:20,205 INFO [org.rhq.enterprise.server.core.AgentManagerBean] (RHQScheduler_Worker-5) Have not heard from agent [dev20-03-agent-16296] since [Thu Apr 03 10:09:14 EDT 2014]. Will be backfilled since we suspect it is down
<b>You cannot view the attachment while viewing its details because your browser does not support IFRAMEs. <a href="attachment.cgi?id=882685">View the attachment on a separate page</a>.</b>
View Attachment As Raw
Actions:
View
Attachments on
bug 1084481
: 882685 |
882696
|
882697