Description of problem: RHQ is pretty lousy (slow) with uninventorying resources. It holds open a database connection while busily communicating with agents. (Not allowed by EJB.) When ignoring a resource type from a single agent, with about 200 resources, a database transaction basically times out with the following: Exception in thread "main" org.rhq.enterprise.client.script.CLIScriptException: org.rhq.scripting.javascript.engine.util.ExtendedScriptException: org.mozilla.javascript.WrappedException: Wrapped javax.ejb.EJBTransactionRolledbackException: [Warning] Transaction rolled back (<Unknown source>#66) in <Unknown source> at line number 66 at org.rhq.enterprise.client.commands.ScriptCommand.executeScriptFile(ScriptCommand.java:282) at org.rhq.enterprise.client.commands.ScriptCommand.execute(ScriptCommand.java:92) at org.rhq.enterprise.client.ClientMain$StartupConfiguration.process(ClientMain.java:157) at org.rhq.enterprise.client.ClientMain.main(ClientMain.java:177) Caused by: org.rhq.scripting.javascript.engine.util.ExtendedScriptException: org.mozilla.javascript.WrappedException: Wrapped javax.ejb.EJBTransactionRolledbackException: [Warning] Transaction rolled back (<Unknown source>#66) in <Unknown source> at line number 66 at org.rhq.scripting.javascript.engine.RhinoScriptEngine.eval(RhinoScriptEngine.java:375) at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:232) at org.rhq.enterprise.client.commands.ScriptCommand.executeScriptFile(ScriptCommand.java:271) ... 3 more Caused by: org.mozilla.javascript.WrappedException: Wrapped javax.ejb.EJBTransactionRolledbackException: [Warning] Transaction rolled back (<Unknown source>#66) at org.mozilla.javascript.Context.throwAsScriptRuntimeEx(Context.java:1754) at org.mozilla.javascript.MemberBox.invoke(MemberBox.java:148) at org.mozilla.javascript.NativeJavaMethod.call(NativeJavaMethod.java:225) at org.mozilla.javascript.Interpreter.interpretLoop(Interpreter.java:1473) at org.mozilla.javascript.Interpreter.interpret(Interpreter.java:815) at org.mozilla.javascript.InterpretedFunction.call(InterpretedFunction.java:109) at org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:394) at org.rhq.scripting.javascript.engine.RhinoScriptEngine$1.superDoTopCall(RhinoScriptEngine.java:235) at org.rhq.scripting.javascript.engine.RhinoScriptEngine$1.access$000(RhinoScriptEngine.java:199) at org.rhq.scripting.javascript.engine.RhinoScriptEngine$1$1.run(RhinoScriptEngine.java:215) at java.security.AccessController.doPrivileged(Native Method) at org.rhq.scripting.javascript.engine.RhinoScriptEngine$1.doTopCall(RhinoScriptEngine.java:212) at org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3091) at org.mozilla.javascript.InterpretedFunction.exec(InterpretedFunction.java:120) at org.mozilla.javascript.Context.evaluateReader(Context.java:1110) at org.rhq.scripting.javascript.engine.RhinoScriptEngine.eval(RhinoScriptEngine.java:363) ... 5 more Caused by: javax.ejb.EJBTransactionRolledbackException: [Warning] Transaction rolled back at org.rhq.enterprise.server.remote.RemoteSafeInvocationHandler.invoke(RemoteSafeInvocationHandler.java:116) The agent is connected, by the way. Version-Release number of selected component (if applicable): 4.9 How reproducible: Always Steps to Reproduce: 1. Create a hundred or so resources with a single type. 2. Attempt to ignore them using the CLI (or UI, although the UI doesn't say it did anything) Actual results: Transaction aborted Expected results: Resource type ignored, resources removed Additional info: In this case, the resources were in Unknown state. It seems like RHQ couldn't merge all the resources at the time.
It seems in my case there is a database lock preventing uninventory of resources. The query is simply selecting resources of a particular type. "http-/0.0.0.0:7080-241" daemon prio=10 tid=0x00007f399c18a800 nid=0x116c runnable [0x00007f39d4a56000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at oracle.net.ns.Packet.receive(Packet.java:283) at oracle.net.ns.DataPacket.receive(DataPacket.java:103) at oracle.net.ns.NetInputStream.getNextPacket(NetInputStream.java:230) at oracle.net.ns.NetInputStream.read(NetInputStream.java:175) at oracle.net.ns.NetInputStream.read(NetInputStream.java:100) at oracle.net.ns.NetInputStream.read(NetInputStream.java:85) at oracle.jdbc.driver.T4CSocketInputStreamWrapper.readNextPacket(T4CSocketInputStreamWrapper.java:123) at oracle.jdbc.driver.T4CSocketInputStreamWrapper.read(T4CSocketInputStreamWrapper.java:79) at oracle.jdbc.driver.T4CMAREngine.unmarshalUB1(T4CMAREngine.java:1122) at oracle.jdbc.driver.T4CMAREngine.unmarshalSB1(T4CMAREngine.java:1099) at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:288) at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:191) at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:523) at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:207) at oracle.jdbc.driver.T4CPreparedStatement.executeForRows(T4CPreparedStatement.java:1010) at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1185) at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1275) at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3576) at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3620) - locked <0x0000000455de4468> (a oracle.jdbc.driver.T4CConnection) ... at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) at org.rhq.enterprise.server.resource.ResourceManagerLocal$$$view22.uninventoryResourcesOfResourceType(Unknown Source) at org.rhq.enterprise.server.resource.ResourceTypeManagerBean.setResourceTypeIgnoreFlagAndUninventoryResources(ResourceTypeManagerBean.java:96) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
The easiest way to reproduce is to have a hundred agents, disable the CPU resource type, and see the transaction timeout.