Created attachment 893790 [details] Log excerpt showing complete error and stack traces Description of problem: Dead lock errors are reported resulting in database failures being reported. From the stack trace (in the log excerpt) it appears these deadlocks occurred when an inventory merge was being done. ERROR [org.hibernate.event.def.AbstractFlushingEventListener] Could not synchronize database state with session org.hibernate.exception.LockAcquisitionException: Could not execute JDBC batch update at org.hibernate.exception.SQLStateConverter.convert(SQLStateConverter.java:87) at org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:43) at org.hibernate.jdbc.AbstractBatcher.executeBatch(AbstractBatcher.java:254) at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:237) at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:142) ... at sun.proxy.$Proxy332.updateImplicitGroupMembership(Unknown Source) at org.rhq.enterprise.server.discovery.DiscoveryBossBean.addResourceToInventory(DiscoveryBossBean.java:944) at org.rhq.enterprise.server.discovery.DiscoveryBossBean.mergeResource(DiscoveryBossBean.java:711) at org.rhq.enterprise.server.discovery.DiscoveryBossBean.updatePreviouslyInventoriedResource(DiscoveryBossBean.java:899) at org.rhq.enterprise.server.discovery.DiscoveryBossBean.mergeResource(DiscoveryBossBean.java:708) at org.rhq.enterprise.server.discovery.DiscoveryBossBean.mergeInventoryReport(DiscoveryBossBean.java:178) ... at sun.proxy.$Proxy505.mergeInventoryReport(Unknown Source) at org.rhq.enterprise.server.discovery.DiscoveryServerServiceImpl.mergeInventoryReport(DiscoveryServerServiceImpl.java:75) ... at org.rhq.enterprise.communications.command.impl.remotepojo.server.RemotePojoInvocationCommandService.execute(RemotePojoInvocationCommandService.java:184) ... at org.rhq.enterprise.communications.command.server.CommandProcessor.handleIncomingInvocationRequest(CommandProcessor.java:290) at org.rhq.enterprise.communications.command.server.CommandProcessor.invoke(CommandProcessor.java:184) ... Caused by: java.sql.BatchUpdateException: ORA-00060: deadlock detected while waiting for resource at oracle.jdbc.driver.OraclePreparedStatement.executeBatch(OraclePreparedStatement.java:10345) at oracle.jdbc.driver.OracleStatementWrapper.executeBatch(OracleStatementWrapper.java:230) at org.jboss.resource.adapter.jdbc.CachedPreparedStatement.executeBatch(CachedPreparedStatement.java:476) at org.jboss.resource.adapter.jdbc.WrappedStatement.executeBatch(WrappedStatement.java:774) at org.hibernate.jdbc.BatchingBatcher.doExecuteBatch(BatchingBatcher.java:48) at org.hibernate.jdbc.AbstractBatcher.executeBatch(AbstractBatcher.java:247) ... 189 more Version-Release number of selected component (if applicable): 3.1.2.GA Additional info: This seems very similar to the deadlock issue raised in bug 1093265 but this appears to be driven by inventory sync/merge rather then user invoking a function of the UI.
Given the large changes in sync in JON 3.2.0 I'd say the recommendation here should be an upgrade.
I can tell you that the call chain in the log no longer exists. We removed the umbrella Tx in mergeInventoryReport in commit 22053b3794a724a1fa997df07684ed14dd55b564 and further updated relevant Tx delimiters in commit b6e70d104756db633df7b4ff642c29e3fad81d2b. I don't think this problem exists in the current release. I recommend closing. Asking Larry to see if he concurs or otherwise can recommend some other course of action.
I concur. As we have had no reports of this issue past JBoss ON 3.1.2 and the inventory sync has been reworked, this issue seems like it should no longer occur. If we find that this issue happens on 3.2 or -- when its released -- 3.3, then a new bug report should be opened providing steps to reproduce along with relevant logs. Marking as CLOSED WORKSFORME.