Bug 877176

Summary: CLI DiscoveryBoss.importResources fails when upgraded inventory contains empty/null resource availability
Product: [JBoss] JBoss Operations Network Reporter: Larry O'Leary <loleary>
Component: InventoryAssignee: Jay Shaughnessy <jshaughn>
Status: CLOSED CURRENTRELEASE QA Contact: Mike Foley <mfoley>
Severity: urgent Docs Contact:
Priority: urgent    
Version: JON 3.1.1CC: fbrychta
Target Milestone: ---   
Target Release: JON 3.1.2   
Hardware: All   
OS: All   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-09-11 10:58:46 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 865166    
Bug Blocks:    

Description Larry O'Leary 2012-11-15 21:14:52 UTC
Description of problem:
This is essentially the same issue identified in Bug 803516 which was supposed to have been fixed in RHQ 4.4. However, there appears to be other scenarios in where the invalid inventory state can occur. In this case, importing of a resource which has no row in the rhq_resource_avail table will result in the following error:

Exception in thread "main" org.rhq.enterprise.client.script.CLIScriptException: javax.script.ScriptException: sun.org.mozilla.javascript.WrappedException: Wrapped javax.ejb.EJBException: [Warning] java.lang.RuntimeException: javax.transaction.RollbackException: [com.arjuna.ats.internal.jta.transaction.arjunacore.commitwhenaborted] [com.arjuna.ats.internal.jta.transaction.arjunacore.commitwhenaborted] Can't commit because the transaction is in aborted state (<Unknown source>#38) in <Unknown source> at line number 38
	at org.rhq.enterprise.client.commands.ScriptCommand.executeScriptFile(ScriptCommand.java:298)
	at org.rhq.enterprise.client.commands.ScriptCommand.execute(ScriptCommand.java:108)
	at org.rhq.enterprise.client.ClientMain$StartupConfiguration.process(ClientMain.java:141)
	at org.rhq.enterprise.client.ClientMain.main(ClientMain.java:161)
...
Caused by: javax.ejb.EJBException: [Warning] java.lang.RuntimeException: javax.transaction.RollbackException: [com.arjuna.ats.internal.jta.transaction.arjunacore.commitwhenaborted] [com.arjuna.ats.internal.jta.transaction.arjunacore.commitwhenaborted] Can't commit because the transaction is in aborted state
	at org.rhq.enterprise.server.remote.RemoteSafeInvocationHandler.invoke(RemoteSafeInvocationHandler.java:113)
	at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:809)
	at org.jboss.remoting.transport.servlet.ServletServerInvoker.processRequest(ServletServerInvoker.java:232)
	at sun.reflect.GeneratedMethodAccessor296.invoke(Unknown Source)
	...
Caused by: java.lang.RuntimeException: [Warning] javax.transaction.RollbackException: [com.arjuna.ats.internal.jta.transaction.arjunacore.commitwhenaborted] [com.arjuna.ats.internal.jta.transaction.arjunacore.commitwhenaborted] Can't commit because the transaction is in aborted state
	at org.rhq.core.util.exception.WrappedRemotingException.getCause(WrappedRemotingException.java:121)
	at java.lang.Throwable.printStackTraceAsCause(Throwable.java:512)
	at java.lang.Throwable.printStackTraceAsCause(Throwable.java:514)
	at java.lang.Throwable.printStackTraceAsCause(Throwable.java:514)
	at java.lang.Throwable.printStackTrace(Throwable.java:485)
	at java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:1001)
	at java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:992)
	at java.lang.Thread.dispatchUncaughtException(Thread.java:1845)
Caused by: javax.transaction.RollbackException: [Warning] [com.arjuna.ats.internal.jta.transaction.arjunacore.commitwhenaborted] [com.arjuna.ats.internal.jta.transaction.arjunacore.commitwhenaborted] Can't commit because the transaction is in aborted state
	at org.rhq.core.util.exception.WrappedRemotingException.getCause(WrappedRemotingException.java:121)
	at java.lang.Throwable.printStackTraceAsCause(Throwable.java:512)
	at java.lang.Throwable.printStackTraceAsCause(Throwable.java:514)
	... 6 more
Caused by: javax.persistence.PersistenceException: [Warning] org.hibernate.exception.GenericJDBCException: Could not execute JDBC batch update
	at org.rhq.core.util.exception.WrappedRemotingException.getCause(WrappedRemotingException.java:121)
	at java.lang.Throwable.printStackTraceAsCause(Throwable.java:512)
	at java.lang.Throwable.printStackTraceAsCause(Throwable.java:514)
	... 7 more
Caused by: org.hibernate.exception.GenericJDBCException: [Warning] Could not execute JDBC batch update
	at org.rhq.core.util.exception.WrappedRemotingException.getCause(WrappedRemotingException.java:121)
	at java.lang.Throwable.printStackTraceAsCause(Throwable.java:512)
	at java.lang.Throwable.printStackTraceAsCause(Throwable.java:514)
	... 8 more
Caused by: java.sql.BatchUpdateException: [Warning] Batch entry 0 update RHQ_RESOURCE set UUID='4ff64a4e-4566-46d6-88be-c5fdedf001eb', RESOURCE_KEY='localhost.localdomain RHQ Agent', NAME='RHQ Agent', ANCESTRY='10048_:_10001_:_localhost.localdomain', INVENTORY_STATUS='COMMITTED', CONNECTED='0', VERSION='4.2.0.JON.3.0.1.GA', DESCRIPTION='RHQ Management Agent', CTIME='1353005533201', MTIME='1353013357766', ITIME='1353013357766', MODIFIED_BY='admin', LOCATION=NULL, RESOURCE_TYPE_ID='10086', PARENT_RESOURCE_ID='10001', RES_CONFIGURATION_ID='10881', PLUGIN_CONFIGURATION_ID='10882', AGENT_ID='10001', PRODUCT_VERSION_ID='10002' where ID='10051' was aborted.  Call getNextException to see the cause.
	at org.rhq.core.util.exception.WrappedRemotingException.getCause(WrappedRemotingException.java:121)
	at java.lang.Throwable.printStackTraceAsCause(Throwable.java:512)
	at java.lang.Throwable.printStackTraceAsCause(Throwable.java:514)
	... 9 more


Version-Release number of selected component (if applicable):
4.4.0.JON311GA

How reproducible:
Always

Steps to Reproduce:
1.  Start ON 3.0.1 system
2.  Allow agent to discover RHQ Agent resource but DO NOT import it
3.  Shutdown ON 3.0.1 system
4.  Execute the following database SQL query to simulate inventory state that can occur from prior releases:

    DELETE FROM rhq_resource_avail WHERE id=(SELECT ra.id AS Avail_ID FROM rhq_resource r LEFT JOIN rhq_resource_avail ra ON r.id = ra.resource_id WHERE r.name='RHQ Agent' AND ra.id IS NOT NULL);
    
    This should delete 1 row.
    
5.  Perform upgrade to ON 3.1.1
6.  Start ON 3.1.1 server but DO NOT start agent
7.  Execute customer's script
  
Actual results:
Import of resource fails with big long exception which ends with:

java.sql.BatchUpdateException: [Warning] Batch entry 0 update RHQ_RESOURCE set UUID='4ff64a4e-4566-46d6-88be-c5fdedf001eb', RESOURCE_KEY='localhost.localdomain RHQ Agent', NAME='RHQ Agent', ANCESTRY='10048_:_10001_:_localhost.localdomain', INVENTORY_STATUS='COMMITTED', CONNECTED='0', VERSION='4.2.0.JON.3.0.1.GA', DESCRIPTION='RHQ Management Agent', CTIME='1353005533201', MTIME='1353013357766', ITIME='1353013357766', MODIFIED_BY='admin', LOCATION=NULL, RESOURCE_TYPE_ID='10086', PARENT_RESOURCE_ID='10001', RES_CONFIGURATION_ID='10881', PLUGIN_CONFIGURATION_ID='10882', AGENT_ID='10001', PRODUCT_VERSION_ID='10002' where ID='10051' was aborted.  Call getNextException to see the cause.



Expected results:
No error should be reported and resource should be imported into inventory

Additional info:
The fix for this issue has already been done in Bug 865166. This BZ is primarily a means of providing additional test steps to validate it is really fixed this time.

To test, perform the steps mentioned above but instead of upgrading to ON 3.1.1, upgrade to the version that contains the fix for Bug 865166. 

The test script attachment 573024 [details] from Bug 865166 can also be used to test this issue.

Comment 1 Larry O'Leary 2012-11-15 21:18:01 UTC
Assigning to Jay and setting to MODIFIED as he is assigned to Bug 856166 which should fix this bug too.

Comment 2 Simeon Pinder 2012-11-21 21:55:57 UTC
Moving to ON_QA as available for test with build : https://brewweb.devel.redhat.com//buildinfo?buildID=244662.

Comment 3 Filip Brychta 2012-11-29 10:57:52 UTC
Verified that upgrade to 3.1.2.ER2 fixes inconsistent state of rhq_resource_avail table (missing rows, NULL availability_type) and import of resources works for both CLI and GUI.