Bug 582991 - db-upgrade 2.78 broken
Summary: db-upgrade 2.78 broken
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: RHQ Project
Classification: Other
Component: Database
Version: 3.0.0
Hardware: All
OS: Linux
low
medium
Target Milestone: ---
: ---
Assignee: John Sanda
QA Contact: Rajan Timaniya
URL:
Whiteboard:
: 582992 582994 (view as bug list)
Depends On:
Blocks: rhq_triage
TreeView+ depends on / blocked
 
Reported: 2010-04-16 11:15 UTC by Lukas Krejci
Modified: 2010-08-12 16:51 UTC (History)
2 users (show)

Fixed In Version: 2.4
Clone Of:
Environment:
Last Closed: 2010-08-12 16:51:47 UTC
Embargoed:


Attachments (Terms of Use)
logfile (388.43 KB, text/x-log)
2010-05-05 14:07 UTC, Rajan Timaniya
no flags Details
NO Menu on Dashboard (94.46 KB, image/png)
2010-05-05 14:11 UTC, Rajan Timaniya
no flags Details
Error on UI (32.45 KB, image/png)
2010-05-05 14:12 UTC, Rajan Timaniya
no flags Details
UI error stack trace (156.59 KB, image/png)
2010-05-05 14:13 UTC, Rajan Timaniya
no flags Details
PSQLException on UI (28.57 KB, image/png)
2010-05-06 12:34 UTC, Rajan Timaniya
no flags Details
rhq-server-log4j log - jon build# 127 (136.57 KB, text/x-log)
2010-05-06 12:34 UTC, Rajan Timaniya
no flags Details
UI error stack trace (187.74 KB, image/png)
2010-05-06 12:35 UTC, Rajan Timaniya
no flags Details
rhq-server-log4j log - jon build# 128 (153.99 KB, text/x-log)
2010-05-07 11:56 UTC, Rajan Timaniya
no flags Details
UI Error (30.23 KB, image/png)
2010-05-07 11:56 UTC, Rajan Timaniya
no flags Details
UI error stack trace (197.35 KB, image/png)
2010-05-07 11:57 UTC, Rajan Timaniya
no flags Details

Description Lukas Krejci 2010-04-16 11:15:31 UTC
Description of problem:

When trying to upgrade the database from RHQ 1.3.1 to the current master, the script fails when trying to apply the changes for schema version 2.78 complaining about duplicit primary keys.

How reproducible:

always

Steps to Reproduce:
1. Update settings.xml to point to an rhq 1.3.1  database (or modify the below mvn execution to define the overrides of the db connection properties)
2. run mvn -Ddbsetup-upgrade from modules/core/dbutils
3. Wait a bit for the error message
  
Actual results:

Upgrade fails

Expected results:

Upgrade shouldn't fail.

Additional info:

I think this can be easily fixed by changing the create statement to:

CREATE TABLE RHQ_BUNDLE_DEPLOY_HIST ( ID INTEGER )

(because the primary key on the ID column is created a couple of lines after that).

Comment 1 Charles Crouch 2010-04-27 19:11:19 UTC
Need to retest dbupgrade from JON2.3.1, this should now be working

Comment 2 Charles Crouch 2010-04-27 19:12:30 UTC
*** Bug 582992 has been marked as a duplicate of this bug. ***

Comment 3 Charles Crouch 2010-04-27 19:13:23 UTC
*** Bug 582994 has been marked as a duplicate of this bug. ***

Comment 4 Corey Welton 2010-05-04 19:03:12 UTC
qa -> rtimaniy

Comment 5 Rajan Timaniya 2010-05-05 14:01:06 UTC
Description of problem:
Fail upgrade JON 2.3.1 to JON build #126 on Postgres (8.4.2).

Version-Release number of selected component (if applicable):
JON build #126  Revision: 10615 
http://hudson-qe.rhq.rdu.redhat.com:8080/job/jon/126/


Steps to Reproduce:
1) Install JON 2.3.1 on Postgres (8.4.2)
2) Create some alerts, schedule, etc..  (basic operations)
   (i) Alert:
       Alert -1: Created on platform, the alert condition is Free Memory > 0.1
       Alert -2: Created on JBoss AS server, the alert condition is Free Memory
> 0.1
       Alert -3: Created on RHQ Agent, the alert condition is Free Memory > 0.1
   (ii) schedule:
        Created a View Process List schedule for platform
   (iii)Modified JBoss AS Server connection parameters like 'Principal' &
'Credentials'
3) Upgrade JON 2.3.1 to JON build #126 (same database Postgres)
4) Login to JON build #126 and select platform or (any) resource


Actual results:
After upgrade of JON2.3.1 to JON build #126, there is no menu and it gives error on click
of platform or resource.


Expected results:
Upgrade JON 2.3.1 to JON build #126 should work without any error.

Additional info:
2010-05-05 18:43:31,345 INFO  [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] rajanlaptop.usersys.redhat.com took [31]ms to reload global cache
2010-05-05 18:44:01,309 ERROR [org.rhq.enterprise.server.scheduler.jobs.AbstractStatefulJob] Failed to execute job [SavedSearchResultCountRecalculationJob]
javax.ejb.EJBException: java.lang.IllegalArgumentException: Unknown name value for enum class org.rhq.core.domain.search.SearchSubsystem: Resource
	at org.jboss.ejb3.tx.Ejb3TxPolicy.handleExceptionInOurTx(Ejb3TxPolicy.java:63)
	at org.jboss.aspects.tx.TxPolicy.invokeInOurTx(TxPolicy.java:83)
	at org.jboss.aspects.tx.TxInterceptor$Required.invoke(TxInterceptor.java:191)

Comment 6 Rajan Timaniya 2010-05-05 14:07:33 UTC
Created attachment 411616 [details]
logfile

Comment 7 Rajan Timaniya 2010-05-05 14:11:31 UTC
Created attachment 411622 [details]
NO Menu on Dashboard

Comment 8 Rajan Timaniya 2010-05-05 14:12:02 UTC
Created attachment 411623 [details]
Error on UI

Comment 9 Rajan Timaniya 2010-05-05 14:13:01 UTC
Created attachment 411625 [details]
UI error stack trace

Comment 10 John Sanda 2010-05-05 20:34:47 UTC
The problem ultimately stems from the upgrade script inserting the string 'Resource' into the context column of rhq_saved_searches when it should instead be 'RESOURCES'. Hibernate is looking for the string in all caps because that is the value it finds in the enum SearchSubsystem from which the it is ultimately derived. The setup script already uses the correct, all-caps version.

Commit hash - 88b808d3cf1e5fbb9e60208c1aea818d03ebfd48

Comment 11 John Sanda 2010-05-05 20:42:47 UTC
My previous comment was not entirely clear. I modified the upgrade so that the rows that get inserted into rhq_saved_searches use the string 'RESOURCE'.

Comment 12 Rajan Timaniya 2010-05-06 11:03:38 UTC
Steps:
1) Install JON 2.3.1 on Postgres (8.4.2)
2) Create some alerts, schedule, etc..  (basic operations)
   (i) Alert:
       Alert -1: Created on platform, the alert condition is Free Memory > 0.1
       Alert -2: Created on JBoss AS server, the alert condition is Free Memory
> 0.1
       Alert -3: Created on RHQ Agent, the alert condition is Free Memory > 0.1
   (ii) schedule:
        Created a View Process List schedule for platform
   (iii)Modified JBoss AS Server connection parameters like 'Principal' &
'Credentials'
3) Upgrade JON 2.3.1 to JON build #127 (same database Postgres)
4) Login to JON build #127 and select platform or (any) resource

Version-Release number of selected component (if applicable):
JON build #127 - http://hudson-qe.rhq.rdu.redhat.com:8080/job/jon/127/

Actual results:
After upgrade of JON2.3.1 to JON build #127, it gives error on click of platform or resource.

Expected results:
Upgrade JON 2.3.1 to JON build #127 should work without any error.

Additional info:
2010-05-06 15:24:46,329 WARN  [org.hibernate.util.JDBCExceptionReporter] SQL Error: 0, SQLState: 42703
2010-05-06 15:24:46,329 ERROR [org.hibernate.util.JDBCExceptionReporter] ERROR: column alertdefin1_.operation_def_id does not exist
  Position: 13585
2010-05-06 15:24:46,332 INFO  [org.hibernate.event.def.DefaultLoadEventListener] Error performing load command
org.hibernate.exception.SQLGrammarException: could not load an entity: [org.rhq.core.domain.alert.AlertDefinition#10001]
        at org.hibernate.exception.SQLStateConverter.convert(SQLStateConverter.java:67)
        at org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:43)
        at org.hibernate.loader.Loader.loadEntity(Loader.java:1874)

Comment 13 Rajan Timaniya 2010-05-06 12:34:02 UTC
Created attachment 411992 [details]
PSQLException on UI

Comment 14 Rajan Timaniya 2010-05-06 12:34:45 UTC
Created attachment 411993 [details]
rhq-server-log4j log - jon build# 127

Comment 15 Rajan Timaniya 2010-05-06 12:35:27 UTC
Created attachment 411994 [details]
UI error stack trace

Comment 16 John Sanda 2010-05-06 15:59:07 UTC
The operation_def_id column was removed from the rhq_alert_definition table, but the JPA mappings and references to it still exist in the java code, resulting in the errors. Based on discussion with heiko, my understanding is that the field should in fact be deleted along with the corresponding java code. For now though, I have undone the column removal in the upgrade script, and then we can go back later and remove the column and corresponding java code in one fell swoop.

commit hash -  19c8f08897a9b8c75b883fb2206ee97c431989da

Comment 17 Rajan Timaniya 2010-05-07 11:54:45 UTC
Description of problem:
Fail upgrade JON 2.3.1 to JON build #128 on Postgres (8.4.2).

Steps:
1) Install JON 2.3.1 on Postgres (8.4.2)
2) Create some alerts, schedule, etc..  (basic operations)
   (i) Alert:
       Alert -1: Created on platform, the alert condition is Free Memory > 0.1
       Alert -2: Created on JBoss AS server, the alert condition is Free Memory
> 0.1
       Alert -3: Created on RHQ Agent, the alert condition is Free Memory > 0.1
   (ii) schedule:
        Created a View Process List schedule for platform
   (iii)Modified JBoss AS Server connection parameters like 'Principal' &
'Credentials'
   (iv) Group: Created 1-Mixed group, 1-Compatible group, 1-Group Defition 
3) Upgrade JON 2.3.1 to JON build #128 (same database Postgres)
4) Login to JON build #128 and select platform or (any) resource

Version-Release number of selected component (if applicable):
JON build #128 - http://hudson-qe.rhq.rdu.redhat.com:8080/job/jon/128/

Actual results:
After upgrade of JON2.3.1 to JON build #128, it gives error on click of
platform or resource.

Expected results:
Upgrade JON 2.3.1 to JON latest build should work without any error.

Additional info:
2010-05-07 16:27:53,826 INFO  [org.hibernate.event.def.DefaultLoadEventListener] Error performing load command
org.hibernate.HibernateException: Javassist Enhancement failed: org.rhq.core.domain.plugin.Plugin
	at org.hibernate.proxy.pojo.javassist.JavassistLazyInitializer.getProxy(JavassistLazyInitializer.java:118)

2010-05-07 16:27:53,842 ERROR [org.rhq.enterprise.server.core.plugin.ProductPluginDeployer] Failed to register RHQ plugin file [file:/NotBackedUp/install/jon128/jon-server-2.4.0-SNAPSHOT/jbossas/server/default/deploy/rhq.ear/rhq-downloads/rhq-plugins/rhq-jmx-plugin-3.0.0-SNAPSHOT.jar]
javax.ejb.EJBException: javax.persistence.PersistenceException: org.hibernate.HibernateException: Javassist Enhancement failed: org.rhq.core.domain.plugin.Plugin

2010-05-07 16:28:03,266 ERROR [org.rhq.enterprise.server.core.plugin.ProductPluginDeployer] Failed to register RHQ plugin file [file:/NotBackedUp/install/jon128/jon-server-2.4.0-SNAPSHOT/jbossas/server/default/deploy/rhq.ear/rhq-downloads/rhq-plugins/jopr-tomcat-plugin-3.0.0-SNAPSHOT.jar]
javax.ejb.EJBException: org.hibernate.LazyInitializationException: failed to lazily initialize a collection, no session or session was closed

2010-05-07 16:28:53,101 ERROR [org.rhq.enterprise.gui.legacy.portlet.criticalalerts.ViewAction] Dashboard Portlet [CriticalAlerts] experienced an error: javax.persistence.PersistenceException: org.hibernate.PropertyAccessException: Null value was assigned to a property of primitive type setter of org.rhq.core.domain.alert.Alert.acknowledgeTime
2010-05-07 16:29:13,705 INFO  [net.hyperic.hq.common.shared.license.LicenseManager] [License owner=JBoss ON DEVELOPER ONLY!!! email=sales phone=404-467-8555 expiration=never platforms=unlimited serverIPs=any plugins=all support=3]
2010-05-07 16:29:17,785 ERROR [org.rhq.enterprise.gui.legacy.portlet.criticalalerts.ViewAction] Dashboard Portlet [CriticalAlerts] experienced an error: javax.persistence.PersistenceException: org.hibernate.PropertyAccessException: Null value was assigned to a property of primitive type setter of org.rhq.core.domain.alert.Alert.acknowledgeTime
2010-05-07 16:29:35,782 ERROR [org.rhq.enterprise.gui.common.error.GenericErrorUIBean] Error processing user request
javax.faces.el.EvaluationException: javax.el.ELException: /rhq/resource/summary/table/recentAlerts.xhtml @16,62 value="#{ResourceOverviewUIBean.recentAlerts}": Error reading 'recentAlerts' on type org.rhq.enterprise.gui.inventory.resource.ResourceOverviewUIBean

Comment 18 Rajan Timaniya 2010-05-07 11:56:12 UTC
Created attachment 412315 [details]
rhq-server-log4j log - jon build# 128

Comment 19 Rajan Timaniya 2010-05-07 11:56:56 UTC
Created attachment 412316 [details]
UI Error

Comment 20 Rajan Timaniya 2010-05-07 11:57:23 UTC
Created attachment 412317 [details]
UI error stack trace

Comment 21 John Sanda 2010-05-07 17:14:05 UTC
I was able to reproduce the plugin deployment errors that came in the form of the HibernateException; however, after bouncing my server I did not see the plugin-related errors, and http://localhost:7080/rhq/admin/plugin/plugin-list.xhtml shows that my plugins are registered and enabled. While I am not clear on the exact reason for the plugin deployment exceptions, it looks like we might be able to ignore it.

Still investigating the other issues.

Comment 22 John Sanda 2010-05-10 04:15:23 UTC
Just committed and pushed a fix to handle the PropertyAccessException. The reason this was happening is because in the rhq_alert table the ack_time column had a null value. When fetching alerts from the database, hibernate was trying to write the null value into a primitive field which of course does not allow nulls. I changed the type of Alert.ackTime from long to Long to support nulls. This gets past the exception, but I am not yet sure if it fully addresses the functional requirements.

commit hash - 1cdcd06385fae9025f2549043068b5b34e9e06a2

Comment 23 Rajan Timaniya 2010-05-10 14:31:17 UTC
Verified on JON build #131 - Revision: 10615
http://hudson-qe.rhq.rdu.redhat.com:8080/job/jon/131/

Upgrade JON 2.3.1 to JON build #131 working fine for following steps:
1) Install JON 2.3.1 on Postgres (8.4.2) & Oracle 10g
2) Create some alerts, schedule, etc..  (basic operations)
   (i) Alert:
       Alert -1: Created on platform, the alert condition is Free Memory > 0.1
       Alert -2: Created on JBoss AS server, the alert condition is Free Memory
> 0.1
       Alert -3: Created on RHQ Agent, the alert condition is Free Memory > 0.1
   (ii) schedule:
        Created a View Process List schedule for platform
   (iii)Modified JBoss AS Server connection parameters like 'Principal' &
'Credentials'
   (iv) Group: Created 1-Mixed group, 1-Compatible group, 1-Group Defition 
3) Upgrade JON 2.3.1 to JON build #131 (same database Postgres/Oracle)
4) Login to JON build #131 and select platform or (any) resource

Comment 24 Corey Welton 2010-08-12 16:51:47 UTC
Mass-closure of verified bugs against JON.


Note You need to log in before you can comment on or make changes to this bug.