Bug 720716 - Review all plugins to make sure no resource keys have changed unnecessarily
Summary: Review all plugins to make sure no resource keys have changed unnecessarily
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: RHQ Project
Classification: Other
Component: Plugins
Version: 4.1
Hardware: Unspecified
OS: Unspecified
urgent
unspecified
Target Milestone: ---
: ---
Assignee: Lukas Krejci
QA Contact: Mike Foley
URL:
Whiteboard:
Depends On: 720674
Blocks: rhq42
TreeView+ depends on / blocked
 
Reported: 2011-07-12 15:28 UTC by Charles Crouch
Modified: 2015-02-01 23:26 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2012-02-07 19:29:54 UTC
Embargoed:


Attachments (Terms of Use)
discovery component diffs (41.20 KB, application/x-gzip)
2011-10-10 15:08 UTC, Lukas Krejci
no flags Details
RHQ_RESOURCES from 4.1 (120.95 KB, text/plain)
2011-10-11 14:50 UTC, Mike Foley
no flags Details
RHQ_RESOURCES from 2.4.1 (43.09 KB, text/plain)
2011-10-11 15:23 UTC, Mike Foley
no flags Details
error of upgrade failure (69.12 KB, image/png)
2011-10-11 15:35 UTC, Mike Foley
no flags Details
text of upgrade error (1.86 KB, application/octet-stream)
2011-10-11 15:36 UTC, Mike Foley
no flags Details
dbupgrade log (123.85 KB, application/octet-stream)
2011-10-11 15:37 UTC, Mike Foley
no flags Details
server log (36.45 KB, application/octet-stream)
2011-10-11 15:37 UTC, Mike Foley
no flags Details
diffrences between resource keys in JON2->3 and JON3 (8.91 KB, patch)
2011-11-10 17:37 UTC, Libor Zoubek
no flags Details | Diff

Description Charles Crouch 2011-07-12 15:28:19 UTC
Related to https://bugzilla.redhat.com/show_bug.cgi?id=720674

We need to make sure no other plugins had accidental resource key changes.

Comment 1 Heiko W. Rupp 2011-09-28 15:40:04 UTC
Start a fresh JON 2.4.1 and take stuff into inventory.
Record the resource keys from the database.

Do the same with JON3 and check for differences

Comment 2 Charles Crouch 2011-09-30 21:57:18 UTC
fix blocks/depends bz's

Comment 3 Lukas Krejci 2011-10-10 15:08:36 UTC
Created attachment 527254 [details]
discovery component diffs

Comment 4 Lukas Krejci 2011-10-10 15:09:25 UTC
the attached diff between RHQ 3.0.1 and current master was created by running:

$ git diff --no-color release-3.0.1..master `find -name '*DiscoveryComponent.java'` | gzip > discovery-components.diff.gz

by inspecting the diff I found the following:

apache plugin
- ApacheServerDiscoveryComponent - with ResourceUpgradeFacet
- ApacheVirtualHostServiceDiscoveryComponent - with ResourceUpgradeFacet
jboss-as-5 plugin
- AbstractManagedDeploymentDiscoveryComponent - with ResourceUpgradeFacet
- ScriptDiscoveryComponent - with ResourceUpgradeFacet
jboss-as plugin
- ScriptDiscoveryComponent - with ResourceUpgradeFacet
mysql plugin
- MySqlDiscoveryComponent - WITHOUT ResourceUpgradeFacet (but the plugin was unmaintained before Steven actually contributed the code that changed the RK as a consequence of much larger feature set available in the plugin).

This means that the only thing we should care about is the RK change in the MySqlDiscoveryComponent.

I'd argue that before that change, Mysql plugin was not functional at all, so we don't have to worry about it.


These would be the tests to smoke-test this:

1) Install JON 2.4.1 with all plugin-packs, inventory resources of as many different types as possible (ideally for each resource type in each plugin at least 1 resource).
2) Upgrade that installation to the latest RHQ code
3) Make sure no resource upgrade errors exist
4) Install latest RHQ using a fresh new database
5) Inventory the same resources as with JON 2.4.1
6) Compare the RHQ_RESOURCE tables from the 2 databases
7) The corresponding resources should have the same resource keys (the IDs most possibly won't match but that of course is not a problem).

Comment 5 Mike Foley 2011-10-11 14:50:21 UTC
Created attachment 527470 [details]
RHQ_RESOURCES from 4.1

Comment 6 Mike Foley 2011-10-11 15:23:17 UTC
Created attachment 527473 [details]
RHQ_RESOURCES from 2.4.1

Comment 7 Mike Foley 2011-10-11 15:34:59 UTC
verification failed at step #2 ... during the upgrade from JON 2.4.1 to RHQ latest (10/11/2011) build.  attaching documentation:  screenshot, text of error, upgrade log file, server log file

Comment 8 Mike Foley 2011-10-11 15:35:41 UTC
Created attachment 527476 [details]
error of upgrade failure

Comment 9 Mike Foley 2011-10-11 15:36:32 UTC
Created attachment 527477 [details]
text of upgrade error

Comment 10 Mike Foley 2011-10-11 15:37:14 UTC
Created attachment 527478 [details]
dbupgrade log

Comment 11 Mike Foley 2011-10-11 15:37:37 UTC
Created attachment 527479 [details]
server log

Comment 12 Lukas Krejci 2011-10-11 16:12:17 UTC
The upgrade errors are drift related. I think there has been some recent work on getting that work again.

John, could you please push this bug back to ON_QA once you believe the upgrade should work again?

Comment 13 John Sanda 2011-10-12 14:57:39 UTC
There was a dbupgrade failure on oracle due to duplicate constraint names. I reproduced, tested, and fixed locally.

commit hash: 709da8b2c35b04dc252190b4c22166e2a5081c9d

Comment 14 Libor Zoubek 2011-11-10 17:36:17 UTC
verification failed in step #7 I was verifying on JON 3.0.0.CR1

what I did was : select resource_key from rhq_resource to file. Then cat file | sort | uniq -u > file_sorted. (for both databases)

moving ON_DEV

Comment 15 Libor Zoubek 2011-11-10 17:37:57 UTC
Created attachment 532888 [details]
diffrences between resource keys in JON2->3 and JON3

Comment 16 Lukas Krejci 2011-11-16 10:48:29 UTC
Some of the entries in the diff can be explained by the resource upgrade facets on some of the resource types:

/opt/jboss-soa-p-5/jboss-as/bin/<script-name> -> <script-name> is due to the resource upgrade facet that changes the RK from an absolute path to a path relative to server's bin (http://git.fedorahosted.org/git?p=rhq/rhq.git;a=blob;f=modules/plugins/jboss-as-5/src/main/java/org/rhq/plugins/jbossas5/script/ScriptDiscoveryComponent.java;hb=HEAD#l99)

vfsfile:/opt/jboss-soa-p-5/jboss-as/server/<profile>/deploy/<deployment-name> -> {<profile>}<deployment-name> is again due to a resource upgrade facet (http://git.fedorahosted.org/git?p=rhq/rhq.git;a=blob;f=modules/plugins/jboss-as-5/src/main/java/org/rhq/plugins/jbossas5/AbstractManagedDeploymentDiscoveryComponent.java;hb=HEAD#l126)

What I can't explain are these:

* the apparent sudden appearance of the RHQ tables in the new version - was the postgres plugin not deployed / configured properly in the old version?

* /data/rhq/jon-server-3.0.0.CR1-upgrade/jon-server-3.0.0.CR1/jbossas/server/default - not sure how this could have disappeared in the new version. This seems to be a resource key of a JON server, which we definitely should be able to discover. Was the resource imported?

* PersistAction, RedeliverMessagesAction, action, notificationAction - not sure what these are - what is the resource type and resource hierarchy?

* boss.esb:category=MessageCounter,deployment=jbossesb.esb,service-category=JBossESB-Internal,service-name=DeadLetterService
 jboss.esb:category=MessageCounter,deployment=jbossesb.esb,service-category=JBossESB-Internal,service-name=RedeliverService
 jboss.esb:category=MessageCounter,deployment=jbpm.esb,service-category=JBossESB-Internal,service-name=JBpmCallbackService
 jboss.esb:deployment=jbossesb.esb,listener-name=JMS-DLQListener,service-category=JBossESB-Internal,service-name=DeadLetterService
 jboss.esb:deployment=jbossesb.esb,listener-name=redeliver-scheduled-listener,service-category=JBossESB-Internal,service-name=RedeliverService
 jboss.esb:deployment=jbpm.esb,listener-name=JMS-DCQListener,service-category=JBossESB-Internal,service-name=JBpmCallbackService

Not sure what these are either

Comment 17 Mike Foley 2011-11-17 16:02:23 UTC
libor/lukas ... just checking on this ... is there an issue here?  or is it resolved?

Comment 18 Libor Zoubek 2011-11-21 11:24:28 UTC
(In reply to comment #16)

> /data/rhq/jon-server-3.0.0.CR1-upgrade/jon-server-3.0.0.CR1/jbossas/server/default
> - not sure how this could have disappeared in the new version. This seems to be
> a resource key of a JON server, which we definitely should be able to discover.
> Was the resource imported?
We can ignore this one. this item is rhq server on instance 2->3. I forgot to import it on the clean (3 only) one - I did't not realize it would change.
> * PersistAction, RedeliverMessagesAction, action, notificationAction - not sure
> what these are - what is the resource type and resource hierarchy?
All these are ESB Actions :children of following ESB resources

jboss.esb:deployment=jbossesb.esb,listener-name=JMS-DLQListener,service-category=JBossESB-Internal,service-name=DeadLetterService
jboss.esb:deployment=jbossesb.esb,listener-name=redeliver-scheduled-listener,service-category=JBossESB-Internal,service-name=RedeliverService

I've compared both databases once, focusing on ESB resources, and I see them in both now. In fact, I was playing with jon3clean instance a few times since I created diff (just start/stop).

I don't see no issue here, except for one mystery: missing deployed ESB services/actions in jon3clean at one time.

Comment 19 Lukas Krejci 2011-11-21 11:37:34 UTC
Ok, so I think we can close this one. The mystery of missing resources should not affect their resource keys - they were just missing. If you are able to reproduce that Libor, please raise a separate BZ for it.

I'm flipping this BZ to VERIFIED. Please change that if you need to do more with it.

Comment 20 Mike Foley 2012-02-07 19:29:54 UTC
changing status of VERIFIED BZs for JON 2.4.2 and JON 3.0 to CLOSED/CURRENTRELEASE

Comment 21 Mike Foley 2012-02-07 19:30:26 UTC
marking VERIFIED BZs to CLOSED/CURRENTRELEASE


Note You need to log in before you can comment on or make changes to this bug.