Note: this probably has nothing to do with whether the property is required or not. Having a default value is the important part.

However, if the property is required, and you go to edit the plugin configuration for the resource, it will fail validation because its missing that property value.  You have to explicitly enter the value.

the ideal place to do it would be in the plugin update code, where you know exactly which prop defs are new

then you could find all resources of that type and update their plugin (or resource) configs

we already have code in the configdef update code that finds the list of new propdefs ... we only have to execute the update query that updates resources instances when there actually are one or more new propdefs which would not be that often.

the other advantage of doing it in the plugin metadata update code is that we'd update the existing resource plugin configs prior to the agent pulling down the plugin update so when the agent did pull down the plugin update and restarted the PC, server->agent sync would make sure the updated plugin configs got synced to the agent. To ensure the server->agent sync does kick in, make sure the mtime is updated on any Resources whose plugin configs got updated

This is the main piece that needs to be changed:

org.rhq.enterprise.server.resource.metadata.ResourceMetadataManagerBean.updatePluginConfiguration(ResourceType, ResourceType)

In here we need to check to see if the plugin configs found in the two arguments (resourceType and existingType) are the same or different (more specifically, if resourceType's plugin config has 1 or more NEW property definitions compared to existing plugin config). If so, we need to go through all resources of that resource type and ADD new config properties to those resources' plugin configs that match the defaults for all props that are new (plus set that resource's mtime to the current time).

If a property definition does not set a default, then we should not add the property (or if we have to set it, then we must set its value to null).

we could add methods here:

org.rhq.core.clientapi.agent.configuration.ConfigurationUtility

such as:


/** return true if "new" has 1 or more new definitions compared to "old" */
boolean hasNewPropertyDefinitions(Configuration Definition old, ConfigurationDefinition new)

/** adds new properties to the given config - where the new props are the definitions that are added to the new def compared the old def */
void addNewDefaultProperties(Configuration config, ConfigurationDefinition old, ConfigurationDefinition new)

With this, the metadata SLSB bean can compare the two definitions (that first method above). If there is new prop defs, the SLSB bean needs to get all resources of the resource type being processed, and for each one, needs to load the the plugin config, addNewDefaultProperties to it, and recommit that config to the database, and chanaging the resource's mtime.

This will be expensive, but it has to get done. We should output log messages saying whats happening so people know to wait for this process to complete.

I have reviewed this with mazz and decided on the following. ResourceMetadataManagerBean.updatePluginConfiguration(ResourceType, ResourceType) calls ConfigurationMetadataManagerBean.updateConfigurationDefinition(ConfigurationDefininition, ConfigurationDefinition) to do the actual update to the plugin configuration definition. This method currently returns void. I am going to refactor it to return a sort of a report that will among other things detail which new property definitions have been added to the configuration definition.

I have pushed several commits to the bugfixes branch with updates to ResourceMetadataManagerBean and ConfigurationMetadataManagerBean that will update the plugin configurations for existing resources with any new required properties using their default values. After some further review it looks like additional, similar changes are needed for templates as well.

I spent some more timing testing changes with templates, and I do not think that there is additional work that needs to be done for them. Modifying a template definition does not appear to effect an existing resource plugin configurations.

Changing the status to ON_QA as the work has been merged into master.

Steps to reproduce:

1. Manually add a JMX server to inventory. Note that you can actually use any plugin, but I have been using this one for testing.

2. Go ahead and use the default template

3. Now open up the plugin descriptor and add the following property to the <plugin-configuration>,

<c:simple-property name="foo" required="true" default="bar"/>

4. Rebuild and redeploy the jmx plugin,

$ mvn clean install -DskipTests -Pdev

5. Monitor the server log for the plugin meta data update to be reported. You should see something along the lines of,

[org.rhq.enterprise.server.core.plugin.ProductPluginDeployer] Newer version of [JMX] plugin found (version 3.0.0-SNAPSHOT) - older version (3.0.0-SNAPSHOT) will be ignored.
2010-04-01 10:04:10,058 INFO  [org.rhq.enterprise.server.core.plugin.ProductPluginDeployer] Deploying [1] new or updated agent plugins: [JMX]
2010-04-01 10:04:19,179 INFO  [org.rhq.enterprise.server.core.plugin.ProductPluginDeployer] Plugin metadata updates are complete for [1] plugins: [JMX]

6. Once the plugin update has completed go to the connection properties in the UI for your JMX server

7. You should see the new property, 'foo', with a default value of 'bar'.

Here are steps involving the use of templates:

1. Manually add a JMX server to inventory.

2. Select the JDK 5 template when adding the resource

3. Now open up the plugin descriptor and add the following property to the
<plugin-configuration>,

<c:simple-property name="fooBar" required="true" default="bar"/>

and in the JDK 5 template add,

<c:simple-property name="fooBar" required="true" default="bam"/>

4. Rebuild and redeploy the jmx plugin,

$ mvn clean install -DskipTests -Pdev

5. Monitor the server log for the plugin meta data update to be reported. 

6. Once the plugin update has completed go to the connection properties in the
UI for your JMX server

7. You should see the new property, 'foo', with a default value of 'bar'.  

Additional Notes:
Even though we initially selected the JDK 5 template, we only deal with the template at the time the resource is being created and added into inventory. Beyond that we do not track which template is being used; so, we have no way to update the resource's plugin configuration with the value we supplied in the JDK 5 template. Instead we fall back to the value supplied in the default template.

Adding and updating the property to the value specified in the template would require a substantial amount of additional work which I believe is beyond the scope of this bug.

John, thanks for the detailed steps. I was able to see that the updated plugin is identifed in log file. 

2010-04-08 21:00:33,289 INFO  [RHQ Server Polling Thread] (org.rhq.enterprise.agent.PluginUpdate)- {PluginUpdate.downloading}Downloading the plugin [rhq-jmx-plugin-3.0.0-SNAPSHOT.jar]...
2010-04-08 21:00:33,497 INFO  [RHQ Server Polling Thread] (org.rhq.enterprise.agent.PluginUpdate)- {PluginUpdate.downloading-complete}The plugin [JMX] has been updated at [rhq-jmx-plugin-3.0.0-SNAPSHOT.jar].
2010-04-08 21:00:33,497 INFO  [RHQ Server Polling Thread] (org.rhq.enterprise.agent.PluginUpdate)- {PluginUpdate.updating-complete}Completed updating the plugins to their latest versions.

However, I don't see fooBar in the connection descriptor in UI for JMX. 

I'm looking at, Resource -> All Resource -> JMX Server 'Java VM' -> Inventory -> Connection.

Did you add fooBar in the default template (i.e., outside of any templates) as well as in the JDK 5 template? Maybe you could attach your plugin descriptor so I can see exactly what it looks like.

Thanks

John

Alternative verification steps.

1) Install JON 2.2 into a fresh db. 
2) Setup a secured EAP 4.3 instance
3) Discover and import the EAP4.3 and set its credentials so its services are discovered and it goes green
4) Upgrade to the latest JON build, keeping the existing data
5) See the EAP4.3 stay green for a while, and that it is working fine, e.g. deploy a Datasource.
6) Go to the inventory tab of the EAP resource and edit its connection properties. Hit Enter without changing any values.
7) The page should save successfully [with a success message]. In 2.3.1 the page fails to save [with no error message at all, the page just reloads] due to some of the Snapshot related connection properties not having any default values.

We should do these as well as any over tests.

@John, I have sent you the plugin xml in mail. 

@Charles, Will work on the alternative verification steps.

Hi Sudhir,

Thanks for the plugin descriptor. I recreated the plugin jar using your descriptor and went through the steps deploying on geeta's rhq server. I am seeing the new property. The name of my jmx server is John's Test JMX Server 1. Incidentally, I created my plugin jar through the maven build. How are you creating you updated jar?

Interesting. 

I create and deploy the modified jar as bleow,

1. Stop the agent.
2. Extract the jmx jar. 
3. Edit /META-INF/rhq-plugin.xml as earlier to include foobar that I had sent via mail.
4. cd to ./META-INF/maven/org.rhq/rhq-jmx-plugin/pom.xml
5. run, mvn clean install -DskipTests -Penterprise (I'm using enterprise and not dev)
6. mv ./META-INF/maven/org.rhq/rhq-jmx-plugin/target/rhq-jmx-plugin-3.0.0-SNAPSHOT.jar to plugins under rhq-agent.
7. Start the agent.

Oh oh oh... I just extracted the newly build jmx jar and checked if it contained the changes... but it did not!! Obviously I'm doing something wrong here.. what am I missing?

A couple things. First, the plugin should only be deployed to the server. You can update your agent by either 1) running the command, plugins update, from the agent command prompt or 2) restart your agent. Secondly, if you have and can build the plugin from source, then just modify the plugin descriptor at rhq/modules/plugins/jmx/src/main/resources/META-INF/rhq-plugin.xml and then rebuild it with maven as you did. 

And then if want to restore the file to match HEAD you can run the git command, git checkout -f path/to/rhq-plugin.xml

Thanks John. I have verified this bug. I can see the default value "bar" loaded.

Marking it as verified.

Re-opening for more QA. I'm looking for confirmation that the steps in Comment 12 (testing upgrade from 2.2->2.4) were executed successfully AND that the following worked (testing upgrade from 2.2->2.3->2.4):

1) Install JON 2.2 into a fresh db. 
2) Setup a secured EAP 4.3 instance
3) Discover and import the EAP4.3 and set its credentials so its services are
discovered and it goes green
4) Upgrade to JON 2.3.1, keeping the existing data
5) Make sure the EAP4.3 stays green for a while, and that it is working fine, e.g.
deploy a Datasource.
6) Go to the inventory tab of the EAP resource and edit its connection
properties. Change some harmless connection property and hit Enter.
7) The page should reload with no success message, if you open up all the sections on this page you should see validations errors due to
some of the Snapshot related connection properties not having any default
values.
8) Upgrade to the latest JON build, keeping the existing data
9) Make sure the EAP4.3 stays green for a while, and that it is working fine, e.g.deploy a Datasource.
10) Go to the inventory tab of the EAP resource and edit its connection
properties. Change some harmless connection property and hit Enter.
11) The page should save successfully [with a success message].

Upping priority since I'd like to get this QE'd prior to the beta

Heiko
Can you please step in and do the QA mentioned in comments 12 and 19.
Thanks
Charles

Number 19 is clearly failing for me with JON build #172 - the properties are not auto-populated and updating the connection properties (switching shutdown from jmx to script) fails (without a failure message, but with showing the un-populated fields+ message next to them).

Same for #12

With mazz, to investigate whats up with the fix

Ran the test procedure that Charles spells out and I see what the problem is.

The issue is the new properties got introduced in a release prior to this release with the fix. Thus, the "new" properties look "old" to the new code that fixes this problem, thus nothing is done. Here's examples:

1) this works:

a) install plugin with required plugin config property A
b) inventory resource, now has required config property A
c) shutdown, install server with new fix and with new plugin with plugin config property B
d) start server and notice resource has config property A and required B with default value set

Horray it works. That's because when the new server with the new fix ran, it saw the plugin had a new B property introduced during plugin upgrade. Now watch - this doesn't work:

a) install jon server 2.2 with original plugin with config property A
b) inventory resource, now has required config property A
c) install jon server 2.3.1 (this of course DOES NOT HAVE THE NEW FIX) with new plugin that has new plugin config B
d) start server and notice the problem that this BZ reports
e) upgrade to new server that has the new fix in it. when it starts, the plugin is deployed but THERE ARE NO NEW PLUGIN CONFIGS THIS TIME AROUND. the new plugin config B exists in the "old" plugin config definition, so the new server just thinks, "well, nothing new was introduced so there's nothing to do".

There is nothing we can do automatically here because we don't know if B really should be left null or not (perhaps the user explicitly unset it - we don't know).

We need to write a CLI script to help those with legacy JBossAS 4.x and RHQ Agent resources in inventory (because those two are the only ones with these additional properties). Users that want to avoid having to explicitly enter values for these required settings can run the CLI one time with this custom script to fix this. The alternative is the user can go individually to the resources in question and manually set the required plugin configs. The CLI script is only for those with legacy data that want to quickly set all the required snapshot properties without doing it manually.

This bug fix will remain as-is because it will help plugin changes that occur from now on into the future. The only problem left is this legacy issue when an old plugin introduced required properties.

John Sanda will write a CLI script to do this and will attach it to this BZ when finished.

I may be incorrect here:

"There is nothing we can do automatically here because we don't know if B really
should be left null or not (perhaps the user explicitly unset it - we don't
know)."

Because we are only worried about "required" properties, and required properties CAN'T be set to null by the user! The only way they could have been null was during the initial plugin upgrade when the config definition was created but existing resources don't pick up the new config settings.

That said, for us to fix this would require us, at plugin upgrade time, to scan every resource in inventory and look for nulled out "required" properties and if we find any, set them to their definition's default. Checking the entire inventory's config will be very expensive.

IMO, requiring a one-time CLI script invocation should be OK. We are, after all, only needing to fix these legacy properties (and you only need to run the CLI script once). Any future changes will be automatically fixed with the patch that jsanda added as part of his work on this BZ.

It seems to me that the more appropriate fix would simply be that when a "required" configuration value was read and its value is null, that it get set to the definitions default value.

(In reply to comment #27)
> It seems to me that the more appropriate fix would simply be that when a
> "required" configuration value was read and its value is null, that it get set
> to the definitions default value.    

That's what the script jsanda is putting together will do.

Created attachment 423819 [details]
CLI script to reset snapshot report connection properties

(In reply to comment #28)

I understand that but my point is that it should happen by the configuration manager.  When a property is read from the DB that contains a null value and the definition indicates that it can not be null (is required) then the default should be used if available.

I do not think extra scripts or applications are ever a very good solution to a configuration management issue.

(In reply to comment #30)
> I understand that but my point is that it should happen by the configuration
> manager.  When a property is read from the DB that contains a null value and
> the definition indicates that it can not be null (is required) then the default
> should be used if available.

the problem with that is you would now add additional processing in configuration handling code for EVERY configuration property loaded (there are LOTS of them and loading configuration can be expensive due to the recursive nature of its storage - we don't want to add to the processing of config loading).

The way we fixed the code is this will never happen now, that is, a required property will now never be null when initially inserted. Therefore, the ONLY time this will occur is with legacy config properties from an old version. Thus, it makes sense to run a one-time operation (like a cli script) to clean up the very small amount of configs where this is affected (again, only JBossAS server resources and RHQ Agent server resources are affected) and thereafter not have to incur any additional penalty for ALL config for ALL resources.

And finally, even if the script isn't run, this isn't a detrimental bug that has no workaround. Things still work - if the user wants to update the config, he merely has to enter the values manually.

Therefore, because the bug isn't detrimental, has a easy workaround, has a one-time script you can use to bulk change the configs if you want an easier workaround, and runtime code fix in config-manager would otherwise cause detrimental performance implications, we opted for the current solution.

jsanda attached the CLI script that you use one-time-only to convert your legacy plugin configs so they are not-null. Once you run this CLI script, your plugin configs are set correctly and you don't have to worry about this anymore.

John - can you provide a quick blurb on how to execute this CLI script just to give people a bit of documentation if this hit this problem and want to fix it with your script?

To run the script, first start up the CLI shell and execute the following commands,

$ login <username> <password>
$ exec -f path/to/reset_conn_props.js

The script is in the git repo as well in the release-3.0.0 branch located at rhq/etc/cli-scripts/reset_conn_props.js. It can be viewed at http://git.fedorahosted.org/git/?p=rhq/rhq.git;a=blob;f=etc/cli-scripts/reset_conn_props.js;h=499363848e7d7d34d4ac56f2d99340c3ad2b515b;hb=refs/heads/release-3.0.0. 

Right now the script uses some hard-coded values to reset properties for the jboss and agent plugins. It may be worthwhile for me to refactor the script so that it can support resetting any connection properties that the caller specifies.

Tested as mentioned in comment-19 with version: 2.4.0.GA build number: 10894:2f8ffb1

Steps:
1) Install JON 2.2 into a fresh db. 
2) Setup a secured EAP 4.3 instance
3) Discover and import the EAP4.3 and set its credentials so its services are
discovered and it goes green
4) Upgrade to JON 2.3.1, keeping the existing data
5) Make sure the EAP4.3 stays green for a while, and that it is working fine,
e.g.
deploy a Datasource.
6) Go to the inventory tab of the EAP resource and edit its connection
properties. Change some harmless connection property and hit Enter.
7) The page should reload with no success message, if you open up all the
sections on this page you should see validations errors due to
some of the Snapshot related connection properties not having any default
values.
8) Upgrade to the latest JON build, keeping the existing data
9) Make sure the EAP4.3 stays green for a while, and that it is working fine,
e.g.deploy a Datasource.
10) Go to the inventory tab of the EAP resource and edit its connection
properties. Change some harmless connection property and hit Enter.
11) The page should save successfully [with a success message].    

Observation:
It failed at step-11: Still its not giving success message and if user open up all the sections on this page user can see validation errors due to some of the Snapshot related connection properties not having any default values.

From https://bugzilla.redhat.com/show_bug.cgi?id=573034#c34

"Observation:
It failed at step-11: Still its not giving success message and if user open up
all the sections on this page user can see validation errors due to some of the
Snapshot related connection properties not having any default values."

This is expected, just upgrading to 2.4 will not fix the issue. Once you've upgraded from 2.2 to 2.3 you need to run the script attached by John Sanda in order to resolve the problem. If you run it after step 7) below you should be able to repeat step 6) and have it work.

The other test that is needed is to run through steps 1) to 11) but instead of upgrading from 2.2 to 2.3.1 to upgrade from 2.2 to 2.4. In the latter case step 11) should pass.

There was a typo in the CLI script in the resource branch. I fixed the error. I have committed the change though in master since 1) the script is not packaged with release and 2) since I plan refactoring it and making some enhancements for better reuse. The script is in the same location in the source tree and can be viewed at http://git.fedorahosted.org/git/?p=rhq/rhq.git;a=blob;f=etc/cli-scripts/reset_conn_props.js;h=95997f20fd8fbe68188a8ae41ad3c15c8c4d7430;hb=4c30f50c40e619a5b8c4a08ff2d5d87b1877029d.

Retested with updated JS script (as in comment-36) as per suggested in comment-35

Case-1:
1) Install JON 2.2 into a fresh db. 
2) Setup a secured EAP 4.3 instance
3) Discover and import the EAP4.3 and set its credentials so its services are
discovered and it goes green
4) Upgrade to JON 2.3.1, keeping the existing data
5) Make sure the EAP4.3 stays green for a while, and that it is working fine,
e.g.
deploy a Datasource.
6) Go to the inventory tab of the EAP resource and edit its connection
properties. Change some harmless connection property and hit Enter.
7-a) The page should reload with no success message, if you open up all the
sections on this page you should see validations errors due to
some of the Snapshot related connection properties not having any default
values.
7-b) Run updated JS script (as in comment-36)
8) Upgrade to the latest JON build, keeping the existing data
9) Make sure the EAP4.3 stays green for a while, and that it is working fine,
e.g.deploy a Datasource.
10) Go to the inventory tab of the EAP resource and edit its connection
properties. Change some harmless connection property and hit Enter.
11) The page should save successfully [with a success message].    

Case-2:
1) Install JON 2.2 into a fresh db. 
2) Setup a secured EAP 4.3 instance
3) Discover and import the EAP4.3 and set its credentials so its services are
discovered and it goes green
4) Upgrade JON2.2 to JON2.4
5) Run updated JS script (as in comment-36)
6) Go to the inventory tab of the EAP resource and edit its connection
properties. Change some harmless connection property and hit Enter.
7) The page should save successfully [with a success message]. 

Observation:
In both cases, still its not giving success message and if user open up
all the sections on this page user can see validation errors due to some of the
Snapshot related connection properties not having any default values.
(failed step-11 for case-1 and failed step-7 for case-2)

The test cases are still wrong, they should be as follows

Case-1:
1) Install JON 2.2 into a fresh db. 
2) Setup a secured EAP 4.3 instance
3) Discover and import the EAP4.3 and set its credentials so its services are
discovered and it goes green
4) Upgrade to JON 2.3.1, keeping the existing data
5) Make sure the EAP4.3 stays green for a while, and that it is working fine,
e.g.
deploy a Datasource.
6) Go to the inventory tab of the EAP resource and edit its connection
properties. Change some harmless connection property and hit Enter.
7-a) The page should reload with no success message, if you open up all the
sections on this page you should see validations errors due to
some of the Snapshot related connection properties not having any default
values.
7-b) Run updated JS script (as in comment-36)
8) Go to the inventory tab of the EAP resource and edit its connection
properties. Change some harmless connection property and hit Enter.
9) The page should save successfully [with a success message].    

Case-2:
1) Install JON 2.2 into a fresh db. 
2) Setup a secured EAP 4.3 instance
3) Discover and import the EAP4.3 and set its credentials so its services are
discovered and it goes green
4) Upgrade JON2.2 to JON2.4, keeping the existing data
5) Go to the inventory tab of the EAP resource and edit its connection
properties. Change some harmless connection property and hit Enter.
6) The page should save successfully [with a success message].

Retested both the cases:

Case-1:
1) Install JON 2.2 into a fresh db. 
2) Setup a secured EAP 4.3 instance
3) Discover and import the EAP4.3 and set its credentials so its services are
discovered and it goes green
4) Upgrade to JON 2.3.1, keeping the existing data
5) Make sure the EAP4.3 stays green for a while, and that it is working fine,
e.g.
deploy a Datasource.
6) Go to the inventory tab of the EAP resource and edit its connection
properties. Change some harmless connection property and hit Enter.
7-a) The page should reload with no success message, if you open up all the
sections on this page you should see validations errors due to
some of the Snapshot related connection properties not having any default
values.
7-b) Run updated JS script (as in comment-36)
8) Go to the inventory tab of the EAP resource and edit its connection
properties. Change some harmless connection property and hit Enter.
9) The page should save successfully [with a success message].    

Observation: Still step-9 failed, its not giving success message.

CLI console:
rhqadmin.193.1:7080$ exec -f /home/rtimaniy/Desktop/reset_conn_props.js
DEBUG   Found 0 JBossAS Server resources 
DEBUG   Loading plugin configuration for [resourceType=JBossAS Server, plugin=JBossAS]
DEBUG   Found 0 RHQ Agent resources 
DEBUG   Loading plugin configuration for [resourceType=RHQ Agent, plugin=RHQAgent]


Case-2: 
1) Install JON 2.2 into a fresh db. 
2) Setup a secured EAP 4.3 instance
3) Discover and import the EAP4.3 and set its credentials so its services are
discovered and it goes green
4) Upgrade JON2.2 to JON2.4, keeping the existing data
5) Go to the inventory tab of the EAP resource and edit its connection
properties. Change some harmless connection property and hit Enter.
6) The page should save successfully [with a success message].   

Observation: Still step-6 failed, its not giving success message.

For case-1, I reproduced the same results from the CLI. I wrote the script against the 2.4.0 api. The functions in the script that query for the resources were returning empty result sets. I have updated the script so that it should not work against both 2.3.1 and 2.4.0.

commit hash: ebe40d281993c044317e9004bf1019e485b2611c

John, can you please point to updated JS file?

Got updated JS file: <jsanda> http://git.fedorahosted.org/git/?p=rhq/rhq.git;a=blob;f=etc/cli-scripts/reset_conn_props.js;h=bb2ac4654ceb038f63374552996d479e0cd8b346;hb=ebe40d281993c044317e9004bf1019e485b2611c

Retested case-1:
1) Install JON 2.2 into a fresh db. 
2) Setup a secured EAP 4.3 instance
3) Discover and import the EAP4.3 and set its credentials so its services are
discovered and it goes green
4) Upgrade to JON 2.3.1, keeping the existing data
5) Make sure the EAP4.3 stays green for a while, and that it is working fine,
e.g.
deploy a Datasource.
6) Go to the inventory tab of the EAP resource and edit its connection
properties. Change some harmless connection property and hit Enter.
7-a) The page should reload with no success message, if you open up all the
sections on this page you should see validations errors due to
some of the Snapshot related connection properties not having any default
values.
7-b) Run updated JS script (as in comment-36)
8) Go to the inventory tab of the EAP resource and edit its connection
properties. Change some harmless connection property and hit Enter.
9) The page should save successfully [with a success message].    

Observation:
After exec. updated JS on CLI, all JBossAS server shows 'DOWN'.
JBossEAP 4.3.0's INVENTORY -> CONNECTION shows error in connection - The agent reported the following error on its last attempt (7/28/10, 5:07:12 PM, IST) to connect to this resource:Failure

CLI console:
rhqadmin.193.1:7080$ exec -f /home/rtimaniy/Desktop/reset_conn_props.js
DEBUG   Found 3 JBossAS Server resources 
DEBUG   Loading plugin configuration for [resourceType=JBossAS Server, plugin=JBossAS]
DEBUG   Preparing to reset plugin configuration properties for Resource[id=500052, type=JBossAS Server, key=/NotBackedUp/install/jon2.2_post/jon-server-2.2.0.GA/jbossas/server/default, name=rajanlaptop RHQ Server,  JBossAS 4.2.1.GA default (0.0.0.0:2099), version=4.2.1.GA]
DEBUG   Updated plugin configuration for Resource[id=500052, type=JBossAS Server, key=/NotBackedUp/install/jon2.2_post/jon-server-2.2.0.GA/jbossas/server/default, name=rajanlaptop RHQ Server,  JBossAS 4.2.1.GA default (0.0.0.0:2099), version=4.2.1.GA]
DEBUG   Preparing to reset plugin configuration properties for Resource[id=500053, type=JBossAS Server, key=/NotBackedUp/software/jboss-eap-4.3/jboss-as/server/production, name=rajanlaptop JBossEAP 4.3.0.GA production (10.65.193.1:1099), version=4.3.0.GA]
DEBUG   Updated plugin configuration for Resource[id=500053, type=JBossAS Server, key=/NotBackedUp/software/jboss-eap-4.3/jboss-as/server/production, name=rajanlaptop JBossEAP 4.3.0.GA production (10.65.193.1:1099), version=4.3.0.GA]
DEBUG   Preparing to reset plugin configuration properties for Resource[id=10041, type=JBossAS Server, key=/NotBackedUp/install/jon2.3.1_post/jon-server-2.3.1.GA/jbossas/server/default, name=rajanlaptop RHQ Server,  JBoss AS 4.2.3.GA default (0.0.0.0:2099), version=4.2.3.GA]
DEBUG   Updated plugin configuration for Resource[id=10041, type=JBossAS Server, key=/NotBackedUp/install/jon2.3.1_post/jon-server-2.3.1.GA/jbossas/server/default, name=rajanlaptop RHQ Server,  JBoss AS 4.2.3.GA default (0.0.0.0:2099), version=4.2.3.GA]
DEBUG   Found 1 RHQ Agent resources 
DEBUG   Loading plugin configuration for [resourceType=RHQ Agent, plugin=RHQAgent]
DEBUG   Preparing to reset plugin configuration properties for Resource[id=500051, type=RHQ Agent, key=10.65.193.1 RHQ Agent, name=10.65.193.1 RHQ Agent, version=1.3.1.GA]
DEBUG   Updated plugin configuration for Resource[id=500051, type=RHQ Agent, key=10.65.193.1 RHQ Agent, name=10.65.193.1 RHQ Agent, version=1.3.1.GA]

Created attachment 434991 [details]
screenshot for JBossAS down and connection fail

Created attachment 434992 [details]
JBossEAP43_connection_UI_error_stacktrace

(In reply to comment #40)
> For case-1, I reproduced the same results from the CLI. I wrote the script
> against the 2.4.0 api. The functions in the script that query for the resources
> were returning empty result sets. I have updated the script so that it should
> not work against both 2.3.1 and 2.4.0.
> 
> commit hash: ebe40d281993c044317e9004bf1019e485b2611c    

There was a typo in this comment. Should have read, "...updated the script so it now works against both 2.3.1 and 2.4.0." as opposed to not work.

After further debugging, I see why the CLI script is essentially wiping out the connection properties. When run against a 2.3.1 server, the function that fetches resources specifies in the criteria to also fetch the pluginConfiguration property of the Resource objects; however, the plugin configurations are not getting fetched along with the resources. Instead an empty configuration is returned. And that configuration object is used to update the connection properties. This results in removing all existing connection properties.

I will need to rewrite some of the logic for how the plugin configurations are fetched when running the script against a 2.3.1 server.

Additional changes were needed to apply defaults when run against a 2.3.1 server. The script originally fetched the default values for connection properties from the default template and used the values from the default template to apply default values to connection properties of resources. This is a nice solution because it is generic and applicable across resource types; however, since 2.3.1 does not have the code changes in the plugin upgrade code, the new properties do not exist in the default template in 2.3.1. Consequently, the values are essentially hard-coded on a per resource type basis.

commit hash: a2addcee76d260a82fff7faf848773a5932bd803

Got updated JS file:<jsanda> rajan: http://git.fedorahosted.org/git/?p=rhq/rhq.git;a=blob;f=etc/cli-scripts/reset_conn_props.js;h=274bc5c62d5c8f0da9304a8c2768bb11f02b4eb1;hb=a2addcee76d260a82fff7faf848773a5932bd803

Tested case-1A:
1) Install JON 2.2 into a fresh db. 
2) Setup a secured EAP 4.3 instance
3) Discover and import the EAP4.3 and set its credentials so its services are
discovered and it goes green
4) Upgrade to JON 2.3.1, keeping the existing data
5) Make sure the EAP4.3 stays green for a while, and that it is working fine,
e.g.
deploy a Datasource.
6) Go to the inventory tab of the EAP resource and edit its connection
properties. Change some harmless connection property and hit Enter.
7-a) The page should reload with no success message, if you open up all the
sections on this page you should see validations errors due to
some of the Snapshot related connection properties not having any default
values.
7-b) Run updated JS script 
8) Go to the inventory tab of the EAP resource and edit its connection
properties. Change some harmless connection property and hit Enter.
9) The page should save successfully [with a success message].

Observation:
After run JS script step-9 passed 

Tested case-1B:
1) Install JON 2.2 into a fresh db. 
2) Setup a secured EAP 4.3 instance
3) Discover and import the EAP4.3 and set its credentials so its services are
discovered and it goes green
4) Upgrade to JON 2.4 (build #93), keeping the existing data
5) Make sure the EAP4.3 stays green for a while, and that it is working fine,
e.g.
deploy a Datasource.
6) Go to the inventory tab of the EAP resource and edit its connection
properties. Change some harmless connection property and hit Enter.
7-a) The page should reload with no success message, if you open up all the
sections on this page you should see validations errors due to
some of the Snapshot related connection properties not having any default
values.
7-b) Run updated JS script 
8) Go to the inventory tab of the EAP resource and edit its connection
properties. Change some harmless connection property and hit Enter.
9) The page should save successfully [with a success message].

Observation:
After run JS script step-9 passed 

Need to test case-2, once we have updated build.

*** Bug 619248 has been marked as a duplicate of this bug. ***

jsanda/mazz - can we get a 5-cent explanation of what's going on here?

John S. fixed this (in master) and provided a CLI script to correct legacy resources already in inventory. He has a link to that CLI script in this BZ (see reset_conn_props.js)

For the 5-cent description of the problem we are trying to solve, see the "description" of this issue - it explains the issue (https://bugzilla.redhat.com/show_bug.cgi?id=573034#c51)

jsanda - if this is done, please push to ON_QA

Tested case 2 with upgrade from Jon2.2 to Jon241 build#42 (build number: 10963:0a53ca3)

Case-1: FAILED
1) Installed JON 2.2 into a fresh db. 
2) Setup a secured EAP 4.3 instance
3) Discovered and imported the EAP4.3 and set its credentials so its services are
discovered and it goes green
4) Upgraded JON2.2 to JON2.4.1, keeping the existing data
5) Navigated to the inventory tab of the EAP resource and edited its connection
properties. Changed 'Binding Address' connection property and clicked OK.

Observation: It gives validations errors due to connection properties not having any default values.

Case-2 (Using CLI-JS): PASSED

JS location: http://git.fedorahosted.org/git/?p=rhq/rhq.git;a=blob;f=etc/cli-scripts/reset_conn_props.js;h=274bc5c62d5c8f0da9304a8c2768bb11f02b4eb1;hb=a2addcee76d260a82fff7faf848773a5932bd803

Steps:
1) Installed JON 2.2 into a fresh db. 
2) Setup a secured EAP 4.3 instance
3) Discovered and imported the EAP4.3 and set its credentials so its services are
discovered and it goes green
4) Upgraded to JON 2.4.1, keeping the existing data
5) Observed EAP4.3 stays green.
6) Navigated to the inventory tab of the EAP resource and edited its connection
properties. Changed 'Binding Address' connection property and clicked OK.

7-a) The page reloaded with no success message, opening up all the
sections on this page display validations errors due to connection properties not having any default values.

7-b) Run updated JS script 

8) Navigated to the inventory tab of the EAP resource and edited its connection
properties. Changed 'Binding Address' connection property and clicked OK.

9) The page displays success message and saves the connection properties.    


Reopening the bug as it gives validation errors on Jon241 build#42 if we don't run the JS script on cli.

Turns out I had committed a fix for this but it only went into master. Here is the commit hash,

f0fffb5636ef2cdf89f6a1a1b1808e5d2fd98b6c

And here is a summary of the changes from the commit log,
    
Adding logic to handle three additional scenarios each of which involves     connection properties that are required and specify a default value:
    
  1) A new property is added to a new group
  2) An existing property not part of a group added to an existing group
  3) A property previously in a group moved outside of a group
    
There were also some scenarios when properties were moved into or out of a group and the group reference was not getting set or unset. ConfigurationMetadataManagerBean has been modifed to update these references.

I cherry picked the commit over to the release-3.0.0 branch. Moving back to ON_QA so this can be retested in the next build that includes this commit.

Verified with upgrade from Jon2.2 to Jon241 build#47 (build number: 10963:e45ddc8)


1) Installed JON 2.2 into a fresh db. 
2) Setup a secured EAP 4.3 instance
3) Discovered and imported the EAP4.3 and set its credentials so its services
are
discovered and it goes green
4) Upgraded JON2.2 to JON2.4.1, keeping the existing data.
5) Navigated to the inventory tab of the EAP resource and edited its connection
properties. Changed 'Binding Address' connection property and clicked OK.
6) The page is saved successfully with a success message 'Connection properties updated'.
Marking this as verified.

This has been done & verified a long time ago