Bug 824010

per BZ Triage 5/29/2012 (ccrouch, loleary, asantos, mfoley, myarborough) moving these to JON 3.1.1 or later

One idea, although it would still use the Java Preference API, is to simply have the agent's rhq-agent-env.sh define the location of the Java preference store directory using a path made absolute from a relative path to RHQ_AGENT_HOME or RHQ_AGENT_BIN_DIR_PATH. For example:

RHQ_AGENT_JAVA_PREFS_SYSTEM="$(cd ${RHQ_AGENT_BIN_DIR_PATH}/../..; pwd)/conf/.javaPrefsSystem"
RHQ_AGENT_JAVA_PREFS_USER="$(cd ${RHQ_AGENT_BIN_DIR_PATH}/../..; pwd)/conf/.javaPrefsUser"

...

RHQ_AGENT_ADDITIONAL_JAVA_OPTS="-Djava.util.prefs.systemRoot='${RHQ_AGENT_JAVA_PREFS_SYSTEM}' ${RHQ_AGENT_ADDITIONAL_JAVA_OPTS}"
RHQ_AGENT_ADDITIONAL_JAVA_OPTS="-Djava.util.prefs.userRoot='${RHQ_AGENT_JAVA_PREFS_USER}' ${RHQ_AGENT_ADDITIONAL_JAVA_OPTS}"



Obviously we should probably handle the case in where the locations are undefined and then not set -Djava.util.prefs.systemRoot/userRoot but this should at least give an idea.

Of course, the more I think about it, the more I realize that perhaps agent-configuration.xml is essentially the same? If we can resolve the configuration relative to the agent's binary and we can locate the agent's default configuration file, then why do we need Java Preferences at all?

Also, the above suggestion only works for Linux/Unix and doesn't handle the same problem in relation to a Windows system. Keep in mind that domain users and Windows roaming profiles and switching agent users from local accounts to system account means that the userPrefs node stored in the Windows registry becomes disconnected and we have the same issues there. I don't think the java.util.prefs.userRoot system property helps us out there.

Without realizing this BZ existed I ended up doing sort of what Larry was recommending in Comment 3.  It's not quite the same.

Basically we now have our own implementation of Java Preferences.  We extend AbstractPreferences and our implementation is a simple, single properties-style file storing the prefs.

The file can be put anywhere on the file system as long as the agent has write permissions to the create and manipulate the file.  In general we would expect people to want to put it under the Agent's conf directory.  In that way, when the Agent gets wiped it doesn't leave any Preferences litter.  This is especially nice on Windows where the native Java impl uses the registry.

To override the default Preferences you must set the following system prop:

java.util.prefs.PreferencesFactory=org.rhq.core.util.preferences.FilePreferencesFactory.file

By default it will create the file as ${user.home}/.fileprefs but in general users would want to set the file by setting the system property:

org.rhq.core.util.preferences.FilePreferencesFactory

For example, on Windows maybe:

org.rhq.core.util.preferences.FilePreferenceFactory.file=%RHQ_AGENT_HOME%\conf\agent-preferences.txt

master commit 3c08e936a3b3af65189e839d66c399a8ad9b835c
Author: Jay Shaughnessy <jshaughn>
Date:   Wed Mar 20 14:31:16 2013 -0400

    Adds a file-based implementation of java Preferences that allows us to
    override the native java impl and store prefs in a user-configured file.


One quick note: since the prefs file is basically a properties file, perhaps a better example than above:

org.rhq.core.util.preferences.FilePreferenceFactory.file=%RHQ_AGENT_HOME%\conf\agent-prefs.properties

Asking Larry if this satisfies the BZ...

master commit 9b525e0d1655d0e28ba086604391e5ac07bfc7b7
Author: Jay Shaughnessy <jshaughn>
Date:   Thu Mar 21 15:40:29 2013 -0400

    Rename the file name property to be more succinct:

    Was: org.rhq.core.util.preferences.FilePreferencesFactory.file
    Now: rhq.preferences.file

A couple of things. First off, is there any reason why we wouldn't make this %RHQ_AGENT_HOME%\conf\agent-prefs.properties by default? Using the user's home directory only makes sense if the application can be executed by multiple people and run at the same time. The RHQ agent is a service and we only intend and support a single instance per machine. It should not be bound to a specific user's home directory. Furthermore, on Windows, the user's home directory is normally stored on a remote domain controller (Active Directory or CIFS/Samba or using NT Roaming Profiles). 

Second, how will the existing properties and Java preference node implementation be migrated?

The purpose of the original request was to do the following:

 - pull agent configuration into the agent so the location of the agent's configuration is either stored in the agent installation or in a user-defined location. This prevents shared service accounts and domain level account from clobbering each other's configuration for agents across many hosts.

 - retain all agent configuration post upgrade. Essentially, all configuration (persisted) should be in one place and support modifications just like we currently do with agent-configuration.xml. The goal is a single place for configuration so there is no need to figure out which set of properties is used at one time (install time vs. runtime) and to ensure that customizations to configuration are maintained across updates.

This BZ represents part of the original request as it was to give the RHQ agent full control over its configuration by putting it in a file in a location well known to the agent that is not dependent on a OS specific location or user.

Larry, so far this is actually just a generic feature available to the user, and not really a new feature of the agent.  As such it probably does not yet fulfill the intent of the BZ.  The user.home default is just that, a default for what amounts to a generic utility.  The default file is not really a big deal, because the custom Preferences impl is not activated by default.

There are two things you bring up that may be ways the agent needs to leverage the the new Preferences impl.

First, it could be used as a default and not as an option.  There really isn't a reason why we couldn't do this as a default for the agent, always locating under %RHQ_AGENT_HOME%/conf.  

The second is handling upgrade/migration.  First, how do you make the switch to the new approach.  My only thought right now is this sort of logic:
  IF ( the custom Preferences impl is active and the custom file does not exist )
THEN
     1: temporarily set back to the native impl
     2: pull in the properties
     3:   IF ( legacy properties exist )
        THEN 
             1: pull then into memory
             2: clear the legacy properties
             3: switch back to the custom impl
             4) flush the properties
             5) continue with the new approach

Also related is how to handle agent upgrade.  I think if the custom impl is defined and the file exists under the agent home dir, we'd need to maintain the values.  Either by maintaining the file, or writing it back to the native impl and picking it back up after the upgrade (with the above logic).

Of course, another possibility is we don't even offer an option, we force use of the custom impl, using a non-custom, well-defined file under agentHome/conf. And  we manage upgrades.

The approach in comment 8 doesn't work since it's not possible to switch PreferenceFactory implementations in the JVM.

Taking into consideration the discussion so far the proposal is to do the following, which I think is very close, if not exactly, Larry's thinking.

We continue to use Java Preferences but we use the new, custom FilePreferences implementation.  The RHQ Agent always uses this impl. By default it will put the preferences in AGENT_HOME/conf/agent-prefs.properties.  The file location can be customized by the user by changing the agent's default value of the rhq.preferences.file system property. (Although not anticipated it will be possible to change the location by resetting the system property and moving file accordingly.)

New agents will simply start out with this approach.

Existing agents will have their preferences migrated during the agent auto-update process.  The updated agent will immediately start using the new file-based prefs. Future updates will keep the prefs file intact, if necessary copying it from the old /conf dir to the new /conf dir.

Would it make sense to have a one time migration tool, that just reads the old prefs and writes the properties file and then terminates without adding the functionality to the agent? Or is that too error prone for the user?

Storing the preferences at agent level (instead of per user) is good, as this allows to run multiple agents on a box.
On the other hand on Linux, normally binaries, variable data and conf data live in different places in the file system hierarchy.

To a degree this is a one time migration tool.  Since it's not built into the agent but rather the agentupdate task. So basically this is what you're suggesting , I think, but the benefit is that there is no manual step for the user, no additional scripts, etc, which are more points of failure for the user.

So, a user not interested in the preferences things, at all, will have no idea that the agent has started storing prefs under its conf directory.  They should basically just update and keep going.

The only thing built into the agent is that it will now always use the FilePreferences.

I think the approach sounds good. However, please keep in mind that my point was not to drive the complete feature implementation within this bug. It was only to highlight the bigger picture that this bug was part of.

I think the important points you have described help to accomplish that bigger goal.

 * by default, storing preferences in a file that the agent controls
 * by default, storing the preference file within the agent's installation directory itself
 * migrate existing properties from the legacy store to the new file store
 * allow the user to override the directory where the preference file is stored (this allows the RPM to store it in /etc/rhq-agent/... for example) 
 * allow the user to override the file name of the preference file (this allows multiple agents to store their configuration in the same location but with difference names)


Additional tasks that should probably happen in a different BZ:
 * Support the use of this preference file as the only source file for preferences. In other words, we need to get rid of agent-configuration.xml as this is essentially replacing it and two configuration files will become even more confusing then just having one that doesn't do anything.
 * Retain the new configuration file across agent upgrades.

master commit baef5c9d8ed4ca872ccb4f3240bbb603f00abd5c
Author: Jay Shaughnessy <jshaughn>
Date:   Mon Mar 25 14:01:25 2013 -0400

    The RHQ Agent now uses our custom FilePreferences Java Prefs impl to
    store the ageng configuration.  By default the prefs file is stored in
    RHQ_AGENT_HOME/agent-prefs.properties, but the file placement is
    configurable.  The value is set in the RHQ_AGENT_JAVA_OPTS
    as part of rhq-agent-env.sh|bat.

    New agents will start using the new prefs approach immediately. For
    existing agents the auto-update procedure will perform the one-time
    migration from the native prefs store to the file prefs. By default
    the native prefs will be removed as part of the migration, to leave
    the native prefs clean of RHQ debris.

    Also:
    - make sure streams get closed in FilePreferences impl

With respect to comment 12:
   "* Retain the new configuration file across agent upgrades."

This is already true as long as the file is under RHQ_AGENT_HOME/conf/ and the name matches *prefs*.  Or, if the file is outside of the agent directory it will not be touched during the agent update.


Also, a small commit:

master commit c696de2117ac5d377b7dc3165a71a7bd485fe8fe
Author: Jay Shaughnessy <jshaughn>
Date:   Mon Mar 25 15:01:03 2013 -0400

    - remove extraneous parens

master commit 399bc2863d82d499158c00ccc7ae4e3b50cdfa04
Author: Jay Shaughnessy <jshaughn>
Date:   Tue Mar 26 09:05:15 2013 -0400

    Don't use custom FilePreferences by default at this time.  Stick with
    native prefs storage by default.  We need to further evaluate upgrade
    scenarios.

master commit d3ee49b474616fdbd0b3829379520c6da1406ec0
Author: Jay Shaughnessy <jshaughn>
Date:   Tue Mar 26 09:56:10 2013 -0400

    Until it's clear we can wipe them, make sure the autoupdater migration
    task does not wipe the native prefs.

The FilePreferences implementation is available to users but not planned to be integrated into RHQ at this time.  Setting back to NEW without a target release.