Bug 1232006

rhq-agent-env.sh needs to have the RHQ_AGENT_JAVA_OPTS default value split into a RHQ_AGENT_JAVA_OPTS and a RHQ_AGENT_PREF_PROPERTIES value.  For example, the current default value in rhq-agent-env.sh:
#RHQ_AGENT_JAVA_OPTS="-Xms64m -Xmx128m -Djava.net.preferIPv4Stack=true -Drhq.preferences.file=${RHQ_AGENT_HOME}/conf/agent-prefs.properties"

should be changed to:
#RHQ_AGENT_JAVA_OPTS="-Xms64m -Xmx128m -Djava.net.preferIPv4Stack=true"
#RHQ_AGENT_PREF_PROPERTIES="-Drhq.preferences.file=${RHQ_AGENT_HOME}/conf/agent-prefs.properties"

This will allow the user to still have a template/example value to work with.

Alternatively, removing the quotes from around $RHQ_AGENT_PREF_PROPERTIES in the exec commands in rhq-agent.sh also provides a fix to the error.

# Run the VM - put it in background if the caller wants it to be
if [ -z "$RHQ_AGENT_IN_BACKGROUND" ]; then
   exec $RHQ_JAVA_EXE_FILE_PATH "$_JAVA_ENDORSED_DIRS_OPT" "$_JAVA_LIBRARY_PATH_OPT" $RHQ_AGENT_JAVA_OPTS $RHQ_AGENT_PREF_PROPERTIES $RHQ_AGENT_ADDITIONAL_JAVA_OPTS $_LOG_CONFIG -cp "$CLASSPATH" $RHQ_AGENT_MAINCLASS $RHQ_AGENT_CMDLINE_OPTS
else
   exec $RHQ_JAVA_EXE_FILE_PATH "$_JAVA_ENDORSED_DIRS_OPT" "$_JAVA_LIBRARY_PATH_OPT" $RHQ_AGENT_JAVA_OPTS $RHQ_AGENT_PREF_PROPERTIES $RHQ_AGENT_ADDITIONAL_JAVA_OPTS $_LOG_CONFIG -cp "$CLASSPATH" $RHQ_AGENT_MAINCLASS $RHQ_AGENT_CMDLINE_OPTS &
   RHQ_AGENT_BACKGROUND_PID=$!
   export RHQ_AGENT_BACKGROUND_PID
   if [ "$RHQ_AGENT_IN_BACKGROUND" != "nofile" ]; then
      echo $RHQ_AGENT_BACKGROUND_PID > "$RHQ_AGENT_IN_BACKGROUND"
   fi
fi

This was an option that Jay added that isn't even used today. You can take out the PREF_PROPERTIES thing and it won't affect anything. Putting a NEEDINFO for Jay to enter his thoughts.

Mazz is right, we never ended up using the option to store preferences in a file, so that mechanism is a no-op.  You can do anything you like to that RHQ_AGENT_PREF_PROPERTIES property to make things work.

I think the file store for prefs is still there, I don't know whether it works at this point, which means it probably doesn't.  It's not necessary that we provide a way to enable it going forward.

I do not know if file store for prefs works at this point either. I do know that customers are using this though. My impression is that it does work.

I think the appropriate resolution is a check to see if RHQ_AGENT_PREF_PROPERTIES is empty and if it is, exclude it from the command-line.

Also, I think eval was preventing this issue previously. If we can go back to properly evaluating the variables when building the command before we pass it to exec, this issue may go away and things would return to their pre-3.3.2 era.

Eval has its own issues and is not really a recommended way of launching stuff, especially these that spawn a new process (it generates a heap of other issues - JON 3.2 really did not work - it just wasn't tested correctly).

Fixed in the master (tested with dash & bash):

commit 7369beca36f8f3313aaedb2c26cc4b2df4e914f8
Author: Michael Burman <miburman>
Date:   Thu Jun 18 13:33:43 2015 +0300

    [BZ 1232006] Remove quotes around the RHQ_AGENT_PREF_PROPERTIES

For testing we need to make sure that RHQ_AGENT_PREF_PROPERTIES uses a value that includes a directory which contains spaces or other special characters.

Available for test with 3.3.3 ER01 build: 
https://brewweb.devel.redhat.com/buildinfo?buildID=446732
 *Note: jon-server-patch-3.3.0.GA.zip maps to ER01 build of
 jon-server-3.3.0.GA-update-03.zip.

Created attachment 1054051 [details]
agent.log

uncommented RHQ_AGENT_JAVA_OPTS and add -Dbz1232006 to make it easier to see the options in effect

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-1525.html