Bug 1232006

Summary: Agent does not start if RHQ_AGENT_JAVA_OPTS is set in rhq-agent-env.sh
Product: [JBoss] JBoss Operations Network Reporter: Marc Shirley <mshirley>
Component: AgentAssignee: Michael Burman <miburman>
Status: CLOSED ERRATA QA Contact: Filip Brychta <fbrychta>
Severity: high Docs Contact:
Priority: high    
Version: JON 3.3.2CC: fbrychta, jshaughn, loleary, miburman, spinder, vnguyen
Target Milestone: ER01Keywords: Regression, TestCaseNeeded, Triaged
Target Release: JON 3.3.3   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-07-30 16:43:40 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
Attachments:
Description Flags
agent.log none

Description Marc Shirley 2015-06-15 19:42:22 UTC
Description of problem:
If the RHQ_AGENT_JAVA_OPTS variable is uncommented/set in rhq-agent-env.sh, the agent fails to start up with "Error: Could not find or load main class".

Version-Release number of selected component (if applicable):
JBoss ON 3.3 update 2

How reproducible:
Always

Steps to Reproduce:
1. Update to 3.3 update 2
2. Edit rhq-agent-env.sh to uncomment RHQ_AGENT_JAVA_OPTS variable
3. ./rhq-agent.sh

Actual results:
Agent startup fails with "Error: Could not find or load main class"

Expected results:
Agent starts up successfully.

Additional info:

Comment 1 Marc Shirley 2015-06-15 20:27:04 UTC
rhq-agent-env.sh needs to have the RHQ_AGENT_JAVA_OPTS default value split into a RHQ_AGENT_JAVA_OPTS and a RHQ_AGENT_PREF_PROPERTIES value.  For example, the current default value in rhq-agent-env.sh:
#RHQ_AGENT_JAVA_OPTS="-Xms64m -Xmx128m -Djava.net.preferIPv4Stack=true -Drhq.preferences.file=${RHQ_AGENT_HOME}/conf/agent-prefs.properties"

should be changed to:
#RHQ_AGENT_JAVA_OPTS="-Xms64m -Xmx128m -Djava.net.preferIPv4Stack=true"
#RHQ_AGENT_PREF_PROPERTIES="-Drhq.preferences.file=${RHQ_AGENT_HOME}/conf/agent-prefs.properties"

This will allow the user to still have a template/example value to work with.

Comment 2 Marc Shirley 2015-06-15 20:34:58 UTC
Alternatively, removing the quotes from around $RHQ_AGENT_PREF_PROPERTIES in the exec commands in rhq-agent.sh also provides a fix to the error.

# Run the VM - put it in background if the caller wants it to be
if [ -z "$RHQ_AGENT_IN_BACKGROUND" ]; then
   exec $RHQ_JAVA_EXE_FILE_PATH "$_JAVA_ENDORSED_DIRS_OPT" "$_JAVA_LIBRARY_PATH_OPT" $RHQ_AGENT_JAVA_OPTS $RHQ_AGENT_PREF_PROPERTIES $RHQ_AGENT_ADDITIONAL_JAVA_OPTS $_LOG_CONFIG -cp "$CLASSPATH" $RHQ_AGENT_MAINCLASS $RHQ_AGENT_CMDLINE_OPTS
else
   exec $RHQ_JAVA_EXE_FILE_PATH "$_JAVA_ENDORSED_DIRS_OPT" "$_JAVA_LIBRARY_PATH_OPT" $RHQ_AGENT_JAVA_OPTS $RHQ_AGENT_PREF_PROPERTIES $RHQ_AGENT_ADDITIONAL_JAVA_OPTS $_LOG_CONFIG -cp "$CLASSPATH" $RHQ_AGENT_MAINCLASS $RHQ_AGENT_CMDLINE_OPTS &
   RHQ_AGENT_BACKGROUND_PID=$!
   export RHQ_AGENT_BACKGROUND_PID
   if [ "$RHQ_AGENT_IN_BACKGROUND" != "nofile" ]; then
      echo $RHQ_AGENT_BACKGROUND_PID > "$RHQ_AGENT_IN_BACKGROUND"
   fi
fi

Comment 4 John Mazzitelli 2015-06-16 01:15:51 UTC
This was an option that Jay added that isn't even used today. You can take out the PREF_PROPERTIES thing and it won't affect anything. Putting a NEEDINFO for Jay to enter his thoughts.

Comment 5 Jay Shaughnessy 2015-06-16 13:28:17 UTC
Mazz is right, we never ended up using the option to store preferences in a file, so that mechanism is a no-op.  You can do anything you like to that RHQ_AGENT_PREF_PROPERTIES property to make things work.

I think the file store for prefs is still there, I don't know whether it works at this point, which means it probably doesn't.  It's not necessary that we provide a way to enable it going forward.

Comment 6 Larry O'Leary 2015-06-16 23:41:41 UTC
I do not know if file store for prefs works at this point either. I do know that customers are using this though. My impression is that it does work.

I think the appropriate resolution is a check to see if RHQ_AGENT_PREF_PROPERTIES is empty and if it is, exclude it from the command-line.

Also, I think eval was preventing this issue previously. If we can go back to properly evaluating the variables when building the command before we pass it to exec, this issue may go away and things would return to their pre-3.3.2 era.

Comment 7 Michael Burman 2015-06-17 12:06:45 UTC
Eval has its own issues and is not really a recommended way of launching stuff, especially these that spawn a new process (it generates a heap of other issues - JON 3.2 really did not work - it just wasn't tested correctly).

Comment 8 Michael Burman 2015-06-18 10:34:59 UTC
Fixed in the master (tested with dash & bash):

commit 7369beca36f8f3313aaedb2c26cc4b2df4e914f8
Author: Michael Burman <miburman@redhat.com>
Date:   Thu Jun 18 13:33:43 2015 +0300

    [BZ 1232006] Remove quotes around the RHQ_AGENT_PREF_PROPERTIES

Comment 9 Larry O'Leary 2015-06-18 13:28:05 UTC
For testing we need to make sure that RHQ_AGENT_PREF_PROPERTIES uses a value that includes a directory which contains spaces or other special characters.

Comment 11 Simeon Pinder 2015-07-10 18:55:32 UTC
Available for test with 3.3.3 ER01 build: 
https://brewweb.devel.redhat.com/buildinfo?buildID=446732
 *Note: jon-server-patch-3.3.0.GA.zip maps to ER01 build of
 jon-server-3.3.0.GA-update-03.zip.

Comment 12 Viet Nguyen 2015-07-20 22:07:41 UTC
Created attachment 1054051 [details]
agent.log

uncommented RHQ_AGENT_JAVA_OPTS and add -Dbz1232006 to make it easier to see the options in effect

Comment 14 errata-xmlrpc 2015-07-30 16:43:40 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-1525.html