Bug 589611

Summary:	Test mixed JDK environment (Agent/EAP)
Product:	[Other] RHQ Project	Reporter:	Charles Crouch <ccrouch>
Component:	No Component	Assignee:	Lukas Krejci <lkrejci>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Corey Welton <cwelton>
Severity:	medium	Docs Contact:
Priority:	high
Version:	unspecified	CC:	hbrock
Target Milestone:	---	Keywords:	Reopened
Target Release:	---
Hardware:	All
OS:	Linux
Whiteboard:
Fixed In Version:	2.4	Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2010-08-12 16:52:00 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Charles Crouch 2010-05-06 14:47:58 UTC

General notes
-------------
-Use latest jdk5 and jdk6 bits
-When setting up each test case we need to make sure that the agent and EAP instance are inventoried correctly into JON, i.e. they have their JAVA_HOME connection properties show up correctly in the JON inventory tab
-Make sure the JVM child resources of EAP are in the inventory too
-After the test make sure all resources continue to be green and that metrics are being collected from the EAP and JVM resources.
-EAP5 requires jdk6, EAP4.3 should work with jdk5 or 6
-Expectation is that most customers using the operations features will run EAP and the agent as the same user, so we should do the same in our testing

Tests
-----
1) Agent running jdk6, EAP4.3 running jdk5 (stop and start)
a) Record (from /proc) the full command line and environment of both the agent and EAP instance. Make sure its clear from the process information which JDK is being used.
b) Use the Stop operation on the EAP instance 
c) Use the Start operation on the EAP instance 
d) Repeat step a) for the EAP instance, and compare results. Check for changes to environment settings and jdk being used.

Repeat the test procedures for the following scenarios
2) Agent running jdk5, EAP4.3 running jdk6 (stop and start) 
3) Agent running jdk5, EAP5.0 running jdk6 (stop and start)

Comment 2 Charles Crouch 2010-05-06 14:57:50 UTC

This issue is likely to be relevant: https://jira.jboss.org/jira/browse/JOPR-368

Comment 3 Corey Welton 2010-05-07 14:19:23 UTC

Above jira did not seem to affect things; notably I was unable to reproduce the classpath error when using standalone agent vs embedded.

Test Execution Results:

Agent: Java 6 ; EAP: 4.3 / Java 5
---------------------------------

[root@jiaozi bin]# cat /proc/18346/cmdline 
/opt/jdk1.6.0_20/bin/java-Djava.endorsed.dirs=/opt/rhq-agent/lib/endorsed-Djava.library.path=/opt/rhq-agent/lib-Xms64m-Xmx128m-Djava.net.preferIPv4Stack=true-Dlog4j.configuration=log4j.xml-cp/opt/rhq-agent/conf:/opt/rhq-agent/lib/commons-httpclient-2.0.2.jar:/opt/rhq-agent/lib/commons-logging-1.1.0.jboss.jar:/opt/rhq-agent/lib/concurrent-1.3.4-jboss-update1.jar:/opt/rhq-agent/lib/dom4j-1.6.jar:/opt/rhq-agent/lib/getopt-1.0.13.jar:/opt/rhq-agent/lib/i18nlog-1.0.9.jar:/opt/rhq-agent/lib/jboss-common-1.2.1.GA.jar:/opt/rhq-agent/lib/jboss-jmx-4.2.3.GA.jar:/opt/rhq-agent/lib/jboss-remoting-2.2.2.SP8.jar:/opt/rhq-agent/lib/jboss-serialization-1.0.3.GA.jar:/opt/rhq-agent/lib/log4j-1.2.14.jar:/opt/rhq-agent/lib/persistence-api-1.0.jar:/opt/rhq-agent/lib/rhq-core-client-api-1.3.1.GA.jar:/opt/rhq-agent/lib/rhq-core-comm-api-1.3.1.GA.jar:/opt/rhq-agent/lib/rhq-core-domain-1.3.1.GA.jar:/opt/rhq-agent/lib/rhq-core-native-system-1.3.1.GA.jar:/opt/rhq-agent/lib/rhq-core-plugin-api-1.3.1.GA.jar:/opt/rhq-agent/lib/rhq-core-plugin-container-1.3.1.GA.jar:/opt/rhq-agent/lib/rhq-core-util-1.3.1.GA.jar:/opt/rhq-agent/lib/rhq-enterprise-agent-1.3.1.GA.jar:/opt/rhq-agent/lib/rhq-enterprise-comm-1.3.1.GA.jar:/opt/rhq-agent/lib/sigar-1.6.3.82.jarorg.rhq.enterprise.agent.AgentMain--daemon

[root@jiaozi bin]# ls -alF /proc/18346/exe
lrwxrwxrwx. 1 root root 0 2010-05-07 10:03 /proc/18346/exe -> /opt/jdk1.6.0_20/bin/java*

[root@jiaozi bin]# cat /proc/18561/cmdline
/opt/jdk1.5.0_22//bin/java-Dprogram.name=run.sh-server-Xms128m-Xmx512m-Dsun.rmi.dgc.client.gcInterval=3600000-Dsun.rmi.dgc.server.gcInterval=3600000-Djava.net.preferIPv4Stack=true-Djava.endorsed.dirs=/root/jboss/jboss-eap-4.3/jboss-as/lib/endorsed-classpath/root/jboss/jboss-eap-4.3/jboss-as/bin/run.jar:/opt/jdk1.5.0_22//lib/tools.jarorg.jboss.Main-b0.0.0.0-cdefault

[root@jiaozi bin]# ls -alF /proc/18561/exe
lrwxrwxrwx. 1 root root 0 2010-05-07 10:06 /proc/18561/exe -> /opt/jdk1.5.0_22/bin/java*

[root@jiaozi bin]# cat /proc/18561/environ 
HOSTNAME=jiaozi.usersys.redhat.comSELINUX_ROLE_REQUESTED=SHELL=/bin/bashTERM=xtermHISTSIZE=1000SSH_CLIENT=10.11.9.65 42870 22SELINUX_USE_CURRENT_RANGE=QTDIR=/usr/lib64/qt-3.3JBOSS_HOME=/root/jboss/jboss-eap-4.3/jboss-asQTINC=/usr/lib64/qt-3.3/includeSSH_TTY=/dev/pts/2USER=rootLS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lz=01;31:*.xz=01;31:*.bz2=01;31:*.tbz=01;31:*.tbz2=01;31:*.bz=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.rar=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=01;36:*.au=01;36:*.flac=01;36:*.mid=01;36:*.midi=01;36:*.mka=01;36:*.mp3=01;36:*.mpc=01;36:*.ogg=01;36:*.ra=01;36:*.wav=01;36:*.axa=01;36:*.oga=01;36:*.spx=01;36:*.xspf=01;36:CCACHE_DIR=/var/cache/ccachePATH=/usr/lib64/qt-3.3/bin:/usr/kerberos/sbin:/usr/kerberos/bin:/usr/lib64/ccache:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/binMAIL=/var/spool/mail/rootPWD=/opt/rhq-agent/binJAVA_HOME=/opt/jdk1.5.0_22/CCACHE_UMASK=002LANG=en_US.UTF-8SELINUX_LEVEL_REQUESTED=HISTCONTROL=ignorebothSSH_ASKPASS=/usr/libexec/openssh/gnome-ssh-askpassHOME=/rootSHLVL=2LOGNAME=rootCVS_RSH=sshQTLIB=/usr/lib64/qt-3.3
/libSSH_CONNECTION=10.11.9.65 42870 10.11.231.107 22LESSOPEN=|/usr/bin/lesspipe.sh %sDISPLAY=localhost:10.0G_BROKEN_FILENAMES=1_=/opt/jdk1.5.0_22//bin/javaLD_LIBRARY_PATH=/opt/jdk1.5.0_22/jre/lib/amd64/server:/opt/jdk1.5.0_22/jre/lib/amd64:/opt/jdk1.5.0_22/jre/../lib/amd64

EAP Stop: SUCCESS
EAP Start: SUCCESS

[root@jiaozi bin]# cat /proc/18869/cmdline 
/opt/jdk1.5.0_22/bin/java-Dprogram.name=run.sh-server-Xms128m-Xmx512m-Dsun.rmi.dgc.client.gcInterval=3600000-Dsun.rmi.dgc.server.gcInterval=3600000-Djava.net.preferIPv4Stack=true-Djava.endorsed.dirs=/root/jboss/jboss-eap-4.3/jboss-as/lib/endorsed-classpath/root/jboss/jboss-eap-4.3/jboss-as/bin/run.jar:/opt/jdk1.5.0_22/lib/tools.jarorg.jboss.Main-cdefault-b0.0.0.0

[root@jiaozi bin]# ls -alF /proc/18869/exe
lrwxrwxrwx. 1 root root 0 2010-05-07 10:11 /proc/18869/exe -> /opt/jdk1.5.0_22/bin/java*

[root@jiaozi bin]# cat /proc/18869/environ 
HOSTNAME=jiaozi.usersys.redhat.comSELINUX_ROLE_REQUESTED=SHELL=/bin/bashTERM=xtermHISTSIZE=1000RHQ_AGENT_CMDLINE_OPTS=--daemonSSH_CLIENT=10.11.9.65 42870 22SELINUX_USE_CURRENT_RANGE=QTDIR=/usr/lib64/qt-3.3JBOSS_HOME=/root/jboss/jboss-eap-4.3/jboss-asQTINC=/usr/lib64/qt-3.3/includeSSH_TTY=/dev/pts/2USER=rootLS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lz=01;31:*.xz=01;31:*.bz2=01;31:*.tbz=01;31:*.tbz2=01;31:*.bz=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.rar=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=01;36:*.au=01;36:*.flac=01;36:*.mid=01;36:*.midi=01;36:*.mka=01;36:*.mp3=01;36:*.mpc=01;36:*.ogg=01;36:*.ra=01;36:*.wav=01;36:*.axa=01;36:*.oga=01;36:*.spx=01;36:*.xspf=01;36:LD_LIBRARY_PATH=/opt/jdk1.5.0_22/jre/lib/amd64/server:/opt/jdk1.5.0_22/jre/lib/amd64:/opt/jdk1.5.0_22/jre/../lib/amd64:/opt/jdk1.6.0_20/jre/lib/amd64/server:/opt/jdk1.6.0_20/jre/lib/amd64:/opt/jdk1.6.0_20/jre/../lib/amd64CCACHE_DIR=/var/cache/ccacheNLSPATH=/usr/dt/lib/nls/msg/%L/%N.catPATH=/usr/lib64/qt-3.3/bin:/usr/kerberos/sbin:/usr/kerberos/bin:/usr/lib64/ccache:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr
/sbin:/usr/bin:/root/binMAIL=/var/spool/mail/rootPWD=/root/jboss/jboss-eap-4.3/jboss-as/binJAVA_HOME=/opt/jdk1.5.0_22CCACHE_UMASK=002LANG=en_US.UTF-8XFILESEARCHPATH=/usr/dt/app-defaults/%L/DtSELINUX_LEVEL_REQUESTED=SSH_ASKPASS=/usr/libexec/openssh/gnome-ssh-askpassHISTCONTROL=ignorebothRHQ_AGENT_IN_BACKGROUND=/opt/rhq-agent/bin/rhq-agent.pidHOME=/rootSHLVL=4LOGNAME=rootQTLIB=/usr/lib64/qt-3.3/libCVS_RSH=sshSSH_CONNECTION=10.11.9.65 42870 10.11.231.107 22LESSOPEN=|/usr/bin/lesspipe.sh %sDISPLAY=localhost:10.0G_BROKEN_FILENAMES=1_=/opt/jdk1.5.0_22/bin/java


Agent: Java 5 ; EAP: 4.3 / Java 6
---------------------------------

[root@jiaozi bin]# cat /proc/3745/cmdline 
/opt/jdk1.5.0_22/bin/java-Djava.endorsed.dirs=/opt/rhq-agent/lib/endorsed-Djava.library.path=/opt/rhq-agent/lib-Xms64m-Xmx128m-Djava.net.preferIPv4Stack=true-Dlog4j.configuration=log4j.xml-cp/opt/rhq-agent/conf:/opt/rhq-agent/lib/commons-httpclient-2.0.2.jar:/opt/rhq-agent/lib/commons-logging-1.1.0.jboss.jar:/opt/rhq-agent/lib/concurrent-1.3.4-jboss-update1.jar:/opt/rhq-agent/lib/dom4j-1.6.jar:/opt/rhq-agent/lib/getopt-1.0.13.jar:/opt/rhq-agent/lib/i18nlog-1.0.9.jar:/opt/rhq-agent/lib/jboss-common-1.2.1.GA.jar:/opt/rhq-agent/lib/jboss-jmx-4.2.3.GA.jar:/opt/rhq-agent/lib/jboss-remoting-2.2.2.SP8.jar:/opt/rhq-agent/lib/jboss-serialization-1.0.3.GA.jar:/opt/rhq-agent/lib/log4j-1.2.14.jar:/opt/rhq-agent/lib/persistence-api-1.0.jar:/opt/rhq-agent/lib/rhq-core-client-api-1.3.1.GA.jar:/opt/rhq-agent/lib/rhq-core-comm-api-1.3.1.GA.jar:/opt/rhq-agent/lib/rhq-core-domain-1.3.1.GA.jar:/opt/rhq-agent/lib/rhq-core-native-system-1.3.1.GA.jar:/opt/rhq-agent/lib/rhq-core-plugin-api-1.3.1.GA.jar:/opt/rhq-agent/lib/rhq-core-plugin-container-1.3.1.GA.jar:/opt/rhq-agent/lib/rhq-core-util-1.3.1.GA.jar:/opt/rhq-agent/lib/rhq-enterprise-agent-1.3.1.GA.jar:/opt/rhq-agent/lib/rhq-enterprise-comm-1.3.1.GA.jar:/opt/rhq-agent/lib/sigar-1.6.3.82.jarorg.rhq.enterprise.agent.AgentMain--daemon

[root@jiaozi bin]# ls -alF /proc/3745/exe
lrwxrwxrwx. 1 root root 0 2010-05-06 13:06 /proc/3745/exe -> /opt/jdk1.5.0_22/bin/java*

[root@jiaozi bin]# cat /proc/5402/cmdline
/opt/jdk1.6.0_20//bin/java-Dprogram.name=run.sh-server-Xms128m-Xmx512m-Dsun.rmi.dgc.client.gcInterval=3600000-Dsun.rmi.dgc.server.gcInterval=3600000-Djava.net.preferIPv4Stack=true-Djava.endorsed.dirs=/root/jboss/jboss-eap-4.3/jboss-as/lib/endorsed-classpath/root/jboss/jboss-eap-4.3/jboss-as/bin/run.jar:/opt/jdk1.6.0_20//lib/tools.jarorg.jboss.Main-b0.0.0.0-cdefault1

[root@jiaozi bin]# ls -alF /proc/5402/exe
lrwxrwxrwx. 1 root root 0 2010-05-06 13:26 /proc/5402/exe -> /opt/jdk1.6.0_20/bin/java*

EAP Stop: SUCCESS
EAP Start: SUCCESS

[root@jiaozi bin]# cat /proc/5659/cmdline
/opt/jdk1.6.0_20/bin/java-Dprogram.name=run.sh-server-Xms128m-Xmx512m-Dsun.rmi.dgc.client.gcInterval=3600000-Dsun.rmi.dgc.server.gcInterval=3600000-Djava.net.preferIPv4Stack=true-Djava.endorsed.dirs=/root/jboss/jboss-eap-4.3/jboss-as/lib/endorsed-classpath/root/jboss/jboss-eap-4.3/jboss-as/bin/run.jar:/opt/jdk1.6.0_20/lib/tools.jarorg.jboss.Main-cdefault1-b0.0.0.0

[root@jiaozi bin]# ls -alF /proc/5659/exe 
lrwxrwxrwx. 1 root root 0 2010-05-06 13:31 /proc/5659/exe -> /opt/jdk1.6.0_20/bin/java*


After restarting again (so different procid)

[root@jiaozi jboss-as]# cat /proc/8695/environ 
HOSTNAME=jiaozi.usersys.redhat.comSELINUX_ROLE_REQUESTED=TERM=xtermSHELL=/bin/bashRHQ_AGENT_CMDLINE_OPTS=--daemonHISTSIZE=1000SSH_CLIENT=10.11.231.111 56942 22SELINUX_USE_CURRENT_RANGE=QTDIR=/usr/lib64/qt-3.3JBOSS_HOME=/root/jboss/jboss-eap-4.3/jboss-asQTINC=/usr/lib64/qt-3.3/includeSSH_TTY=/dev/pts/1USER=rootLS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lz=01;31:*.xz=01;31:*.bz2=01;31:*.tbz=01;31:*.tbz2=01;31:*.bz=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.rar=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=01;36:*.au=01;36:*.flac=01;36:*.mid=01;36:*.midi=01;36:*.mka=01;36:*.mp3=01;36:*.mpc=01;36:*.ogg=01;36:*.ra=01;36:*.wav=01;36:*.axa=01;36:*.oga=01;36:*.spx=01;36:*.xspf=01;36:LD_LIBRARY_PATH=/opt/jdk1.6.0_20/jre/lib/amd64/server:/opt/jdk1.6.0_20/jre/lib/amd64:/opt/jdk1.6.0_20/jre/../lib/amd64:/opt/jdk1.5.0_22/jre/lib/amd64/server:/opt/jdk1.5.0_22/jre/lib/amd64:/opt/jdk1.5.0_22/jre/../lib/amd64CCACHE_DIR=/var/cache/ccacheNLSPATH=/usr/dt/lib/nls/msg/%L/%N.catPATH=/usr/lib64/qt-3.3/bin:/usr/kerberos/sbin:/usr/kerberos/bin:/usr/lib64/ccache:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/
usr/sbin:/usr/bin:/root/binMAIL=/var/spool/mail/rootPWD=/root/jboss/jboss-eap-4.3/jboss-as/binJAVA_HOME=/opt/jdk1.6.0_20XMODIFIERS=@im=ibusCCACHE_UMASK=002LANG=en_US.UTF-8XFILESEARCHPATH=/usr/dt/app-defaults/%L/DtSELINUX_LEVEL_REQUESTED=HISTCONTROL=ignorebothSSH_ASKPASS=/usr/libexec/openssh/gnome-ssh-askpassRHQ_AGENT_IN_BACKGROUND=/opt/rhq-agent/bin/rhq-agent.pidSHLVL=4HOME=/rootLOGNAME=rootQTLIB=/usr/lib64/qt-3.3/libCVS_RSH=sshSSH_CONNECTION=10.11.231.111 56942 10.11.231.107 22LESSOPEN=|/usr/bin/lesspipe.sh %sG_BROKEN_FILENAMES=1_=/opt/jdk1.6.0_20/bin/java

Agent: Java 5 ; EAP: 5.0 / Java 6
---------------------------------
[same agent as previous test]

[root@jiaozi jboss-as]# cat /proc/6278/cmdline 
/opt/jdk1.6.0_20//bin/java-Dprogram.name=run.sh-server-Xms1303m-Xmx1303m-XX:MaxPermSize=256m-Dorg.jboss.resolver.warning=true-Dsun.rmi.dgc.client.gcInterval=3600000-Dsun.rmi.dgc.server.gcInterval=3600000-Dsun.lang.ClassLoader.allowArraySyntax=true-Djava.net.preferIPv4Stack=true-Djava.endorsed.dirs=/root/jboss/jboss-eap-5.0/jboss-as/lib/endorsed-classpath/root/jboss/jboss-eap-5.0/jboss-as/bin/run.jar:/opt/jdk1.6.0_20//lib/tools.jarorg.jboss.Main-b0.0.0.0-cdefault

[root@jiaozi jboss-as]# ls -alF /proc/6278/exe
lrwxrwxrwx. 1 root root 0 2010-05-06 13:39 /proc/6278/exe -> /opt/jdk1.6.0_20/bin/java*

EAP Stop: SUCCESS
EAP Start: SUCCESS

[root@jiaozi jboss-as]# cat /proc/6578/cmdline
/opt/jdk1.6.0_20/bin/java-Dprogram.name=run.sh-server-Xms1303m-Xmx1303m-XX:MaxPermSize=256m-Dorg.jboss.resolver.warning=true-Dsun.rmi.dgc.client.gcInterval=3600000-Dsun.rmi.dgc.server.gcInterval=3600000-Dsun.lang.ClassLoader.allowArraySyntax=true-Djava.net.preferIPv4Stack=true-Djava.endorsed.dirs=/root/jboss/jboss-eap-5.0/jboss-as/lib/endorsed-classpath/root/jboss/jboss-eap-5.0/jboss-as/bin/run.jar:/opt/jdk1.6.0_20/lib/tools.jarorg.jboss.Main-cdefault-b0.0.0.0

[root@jiaozi jboss-as]# ls -alF /proc/6578/exe
lrwxrwxrwx. 1 root root 0 2010-05-06 13:44 /proc/6578/exe -> /opt/jdk1.6.0_20/bin/java*

HOSTNAME=jiaozi.usersys.redhat.comSELINUX_ROLE_REQUESTED=TERM=xtermSHELL=/bin/bashRHQ_AGENT_CMDLINE_OPTS=--daemonHISTSIZE=1000SSH_CLIENT=10.11.231.111 56942 22SELINUX_USE_CURRENT_RANGE=QTDIR=/usr/lib64/qt-3.3JBOSS_HOME=/root/jboss/jboss-eap-5.0/jboss-asQTINC=/usr/lib64/qt-3.3/includeSSH_TTY=/dev/pts/1USER=rootLS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lz=01;31:*.xz=01;31:*.bz2=01;31:*.tbz=01;31:*.tbz2=01;31:*.bz=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.rar=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=01;36:*.au=01;36:*.flac=01;36:*.mid=01;36:*.midi=01;36:*.mka=01;36:*.mp3=01;36:*.mpc=01;36:*.ogg=01;36:*.ra=01;36:*.wav=01;36:*.axa=01;36:*.oga=01;36:*.spx=01;36:*.xspf=01;36:LD_LIBRARY_PATH=/opt/jdk1.6.0_20/jre/lib/amd64/server:/opt/jdk1.6.0_20/jre/lib/amd64:/opt/jdk1.6.0_20/jre/../lib/amd64:/opt/jdk1.5.0_22/jre/lib/amd64/server:/opt/jdk1.5.0_22/jre/lib/amd64:/opt/jdk1.5.0_22/jre/../lib/amd64CCACHE_DIR=/var/cache/ccacheNLSPATH=/usr/dt/lib/nls/msg/%L/%N.catPATH=/usr/lib64/qt-3.3/bin:/usr/kerberos/sbin:/usr/kerberos/bin:/usr/lib64/ccache:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/
usr/sbin:/usr/bin:/root/binMAIL=/var/spool/mail/rootPWD=/root/jboss/jboss-eap-5.0/jboss-as/binJAVA_HOME=/opt/jdk1.6.0_20XMODIFIERS=@im=ibusCCACHE_UMASK=002LANG=en_US.UTF-8XFILESEARCHPATH=/usr/dt/app-defaults/%L/DtSELINUX_LEVEL_REQUESTED=HISTCONTROL=ignorebothSSH_ASKPASS=/usr/libexec/openssh/gnome-ssh-askpassRHQ_AGENT_IN_BACKGROUND=/opt/rhq-agent/bin/rhq-agent.pidSHLVL=4HOME=/rootLOGNAME=rootQTLIB=/usr/lib64/qt-3.3/libCVS_RSH=sshSSH_CONNECTION=10.11.231.111 56942 10.11.231.107 22LESSOPEN=|/usr/bin/lesspipe.sh %sG_BROKEN_FILENAMES=1_=/opt/jdk1.6.0_20/bin/java

Comment 4 Corey Welton 2010-05-07 14:21:55 UTC

I think we can close out this bz; it can be reopened if we find problems in the test results, or in the methodology.

Comment 7 Charles Crouch 2010-05-18 03:41:14 UTC

Re-opening for Lukas to have another look at any env. related issues

Comment 8 Lukas Krejci 2010-05-20 11:12:50 UTC

I've tested most the combinations of the following:

agent in java 5 or 6, jboss eap 4.3.0 or jboss eap 5.0.1 with java 5 or 6 with JAVA_HOME set either on commandline or via an env variable.

I first started the AS manually and let the agent discover it, captured the cmdline and environment of the AS and then restarted it using RHQ UI and captured the cmdline and environment again.

And the results highlight 2 potential problems:

we change the commandline args used to start the AS:

AS4:
    "-Djboss.platform.mbeanserver" omitted in the restarted cmdline
    "-b 0.0.0.0" added to the cmdline
AS5:
    "-Djboss.platform.mbeanserver" omitted in the restarted cmdline
    "-b 127.0.0.1" added to the cmdline

This is of course a problem because when the server is restarted using the agent, its JVM child resource becomes permanently unavailable. There is an easy workaround for this - put the required commandline arguments in run.conf of the server or configure the AS resource to use a custom startup script.

The additional commandline arguments are used correctly to ensure that the AS server starts up with the same configuration and binding as it had before.

the environment changes:
-- the AS server is started using the agent's environment variables. The agent only updates the JAVA_HOME variable using the value configured in the AS server's resource configuration.
In particular, I can see a potential problem with LD_LIBRARY_PATH. The AS startup script prepends the values it needs to it, so that might not be a problem (but could if for some reason the admin required different LD_LIBRARY_PATHs for the agent and AS for loading different versions of some native libs).

I.e. in my case, the AS run.sh added the following to the LD_LIBRARY_PATH when started manually:
LD_LIBRARY_PATH=/home/metlos/Progs/jdk1.6.0_20/jre/lib/i386/server:/home/metlos/Progs/jdk1.6.0_20/jre/lib/i386:/home/metlos/Progs/jdk1.6.0_20/jre/../lib/i386

but when the server was restarted from the agent, LD_LIBRARY_PATH looked like:
LD_LIBRARY_PATH=/home/metlos/Progs/jdk1.6.0_20/jre/lib/i386/server:/home/metlos/Progs/jdk1.6.0_20/jre/lib/i386:/home/metlos/Progs/jdk1.6.0_20/jre/../lib/i386:/home/metlos/Progs/jdk1.5.0_22/jre/lib/i386/server:/home/metlos/Progs/jdk1.5.0_22/jre/lib/i386:/home/metlos/Progs/jdk1.5.0_22/jre/../lib/i386:/home/metlos/Projects/build/lib/:

IMHO, none of these is a serious problem. The command line args have an easy workaround and the environment shouldn't cause any problems either. It is supposed that the agent and the AS are run as the same user and thus share the same default environment. Because LD_LIBRARY_PATH is modified during both agent and AS server startup and the new values are always prepended to the existing ones, there shouldn't be any regressions apart from the theoretical possibility outlined above (and again using custom startup scripts and prepending, user can always achieve desired behaviour).

Comment 9 Corey Welton 2010-06-04 13:30:26 UTC

Considering this one QE verified.

Comment 10 Corey Welton 2010-08-12 16:52:00 UTC

Mass-closure of verified bugs against JON.