Bug 604444
Summary: | Tomcat Operation for Shutdown is failing. Could have to do with JRE/JDK assumptions. | ||||||
---|---|---|---|---|---|---|---|
Product: | [Other] RHQ Project | Reporter: | John Sefler <jsefler> | ||||
Component: | Operations | Assignee: | Jay Shaughnessy <jshaughn> | ||||
Status: | CLOSED CURRENTRELEASE | QA Contact: | Corey Welton <cwelton> | ||||
Severity: | high | Docs Contact: | |||||
Priority: | low | ||||||
Version: | 3.0.0 | CC: | sdharane | ||||
Target Milestone: | --- | ||||||
Target Release: | --- | ||||||
Hardware: | All | ||||||
OS: | Linux | ||||||
Whiteboard: | |||||||
Fixed In Version: | 2.4 | Doc Type: | Bug Fix | ||||
Doc Text: | Story Points: | --- | |||||
Clone Of: | Environment: | ||||||
Last Closed: | 2010-08-12 16:45:09 UTC | Type: | --- | ||||
Regression: | --- | Mount Type: | --- | ||||
Documentation: | --- | CRM: | |||||
Verified Versions: | Category: | --- | |||||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
Cloudforms Team: | --- | Target Upstream Version: | |||||
Embargoed: | |||||||
Bug Depends On: | |||||||
Bug Blocks: | 562434, 577051 | ||||||
Attachments: |
|
Description
John Sefler
2010-06-15 21:59:24 UTC
Thinking about this some more, my guess is that the rhq agent is setting its own JAVA_HOME within the spawned shell that it uses to then launch tomcat's shutdown.sh script. Guessing that this JAVA_HOME is different than the JAVA_HOME that tomcat is using, I think that it messes with the environment variables that are set/used in catalina.sh and the result is that catalina then cannot find java. Or something of this sort. When I manually run the tomcat6/bin scripts (startup.sh, shutdown.sh, version.sh, catalina,sh) from the command line with export JAVA_HOME=/usr/lib/jvm/jre-openjdk/ on RHEL5 all is well. For example: [root@auto-rhq01 bin]# ./catalina.sh version Using CATALINA_BASE: /root/JBossEWS100/jboss-ews-1.0/tomcat5 Using CATALINA_HOME: /root/JBossEWS100/jboss-ews-1.0/tomcat5 Using CATALINA_TMPDIR: /root/JBossEWS100/jboss-ews-1.0/tomcat5/temp Using JRE_HOME: /usr/lib/jvm/jre-openjdk/ Server version: Apache Tomcat/5.5.23 Server built: Mar 25 2009 03:58:01 Server number: 5.5.23.0 OS Name: Linux OS Version: 2.6.18-194.el5 Architecture: amd64 JVM Version: 1.6.0_0-b16 JVM Vendor: Sun Microsystems Inc. [root@auto-rhq01 bin]# env | grep JAVA JAVA_HOME=/usr/lib/jvm/jre-openjdk/ Yet when I run the Shutdown (or Restart) operations from within RHQ, the agent.log reports the following warning and never completes the operation: 2010-06-16 10:19:34,914 WARN [ResourceDiscoveryComponent.invoker.daemon-68] (jboss.on.plugins.tomcat.TomcatDiscoveryComponent)- Failed to determine Tomcat Server Version Given: VersionInfo:/root/JBossEWS100/jboss-ews-1.0/tomcat6/bin/catalina.sh: line 362: /usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/bin/java: No such file or directory catalinaHome: /root/JBossEWS100/jboss-ews-1.0/tomcat6 Script:/root/JBossEWS100/jboss-ews-1.0/tomcat6/bin/version.sh timeout=10000 Notice the path it is using to launch java, (/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/bin/java), this path does not actually exist on the file system, however if you insert a jre into the path as follows, the file does exist. [root@auto-rhq01 bin]# ls -l /usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/bin/java -rwxr-xr-x 1 root root 42232 Mar 30 17:06 /usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/bin/java The failure to run catalina.sh from the rhq-agent affects other areas too: 1. See the attached screenshot. 2. Response time measurement collection also fails as noted by the following WARNing in the agent.log // 2010-06-16 10:22:00,928 WARN [MeasurementManager.collector-1] (rhq.core.pc.measurement.MeasurementCollectorRunner)- Failure to collect measurement data for Resource[id=10883, type=Tomcat Server, key=/root/JBossEWS100/jboss-ews-1.0/tomcat6, name=Tomcat (8180), parent=auto-rhq01.usersys.redhat.com, version=Unknown Version] - cause: org.rhq.core.pc.inventory.TimeoutException:Call to [org.jboss.on.plugins.tomcat.TomcatServerComponent.getValues()] with args [[org.rhq.core.domain.measurement.MeasurementReport@22bf991d, [ScheduledMeasurementInfo[res=10883, name=Catalina:type=Server:serverInfo, sched=19141]]]] timed out. Invocation thread will be interrupted Created attachment 424506 [details]
Unknown version results when agent fails to run catalina.sh
*** Bug 598610 has been marked as a duplicate of this bug. *** fix commit: 0225e8ee6395a730295e1af1fc903cd27d09b93e This problem affected any script execution in the tomcat plugin. Most notably server version detection and start and shutdown operations. In short the problem resulted from the fact that the RHQ Agent supports JAVA_HOME being set to a JRE. Tomcat does not and instead requires JRE_HOME to be set in that situation. Also, the RHQ Agent's environment initializes the process execution env. So, the TC plugin had to ensure not to forward on the Agent's setting for JAVA_HOME, and needed to set JRE_HOME and JAVA_HOME as TC would expect. As a note, Tomcat requires only a JRE starting with 5.5. Unless TC is being run in debug, in which case a JDK is required. I think this fix solves some fairly mysterious failure seen in the past. Good find. I tested this with jon-server-2.4.0.GA_QA build and was able to shutdown apache-tomcat. Marking this bug as verified. I am moving this back to ON_QA -- this test wasn't as cut-and-dried as it appears. Doing so, if only to assure we have the correct, very specific environment in place to assure we've got the proper scenario covered. Verified: RHQ version: 3.0.0-SNAPSHOT build number: b9ca90d $ git rev-list b9ca90d | grep 0225e8ee6395a730295e1af1fc903cd27d09b93e 0225e8ee6395a730295e1af1fc903cd27d09b93e The grep match indicates that this RHQ build includes the fix from Jay. Note: verification was done on the same environment where the original problems were discovered. * The Start/Stop/Restart operations are successfully completing on inventoried Tomcat5 and Tomcat6 servers. * The Tomcat5 and Tomcat6 Version is successfully captured by RHQ Could not yet verify in a JON build. Awaiting a newer build with fix included. Verified: JBoss Operations Network version: 2.4.0.GA_QA build number: 10745:647a602 Mass-closure of verified bugs against JON. |