Bug 1129290

Summary: Able to install agent more than one time on different location for the same user on same machine
Product: [JBoss] JBoss Operations Network Reporter: Jeeva Kandasamy <jkandasa>
Component: UIAssignee: John Mazzitelli <mazz>
Status: CLOSED CURRENTRELEASE QA Contact: Jeeva Kandasamy <jkandasa>
Severity: high Docs Contact:
Priority: unspecified    
Version: JON 3.3.0CC: hrupp, jshaughn, mazz, mfoley
Target Milestone: ER03   
Target Release: JON 3.3.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 1090829 Environment:
Last Closed: 2014-12-11 13:59:28 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
Bug Depends On: 1090829    
Bug Blocks: 1070242    
Attachments:
Description Flags
Agent-port-in-use-msg none

Description Jeeva Kandasamy 2014-08-12 14:07:39 UTC
+++ This bug was initially created as a clone of Bug #1090829 +++

Description of problem:
Try to install remote agent more than one time on same machine with different locations with same user. First installation went success and agent up and running. Second time installed on different location and up, but throws an exception saying the port already in use. 

----------------snap--agent.log----------------
2014-04-24 14:29:52,512 ERROR [main] (org.rhq.enterprise.agent.AgentMain)- {AgentMain.start-failure}Failed to start the agent. Cause: java.io.IOException: SocketServerInvoker[10.70.35.78:16163] error creating ServerSocket[10.70.35.78:16163]: Address already in use
----------------snap---------------------------

ps -ef | grep rhq
---------------------------------------------------
[jenkins@rhel-6-gui tmp]$ ps -ef | grep rhq
jenkins  16462     1  0 Apr23 ?        00:01:12 /usr/bin/java -Djava.endorsed.dirs=/tmp/rhq-agent/lib/endorsed -Djava.library.path=/tmp/rhq-agent/lib -Djna.platform.library.path=/tmp/rhq-agent/lib/augeas/lib64 -Xms64m -Xmx128m -Djava.net.preferIPv4Stack=true -Drhq.preferences.file=/tmp/rhq-agent/conf/agent-prefs.properties -Dlog4j.configuration=log4j.xml -cp /tmp/rhq-agent/conf:/tmp/rhq-agent/lib/commons-io-1.4.jar:/tmp/rhq-agent/lib/commons-logging-1.1.0.jboss.jar:/tmp/rhq-agent/lib/concurrent-1.3.4-jboss-update1.jar:/tmp/rhq-agent/lib/getopt-1.0.13.jar:/tmp/rhq-agent/lib/i18nlog-1.0.10.jar:/tmp/rhq-agent/lib/jboss-common-core-2.2.17.GA.jar:/tmp/rhq-agent/lib/jboss-jmx-4.2.3.GA.jar:/tmp/rhq-agent/lib/jboss-logging-3.1.2.GA.jar:/tmp/rhq-agent/lib/jboss-remoting-2.5.4.SP5.jar:/tmp/rhq-agent/lib/jboss-serialization-1.0.3.GA.jar:/tmp/rhq-agent/lib/jline-0.9.94.jar:/tmp/rhq-agent/lib/log4j-1.2.16.jar:/tmp/rhq-agent/lib/persistence-api-1.0.jar:/tmp/rhq-agent/lib/rhq-common-drift-4.11.0-SNAPSHOT.jar:/tmp/rhq-agent/lib/rhq-core-client-api-4.11.0-SNAPSHOT.jar:/tmp/rhq-agent/lib/rhq-core-comm-api-4.11.0-SNAPSHOT.jar:/tmp/rhq-agent/lib/rhq-core-domain-4.11.0-SNAPSHOT.jar:/tmp/rhq-agent/lib/rhq-core-native-system-4.11.0-SNAPSHOT.jar:/tmp/rhq-agent/lib/rhq-core-plugin-api-4.11.0-SNAPSHOT.jar:/tmp/rhq-agent/lib/rhq-core-plugin-container-4.11.0-SNAPSHOT.jar:/tmp/rhq-agent/lib/rhq-core-util-4.11.0-SNAPSHOT.jar:/tmp/rhq-agent/lib/rhq-enterprise-agent-4.11.0-SNAPSHOT.jar:/tmp/rhq-agent/lib/rhq-enterprise-comm-4.11.0-SNAPSHOT.jar:/tmp/rhq-agent/lib/sigar-1.6.5.132-6.jar:/tmp/rhq-agent/lib/trove4j-3.0.3.jar org.rhq.enterprise.agent.AgentMain -Drhq.agent.server.bind-address=rhqbox1.lab.eng.pnq.redhat.com -Drhq.communications.connector.bind-address=10.70.35.78 -Drhq.agent.configuration-setup-flag=true --daemon -Drhq.install.id=10012
jenkins  16899     1  0 Apr23 ?        00:00:54 /usr/bin/java -Djava.endorsed.dirs=/tmp/tmp/rhq-agent/lib/endorsed -Djava.library.path=/tmp/tmp/rhq-agent/lib -Djna.platform.library.path=/tmp/tmp/rhq-agent/lib/augeas/lib64 -Xms64m -Xmx128m -Djava.net.preferIPv4Stack=true -Drhq.preferences.file=/tmp/tmp/rhq-agent/conf/agent-prefs.properties -Dlog4j.configuration=log4j.xml -cp /tmp/tmp/rhq-agent/conf:/tmp/tmp/rhq-agent/lib/commons-io-1.4.jar:/tmp/tmp/rhq-agent/lib/commons-logging-1.1.0.jboss.jar:/tmp/tmp/rhq-agent/lib/concurrent-1.3.4-jboss-update1.jar:/tmp/tmp/rhq-agent/lib/getopt-1.0.13.jar:/tmp/tmp/rhq-agent/lib/i18nlog-1.0.10.jar:/tmp/tmp/rhq-agent/lib/jboss-common-core-2.2.17.GA.jar:/tmp/tmp/rhq-agent/lib/jboss-jmx-4.2.3.GA.jar:/tmp/tmp/rhq-agent/lib/jboss-logging-3.1.2.GA.jar:/tmp/tmp/rhq-agent/lib/jboss-remoting-2.5.4.SP5.jar:/tmp/tmp/rhq-agent/lib/jboss-serialization-1.0.3.GA.jar:/tmp/tmp/rhq-agent/lib/jline-0.9.94.jar:/tmp/tmp/rhq-agent/lib/log4j-1.2.16.jar:/tmp/tmp/rhq-agent/lib/persistence-api-1.0.jar:/tmp/tmp/rhq-agent/lib/rhq-common-drift-4.11.0-SNAPSHOT.jar:/tmp/tmp/rhq-agent/lib/rhq-core-client-api-4.11.0-SNAPSHOT.jar:/tmp/tmp/rhq-agent/lib/rhq-core-comm-api-4.11.0-SNAPSHOT.jar:/tmp/tmp/rhq-agent/lib/rhq-core-domain-4.11.0-SNAPSHOT.jar:/tmp/tmp/rhq-agent/lib/rhq-core-native-system-4.11.0-SNAPSHOT.jar:/tmp/tmp/rhq-agent/lib/rhq-core-plugin-api-4.11.0-SNAPSHOT.jar:/tmp/tmp/rhq-agent/lib/rhq-core-plugin-container-4.11.0-SNAPSHOT.jar:/tmp/tmp/rhq-agent/lib/rhq-core-util-4.11.0-SNAPSHOT.jar:/tmp/tmp/rhq-agent/lib/rhq-enterprise-agent-4.11.0-SNAPSHOT.jar:/tmp/tmp/rhq-agent/lib/rhq-enterprise-comm-4.11.0-SNAPSHOT.jar:/tmp/tmp/rhq-agent/lib/sigar-1.6.5.132-6.jar:/tmp/tmp/rhq-agent/lib/trove4j-3.0.3.jar org.rhq.enterprise.agent.AgentMain -Drhq.agent.server.bind-address=rhqbox1.lab.eng.pnq.redhat.com -Drhq.communications.connector.bind-address=10.70.35.78 -Drhq.agent.configuration-setup-flag=true --daemon -Drhq.install.id=10031
jenkins  22597 22206  0 14:32 pts/4    00:00:00 grep rhq
---------------------------------------------------


Version-Release number of selected component (if applicable):
Version : 4.11.0-SNAPSHOT
Build Number : 1bc9904
GWT Version : 2.5.0
SmartGWT Version : 3.0

Browser: Firefox 28

How reproducible:
always

Steps to Reproduce:
1. Navigate to "Administration-->Agents-->New"
2. Enter remote box details (Agent should not be installed), give some location, example: /tmp/
3. Click on "install Agent", will be installed successfully
4. Again Navigate to "Administration-->Agents-->New"
5. Enter remote box details same as in step #2, give different location location, example: /tmp/another/
6. Click on "install Agent", will be installed successfully. But the port already use with agent installed on step #3.
7. Second installation should refer first installation location and should not proceed more than one time.

Actual results:
Permits second installation on different location for the same user on the same machine.

Expected results:
Should not install more than one time, if a agent is already installed on different location. We should track with some reference in common location.

--- Additional comment from Jeeva Kandasamy on 2014-04-24 11:06:21 EDT ---

in the above case, If we install multiple times, we can see more than one entry in "rhq_agent_install" table,

rhq=# select * from rhq_agent_install;
  id   |     mtime     |           agent_name           |                 install_location                 |   ssh_host   | ssh_port | ssh_username |   ssh_password   
-------+---------------+--------------------------------+--------------------------------------------------+--------------+----------+--------------+------------------
 10001 | 1398210095484 | rhqbox1.lab.eng.pnq.redhat.com | /home/hudson/rhq-server-location-fresh/rhq-agent | 10.65.206.42 |          |              | 
 10011 | 1398229679011 |                                |                                                  | 10.70.35.78  |       22 |              | 
 10013 | 1398308668469 |                                |                                                  | 10.70.35.78  |       22 | jenkins      | 606f83b302d3426d
 10012 | 1398309707006 | rhel-6-gui                     | /tmp/tmp/tmp/rhq-agent                           | 10.70.35.78  |       22 | jenkins      | 606f83b302d3426d
 10031 | 1398235247519 |                                |                                                  | 10.70.35.78  |       22 |              | 
(5 rows)
 
rhq=#

Screen shot is attached.

--- Additional comment from John Mazzitelli on 2014-04-24 12:10:09 EDT ---

Its possible multiple rows in rhq_agent_install table will show up under a few scenarios. We clean it up later when other things happen. In other words, rhq_agent_install table is an internal implementation detail that shouldn't concern end users.

As for the issue of installing multiple agents on the same machine, I am going to look into doing something like looking in the rhq_agent table to see if one is already registered. I might be able to use rhq_agent_install table as well, but I'll figure this out once I start working on it.

--- Additional comment from John Mazzitelli on 2014-04-24 16:21:41 EDT ---

I think the best thing to do here is to see if the port is already in use. If the port is in use, not only will this abort if another agent was installed, but it will abort if anything has taken the port the agent wants to use. This is helpful for other use-cases as well (as in the case of installing an agent on a machine where some other application is using the port the agent wants to use).

This has been implemented, git commit to master: c911e9a

--- Additional comment from Heiko W. Rupp on 2014-07-21 06:14:05 EDT ---

Bulk closing of RHQ 4.11 issues, now that RHQ 4.12 is out.

If you find an issue with those, please open a new BZ, linking to the old one.

Comment 1 Jeeva Kandasamy 2014-08-12 14:08:58 UTC
This issue is happening in JON 3.3

Version:
JBoss Operations Network
Version : 3.3.0.DR01
Build Number : 6468454:dda0a47
GWT Version : 2.5.0
SmartGWT Version : 3.0p

Comment 2 John Mazzitelli 2014-09-05 15:31:13 UTC
I confirmed that the patch c911e9a made it into the jon 3.3 branch. not sure why 3.3 still exhibits this problem. I will test to confirm and see what's going on.

Comment 3 John Mazzitelli 2014-09-05 20:08:28 UTC
i tried in RHQ master and it works better.

In JON 3.3 is still failed, but there was no real indication that it did other than at the end of the "installation information", the last list item shows "port in use".

I will see what additional code is in master that need to be cherry picked over for this to show a better error message

Comment 4 John Mazzitelli 2014-09-05 20:37:33 UTC
I have a feeling since RHQ is working ok, that we just need to wait for the next JON build. I think some code is already in the release branch that might address this.

Comment 5 Jay Shaughnessy 2014-09-09 18:44:02 UTC
Setting to MODIFIED for ER03 given Comment 4.

Comment 6 Simeon Pinder 2014-09-17 02:49:13 UTC
Moving to ON_QA as available for test with the following brew build:
https://brewweb.devel.redhat.com//buildinfo?buildID=385149

Comment 7 Jeeva Kandasamy 2014-09-18 13:18:57 UTC
Created attachment 938905 [details]
Agent-port-in-use-msg

Works as expected, screen shot is attached.

Version:
JBoss Operations Network
Version : 3.3.0.ER03
Build Number : 4aefe39:44e33a4
GWT Version : 2.5.0
SmartGWT Version : 3.0p