+++ This bug was initially created as a clone of Bug #1090829 +++ Description of problem: Try to install remote agent more than one time on same machine with different locations with same user. First installation went success and agent up and running. Second time installed on different location and up, but throws an exception saying the port already in use. ----------------snap--agent.log---------------- 2014-04-24 14:29:52,512 ERROR [main] (org.rhq.enterprise.agent.AgentMain)- {AgentMain.start-failure}Failed to start the agent. Cause: java.io.IOException: SocketServerInvoker[10.70.35.78:16163] error creating ServerSocket[10.70.35.78:16163]: Address already in use ----------------snap--------------------------- ps -ef | grep rhq --------------------------------------------------- [jenkins@rhel-6-gui tmp]$ ps -ef | grep rhq jenkins 16462 1 0 Apr23 ? 00:01:12 /usr/bin/java -Djava.endorsed.dirs=/tmp/rhq-agent/lib/endorsed -Djava.library.path=/tmp/rhq-agent/lib -Djna.platform.library.path=/tmp/rhq-agent/lib/augeas/lib64 -Xms64m -Xmx128m -Djava.net.preferIPv4Stack=true -Drhq.preferences.file=/tmp/rhq-agent/conf/agent-prefs.properties -Dlog4j.configuration=log4j.xml -cp /tmp/rhq-agent/conf:/tmp/rhq-agent/lib/commons-io-1.4.jar:/tmp/rhq-agent/lib/commons-logging-1.1.0.jboss.jar:/tmp/rhq-agent/lib/concurrent-1.3.4-jboss-update1.jar:/tmp/rhq-agent/lib/getopt-1.0.13.jar:/tmp/rhq-agent/lib/i18nlog-1.0.10.jar:/tmp/rhq-agent/lib/jboss-common-core-2.2.17.GA.jar:/tmp/rhq-agent/lib/jboss-jmx-4.2.3.GA.jar:/tmp/rhq-agent/lib/jboss-logging-3.1.2.GA.jar:/tmp/rhq-agent/lib/jboss-remoting-2.5.4.SP5.jar:/tmp/rhq-agent/lib/jboss-serialization-1.0.3.GA.jar:/tmp/rhq-agent/lib/jline-0.9.94.jar:/tmp/rhq-agent/lib/log4j-1.2.16.jar:/tmp/rhq-agent/lib/persistence-api-1.0.jar:/tmp/rhq-agent/lib/rhq-common-drift-4.11.0-SNAPSHOT.jar:/tmp/rhq-agent/lib/rhq-core-client-api-4.11.0-SNAPSHOT.jar:/tmp/rhq-agent/lib/rhq-core-comm-api-4.11.0-SNAPSHOT.jar:/tmp/rhq-agent/lib/rhq-core-domain-4.11.0-SNAPSHOT.jar:/tmp/rhq-agent/lib/rhq-core-native-system-4.11.0-SNAPSHOT.jar:/tmp/rhq-agent/lib/rhq-core-plugin-api-4.11.0-SNAPSHOT.jar:/tmp/rhq-agent/lib/rhq-core-plugin-container-4.11.0-SNAPSHOT.jar:/tmp/rhq-agent/lib/rhq-core-util-4.11.0-SNAPSHOT.jar:/tmp/rhq-agent/lib/rhq-enterprise-agent-4.11.0-SNAPSHOT.jar:/tmp/rhq-agent/lib/rhq-enterprise-comm-4.11.0-SNAPSHOT.jar:/tmp/rhq-agent/lib/sigar-1.6.5.132-6.jar:/tmp/rhq-agent/lib/trove4j-3.0.3.jar org.rhq.enterprise.agent.AgentMain -Drhq.agent.server.bind-address=rhqbox1.lab.eng.pnq.redhat.com -Drhq.communications.connector.bind-address=10.70.35.78 -Drhq.agent.configuration-setup-flag=true --daemon -Drhq.install.id=10012 jenkins 16899 1 0 Apr23 ? 00:00:54 /usr/bin/java -Djava.endorsed.dirs=/tmp/tmp/rhq-agent/lib/endorsed -Djava.library.path=/tmp/tmp/rhq-agent/lib -Djna.platform.library.path=/tmp/tmp/rhq-agent/lib/augeas/lib64 -Xms64m -Xmx128m -Djava.net.preferIPv4Stack=true -Drhq.preferences.file=/tmp/tmp/rhq-agent/conf/agent-prefs.properties -Dlog4j.configuration=log4j.xml -cp /tmp/tmp/rhq-agent/conf:/tmp/tmp/rhq-agent/lib/commons-io-1.4.jar:/tmp/tmp/rhq-agent/lib/commons-logging-1.1.0.jboss.jar:/tmp/tmp/rhq-agent/lib/concurrent-1.3.4-jboss-update1.jar:/tmp/tmp/rhq-agent/lib/getopt-1.0.13.jar:/tmp/tmp/rhq-agent/lib/i18nlog-1.0.10.jar:/tmp/tmp/rhq-agent/lib/jboss-common-core-2.2.17.GA.jar:/tmp/tmp/rhq-agent/lib/jboss-jmx-4.2.3.GA.jar:/tmp/tmp/rhq-agent/lib/jboss-logging-3.1.2.GA.jar:/tmp/tmp/rhq-agent/lib/jboss-remoting-2.5.4.SP5.jar:/tmp/tmp/rhq-agent/lib/jboss-serialization-1.0.3.GA.jar:/tmp/tmp/rhq-agent/lib/jline-0.9.94.jar:/tmp/tmp/rhq-agent/lib/log4j-1.2.16.jar:/tmp/tmp/rhq-agent/lib/persistence-api-1.0.jar:/tmp/tmp/rhq-agent/lib/rhq-common-drift-4.11.0-SNAPSHOT.jar:/tmp/tmp/rhq-agent/lib/rhq-core-client-api-4.11.0-SNAPSHOT.jar:/tmp/tmp/rhq-agent/lib/rhq-core-comm-api-4.11.0-SNAPSHOT.jar:/tmp/tmp/rhq-agent/lib/rhq-core-domain-4.11.0-SNAPSHOT.jar:/tmp/tmp/rhq-agent/lib/rhq-core-native-system-4.11.0-SNAPSHOT.jar:/tmp/tmp/rhq-agent/lib/rhq-core-plugin-api-4.11.0-SNAPSHOT.jar:/tmp/tmp/rhq-agent/lib/rhq-core-plugin-container-4.11.0-SNAPSHOT.jar:/tmp/tmp/rhq-agent/lib/rhq-core-util-4.11.0-SNAPSHOT.jar:/tmp/tmp/rhq-agent/lib/rhq-enterprise-agent-4.11.0-SNAPSHOT.jar:/tmp/tmp/rhq-agent/lib/rhq-enterprise-comm-4.11.0-SNAPSHOT.jar:/tmp/tmp/rhq-agent/lib/sigar-1.6.5.132-6.jar:/tmp/tmp/rhq-agent/lib/trove4j-3.0.3.jar org.rhq.enterprise.agent.AgentMain -Drhq.agent.server.bind-address=rhqbox1.lab.eng.pnq.redhat.com -Drhq.communications.connector.bind-address=10.70.35.78 -Drhq.agent.configuration-setup-flag=true --daemon -Drhq.install.id=10031 jenkins 22597 22206 0 14:32 pts/4 00:00:00 grep rhq --------------------------------------------------- Version-Release number of selected component (if applicable): Version : 4.11.0-SNAPSHOT Build Number : 1bc9904 GWT Version : 2.5.0 SmartGWT Version : 3.0 Browser: Firefox 28 How reproducible: always Steps to Reproduce: 1. Navigate to "Administration-->Agents-->New" 2. Enter remote box details (Agent should not be installed), give some location, example: /tmp/ 3. Click on "install Agent", will be installed successfully 4. Again Navigate to "Administration-->Agents-->New" 5. Enter remote box details same as in step #2, give different location location, example: /tmp/another/ 6. Click on "install Agent", will be installed successfully. But the port already use with agent installed on step #3. 7. Second installation should refer first installation location and should not proceed more than one time. Actual results: Permits second installation on different location for the same user on the same machine. Expected results: Should not install more than one time, if a agent is already installed on different location. We should track with some reference in common location. --- Additional comment from Jeeva Kandasamy on 2014-04-24 11:06:21 EDT --- in the above case, If we install multiple times, we can see more than one entry in "rhq_agent_install" table, rhq=# select * from rhq_agent_install; id | mtime | agent_name | install_location | ssh_host | ssh_port | ssh_username | ssh_password -------+---------------+--------------------------------+--------------------------------------------------+--------------+----------+--------------+------------------ 10001 | 1398210095484 | rhqbox1.lab.eng.pnq.redhat.com | /home/hudson/rhq-server-location-fresh/rhq-agent | 10.65.206.42 | | | 10011 | 1398229679011 | | | 10.70.35.78 | 22 | | 10013 | 1398308668469 | | | 10.70.35.78 | 22 | jenkins | 606f83b302d3426d 10012 | 1398309707006 | rhel-6-gui | /tmp/tmp/tmp/rhq-agent | 10.70.35.78 | 22 | jenkins | 606f83b302d3426d 10031 | 1398235247519 | | | 10.70.35.78 | 22 | | (5 rows) rhq=# Screen shot is attached. --- Additional comment from John Mazzitelli on 2014-04-24 12:10:09 EDT --- Its possible multiple rows in rhq_agent_install table will show up under a few scenarios. We clean it up later when other things happen. In other words, rhq_agent_install table is an internal implementation detail that shouldn't concern end users. As for the issue of installing multiple agents on the same machine, I am going to look into doing something like looking in the rhq_agent table to see if one is already registered. I might be able to use rhq_agent_install table as well, but I'll figure this out once I start working on it. --- Additional comment from John Mazzitelli on 2014-04-24 16:21:41 EDT --- I think the best thing to do here is to see if the port is already in use. If the port is in use, not only will this abort if another agent was installed, but it will abort if anything has taken the port the agent wants to use. This is helpful for other use-cases as well (as in the case of installing an agent on a machine where some other application is using the port the agent wants to use). This has been implemented, git commit to master: c911e9a --- Additional comment from Heiko W. Rupp on 2014-07-21 06:14:05 EDT --- Bulk closing of RHQ 4.11 issues, now that RHQ 4.12 is out. If you find an issue with those, please open a new BZ, linking to the old one.
This issue is happening in JON 3.3 Version: JBoss Operations Network Version : 3.3.0.DR01 Build Number : 6468454:dda0a47 GWT Version : 2.5.0 SmartGWT Version : 3.0p
I confirmed that the patch c911e9a made it into the jon 3.3 branch. not sure why 3.3 still exhibits this problem. I will test to confirm and see what's going on.
i tried in RHQ master and it works better. In JON 3.3 is still failed, but there was no real indication that it did other than at the end of the "installation information", the last list item shows "port in use". I will see what additional code is in master that need to be cherry picked over for this to show a better error message
I have a feeling since RHQ is working ok, that we just need to wait for the next JON build. I think some code is already in the release branch that might address this.
Setting to MODIFIED for ER03 given Comment 4.
Moving to ON_QA as available for test with the following brew build: https://brewweb.devel.redhat.com//buildinfo?buildID=385149
Created attachment 938905 [details] Agent-port-in-use-msg Works as expected, screen shot is attached. Version: JBoss Operations Network Version : 3.3.0.ER03 Build Number : 4aefe39:44e33a4 GWT Version : 2.5.0 SmartGWT Version : 3.0p