Bug 1035481 - Port 9999 is already in use or unavailable when starting/installing/upgrading
Summary: Port 9999 is already in use or unavailable when starting/installing/upgrading
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: JBoss Operations Network
Classification: JBoss
Component: Core Server, Installer
Version: JON 3.2.3
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ER01
: JON 3.3.4
Assignee: Michael Burman
QA Contact: Sunil Kondkar
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-11-27 21:27 UTC by mgottval
Modified: 2019-06-13 07:55 UTC (History)
10 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-10-28 14:36:23 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 1145183 0 None None None Never
Red Hat Product Errata RHSA-2015:1947 0 normal SHIPPED_LIVE Important: Red Hat JBoss Operations Network 3.3.4 update 2015-10-28 18:36:15 UTC

Description mgottval 2013-11-27 21:27:49 UTC
1. Run JPP 6.1
2. install the JPP plugins into jon-server/plugins
3. Set required properties and install the jon-server by 'rhqctl install' 

ERROR [org.rhq.server.control.RHQControl] Something is already listening to port 9999 - shut it down before installing the server.

The JPP console is set to the same port, so I have to change the config file standalone.xml '<socket-binding name="management-native" interface="management" port="${jboss.management.native.port:9999}"/>' port to another value to be able to install the jon-server.

But when I first install the jon-server, then start JPP, there is no exception and everything runs.

Comment 2 Stefan Negrea 2014-09-05 14:04:20 UTC
JON 3.x is installed on top of stock EAP. Post-installation the ports are different than default 9999; however, there are no modifications done to the container prior to installation. This issue happens only on systems where JON server is colocated with a managed EAP resource that also happens to run on default ports. Co-locating managed resources and JON server components (agent, server, or storage nodes) is not a recommended configuration.

Moving to a future release since this is not a recommended configuration.

Comment 3 Larry O'Leary 2014-09-09 19:11:27 UTC
Resetting target as this is not an enhancement but a bug. Considering this deals with the base image and installer, this can not be addressed post 3.3.

In the current state, JBoss ON requires duplicate ports to run the installer. This does not make any sense. Furthermore, we should not be using the example configuration provided by a JBoss EAP base installation as our installer configuration. Instead, we should provide a JBoss ON configuration file using our own recommended defaults. For example, in this case port 6999 and not port 9999.

Comment 4 Jay Shaughnessy 2014-09-09 20:59:29 UTC
I agree that it's not relevant to a CP release and should not be targeted to 3.2.1.

The idea here is that JON be able to install into a pre-existing EAP instance.  Which actually, I think it still can (although I don't know if the steps are documented).  This is [one reason] why we install via the our settings via the CLI and not via a pre-defined config file.

But as far as I know no one does that and they start up the vanilla EAP that we package with.  And that has all of the default ports in play.

We could feasibly start up the vanilla EAP with a boatload of -D jboss properties (the same port overrides we set via the installer, using the port defs in rhq-server.properties) and then the initial startup would mirror the latter RHQ startup.

But this won't be done for 3.3.0.GA, just capturing the idea.  I think the warning and the workaround of shutting down the currently running process is reasonable.

Comment 7 Larry O'Leary 2015-03-17 21:45:11 UTC
The workaround of shutting down the process that is already using port 9999 is not feasible. In a production system, port 9999 could be occupied by just about anything. Such services may be considered mission critical and shutting them down would mean significant down-time or service disruption across an enterprise.

It should be no problem using our own default ports initially and providing an method to easily override them. There is no EAP/AS restrictions that we use their example configuration files such as was done in JBoss ON 3.2/3.3 of standalone-full.xml. This file was only meant to be an example and the expectation was that a product that used the underlying modules provided by the EAP/AS instance would provide it own configuration file such as jboss-on.xml which included the base/default configuration.

I think the bulk of the issue here is that rhqctl is performing the check to see if anything is using port 9999. This is true even if we override the default port 9999 to use 6999 using system properties passed to the installer. For whatever reason, rhqctl still insists on using 9999 when performing the validation instead of the management port passed to the installer.

Comment 8 Larry O'Leary 2015-04-14 17:00:15 UTC
This issue also occurs when security policy (such as used on Windows Server) or a firewall rule prevents the use of the loopback interface or TCP port 9999.

In such cases the user properly configures the management host and ports but they are being ignored during installation or upgrade. This prevents the user from installing JBoss ON on such systems even when their operating system, platform, and JVM are fully supported.

Comment 9 Michael Burman 2015-09-17 11:38:47 UTC
Fixed in the master, for example:

[miburman@miranda bin]$ export RHQ_CONTROL_ADDITIONAL_JAVA_OPTS="-Djboss.management.native.port=6999"
[miburman@miranda bin]$ ./rhqctl install

Allows installation while another clean instance of EAP is running.

commit 54e9f45a63e14fa84618ff809e7fb955e5e55c0b
Author: Michael Burman <miburman>
Date:   Thu Sep 17 14:36:49 2015 +0300

    [BZ 1035481] Defining jboss.management.native.port in RHQ_CONTROL_ADDITIONAL_JAVA_OPTS will now be used to check if port is available for installation

Comment 10 Michael Burman 2015-09-17 19:38:01 UTC
Additional note: should we use the port 6999 (which is the final value) as the default port by passing the jboss.management.native.port=6999 in the rhqctl?

Comment 11 Larry O'Leary 2015-09-17 19:44:30 UTC
That would definitely be ideal. However, we really need to be using what is already defined in rhq-server.properties. This way, upgrades will work if using a port different then 6999.

Comment 12 Michael Burman 2015-09-18 08:00:15 UTC
Second fix then (remember to merge both in correct order) in the master:

commit 0cca2c07cdefe6f076fc129a0c4173a38f61c0c2
Author: Michael Burman <miburman>
Date:   Fri Sep 18 10:58:40 2015 +0300

    [BZ 1035481] Set the jboss.* properties from rhq-server.properties as System-properties



This reads all the jboss.* parameters from rhq-server.properties and sets them to the System so that the EAP installation will read them.

Comment 13 Libor Zoubek 2015-09-25 08:37:21 UTC
branch:  release/jon3.3.x
link:    https://github.com/rhq-project/rhq/commit/30121af2c
time:    2015-09-25 10:24:49 +0200
commit:  30121af2c24f89e1f760477b0cae9f6470e32ac7
author:  Michael Burman - miburman
message: [BZ 1035481] Set the jboss.* properties from rhq-server.properties as
         System-properties

         (cherry picked from commit
         0cca2c07cdefe6f076fc129a0c4173a38f61c0c2) Signed-off-by: Libor
         Zoubek <lzoubek>



branch:  release/jon3.3.x
link:    https://github.com/rhq-project/rhq/commit/5aeda8469
time:    2015-09-25 10:21:18 +0200
commit:  5aeda8469006dde9500658c845f8632d22301b95
author:  Michael Burman - miburman
message: [BZ 1035481] Defining jboss.management.native.port in
         RHQ_CONTROL_ADDITIONAL_JAVA_OPTS will now be used to check if
         port is available for installation

         (cherry picked from commit
         54e9f45a63e14fa84618ff809e7fb955e5e55c0b) Signed-off-by: Libor
         Zoubek <lzoubek>

Comment 14 Simeon Pinder 2015-10-09 04:40:23 UTC
Moving to ON_QA as available to test with the following build:
https://brewweb.devel.redhat.com/buildinfo?buildID=460382

 *Note: jon-server-patch-3.3.0.GA.zip maps to ER01 build of
 jon-server-3.3.0.GA-update-04.zip.

Comment 15 Sunil Kondkar 2015-10-13 15:12:49 UTC
Verified on Version :3.3.0.GA Update 04 Build Number :821a526:fa7b1a1

Installed and run JPP 6.1. The error is reproduced when tried to install JON 3.3.0 GA with rhqctl install. After applying 3.3.0.GA Update 04, verified that rhqctl install is successful without error.

Also verified for rhqctl start and for rhqctl upgrade from JBoss ON 3.2 GA to  3.3.0.GA. After applying '3.3.0.GA Update 04' to JBOss ON 3.3.0.GA, rhqctl start and rhqctl upgrade works without error.

Comment 17 errata-xmlrpc 2015-10-28 14:36:23 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-1947.html


Note You need to log in before you can comment on or make changes to this bug.