Bug 1037599

Summary: 2nd atempt to rhqctl install fails - hanging connections
Product: [JBoss] JBoss Operations Network Reporter: Libor Zoubek <lzoubek>
Component: InstallerAssignee: RHQ Project Maintainer <rhq-maint>
Status: CLOSED NOTABUG QA Contact: Mike Foley <mfoley>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: JON 3.2CC: theute
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Windows   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-12-03 14:54:39 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Libor Zoubek 2013-12-03 12:56:58 UTC 
Description of problem:

On windows 2008, I've tried to install JON server, but I forgot to create database (postgres). So I canceled installation because I saw PSQL exceptions, and started it again, but failed.


Version-Release number of selected component (if applicable):
JON 3.2.0.ER7

How reproducible: always


Steps to Reproduce:
1. drop 'rhq' database
2. run rhqctl install
3. wait until it fails - it shows PSQLExceptions in console, server reports in when unable to connect to database
4. Ctrl^C installation, wait for installer does REDO steps
5. create database rhq
6. run "rhqctl install" again

Actual results:

It fails again, JON server is not able to start because it cannot bind to 6999. Something is there. I've checked "netstat" or processes and I cannot see any java processes, but I can see some TCP connections aiming to port 6999 and others. Maybe we do not clean up installer's connections properly?

The only workaround I saw sofar is reboot the server

Expected results:

I should be able to install server after I've fixed my environment

Additional info: This seems to be weird enough and maybe out of scope of JON server or installer, not sure.


Before step 2:

C:\jon-server-3.2.0.ER7\bin>netstat

Active Connections

  Proto  Local Address          Foreign Address        State
  TCP    10.16.23.234:3389      ovpn-112-60:57124      ESTABLISHED
  TCP    127.0.0.1:31000        testday2:32000         ESTABLISHED
  TCP    127.0.0.1:32000        testday2:31000         ESTABLISHED
  TCP    127.0.0.1:49160        testday2:49161         ESTABLISHED
  TCP    127.0.0.1:49161        testday2:49160         ESTABLISHED
  TCP    127.0.0.1:49162        testday2:49163         ESTABLISHED
  TCP    127.0.0.1:49163        testday2:49162         ESTABLISHED
  TCP    127.0.0.1:49164        testday2:49165         ESTABLISHED
  TCP    127.0.0.1:49165        testday2:49164         ESTABLISHED
  TCP    127.0.0.1:49166        testday2:49167         ESTABLISHED
  TCP    127.0.0.1:49167        testday2:49166         ESTABLISHED
  TCP    127.0.0.1:49168        testday2:49169         ESTABLISHED
  TCP    127.0.0.1:49169        testday2:49168         ESTABLISHED

After step 6 (no java processes running):

C:\jon-server-3.2.0.ER7\bin>netstat

Active Connections

  Proto  Local Address          Foreign Address        State
  TCP    10.16.23.234:3389      ovpn-112-60:57124      ESTABLISHED
  TCP    127.0.0.1:6999         testday2:49239         TIME_WAIT
  TCP    127.0.0.1:6999         testday2:49249         TIME_WAIT
  TCP    127.0.0.1:31000        testday2:32000         TIME_WAIT
  TCP    127.0.0.1:31001        testday2:32001         TIME_WAIT
  TCP    127.0.0.1:31002        testday2:32000         TIME_WAIT
  TCP    127.0.0.1:49160        testday2:49161         TIME_WAIT
  TCP    127.0.0.1:49162        testday2:49163         TIME_WAIT
  TCP    127.0.0.1:49164        testday2:49165         TIME_WAIT
  TCP    127.0.0.1:49166        testday2:49167         TIME_WAIT
  TCP    127.0.0.1:49197        testday2:49198         TIME_WAIT
  TCP    127.0.0.1:49199        testday2:49200         TIME_WAIT
  TCP    127.0.0.1:49201        testday2:49202         TIME_WAIT
  TCP    127.0.0.1:49204        testday2:49205         TIME_WAIT
  TCP    127.0.0.1:49206        testday2:49207         TIME_WAIT
  TCP    127.0.0.1:49208        testday2:49209         TIME_WAIT
  TCP    127.0.0.1:49210        testday2:49211         TIME_WAIT
  TCP    127.0.0.1:49214        testday2:49215         TIME_WAIT
  TCP    127.0.0.1:49216        testday2:49217         TIME_WAIT
  TCP    127.0.0.1:49219        testday2:49220         TIME_WAIT
  TCP    127.0.0.1:49221        testday2:49222         TIME_WAIT
  TCP    127.0.0.1:49223        testday2:6999          TIME_WAIT
  TCP    127.0.0.1:49224        testday2:49225         TIME_WAIT
  TCP    127.0.0.1:49226        testday2:49227         TIME_WAIT
  TCP    127.0.0.1:49228        testday2:6999          TIME_WAIT
  TCP    127.0.0.1:49229        testday2:49230         TIME_WAIT
  TCP    127.0.0.1:49231        testday2:49232         TIME_WAIT
  TCP    127.0.0.1:49233        testday2:6999          TIME_WAIT
  TCP    127.0.0.1:49235        testday2:49236         TIME_WAIT
  TCP    127.0.0.1:49237        testday2:49238         TIME_WAIT
  TCP    127.0.0.1:49240        testday2:49241         TIME_WAIT
  TCP    127.0.0.1:49242        testday2:49243         TIME_WAIT
  TCP    127.0.0.1:49245        testday2:49246         TIME_WAIT
  TCP    127.0.0.1:49247        testday2:49248         TIME_WAIT
  TCP    127.0.0.1:49250        testday2:49251         TIME_WAIT
  TCP    127.0.0.1:49252        testday2:49253         TIME_WAIT
  TCP    127.0.0.1:49254        testday2:6999          TIME_WAIT
  TCP    127.0.0.1:49255        testday2:49256         TIME_WAIT
  TCP    127.0.0.1:49257        testday2:49258         TIME_WAIT
  TCP    127.0.0.1:49259        testday2:6999          TIME_WAIT
  TCP    127.0.0.1:49260        testday2:49261         TIME_WAIT
  TCP    127.0.0.1:49262        testday2:49263         TIME_WAIT
  TCP    127.0.0.1:49264        testday2:6999          TIME_WAIT
  TCP    127.0.0.1:49265        testday2:49266         TIME_WAIT
  TCP    127.0.0.1:49267        testday2:49268         TIME_WAIT
  TCP    127.0.0.1:49269        testday2:6999          TIME_WAIT
  TCP    127.0.0.1:49270        testday2:49271         TIME_WAIT
  TCP    127.0.0.1:49272        testday2:49273         TIME_WAIT
  TCP    127.0.0.1:49274        testday2:6999          TIME_WAIT
  TCP    127.0.0.1:49276        testday2:49277         TIME_WAIT
  TCP    127.0.0.1:49278        testday2:49279         TIME_WAIT
  TCP    127.0.0.1:49280        testday2:6999          TIME_WAIT
  TCP    127.0.0.1:49281        testday2:49282         TIME_WAIT
  TCP    127.0.0.1:49283        testday2:49284         TIME_WAIT
  TCP    127.0.0.1:49285        testday2:6999          TIME_WAIT
  TCP    127.0.0.1:49286        testday2:49287         TIME_WAIT
  TCP    127.0.0.1:49288        testday2:49289         TIME_WAIT
  TCP    127.0.0.1:49290        testday2:6999          TIME_WAIT
  TCP    [::1]:5432             testday2:49203         ESTABLISHED
  TCP    [::1]:49203            testday2:5432          ESTABLISHED
Comment 1 Libor Zoubek 2013-12-03 14:54:39 UTC 
cannot reproduce anymore (tried 4times)