Bug 852324

Summary:	All apps are not shown after update domain failed
Product:	OKD	Reporter:	joycezhang <jinzhang>
Component:	Master	Assignee:	Ravi Sankar <rpenta>
Status:	CLOSED CURRENTRELEASE	QA Contact:	libra bugs <libra-bugs>
Severity:	low	Docs Contact:
Priority:	high
Version:	2.x	CC:	johnp, rpenta
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	devenv_2337	Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2012-11-06 18:47:49 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description joycezhang 2012-08-28 08:07:03 UTC

Description of problem:
Update domain is failed with time out error displayed, and no apps are shown if using rhc domain show. In addition,it's failed to update if running update namespace command again:

# rhc domain show
Password: *


User Info
=========
Namespace: accep5

Login: jinzhang+6

Namespace accep5's Applications
===============================
RESULT:
Domain accep6 not found


Version-Release number of selected component (if applicable):
devenv_2085
rhc-0.98.3.gem

How reproducible:
Not always (It's happened when domain contains more apps, please refer to my app info list)

Steps to Reproduce:
1. Create some scalable and non-scalable apps
2. Update domain
#rhc domain update accep6
3. Run "rhc domain show" if time out error displayed
4. Run update domain again same as step2
  
Actual results:
1. Update domain is timeout as below:
# rhc domain update accep6 -d
Password: *

D, [2012-08-28T15:26:59.397752 #1907] DEBUG -- : Connecting to https://ec2-23-20-37-22.compute-1.amazonaws.com/broker/rest/api
D, [2012-08-28T15:27:01.789260 #1907] DEBUG -- : Getting all domains

Updating domain 'accep5' to namespace 'accep6'

RESULT:
Connection to server timed out. It is possible the operation finished without being able to report success. Use 'rhc domain show' or 'rhc app status' to check the status of your applications.

2. No apps are displayed when using "rhc domain show":
# rhc domain show
Password: *


User Info
=========
Namespace: accep5

Login: jinzhang+6

Namespace accep5's Applications
===============================
RESULT:
Domain accep6 not found

3. The domain name exists if trying to update domain again 

rhc domain update accep6 -d
Password: *

D, [2012-08-28T15:33:26.857864 #1946] DEBUG -- : Connecting to https://ec2-23-20-37-22.compute-1.amazonaws.com/broker/rest/api
D, [2012-08-28T15:33:29.081908 #1946] DEBUG -- : Getting all domains

Updating domain 'accep6' to namespace 'accep6'

RESULT:
Namespace 'accep6' already in use. Please choose another.


Expected results:
1.Update domain is successful.
2.All apps are displayed even if namespace haven't been updated competely.
3.It should be successful if trying to run update domain again as namespace hasn't been updated successfully in step2. 


Additional info:

Comment 1 joycezhang 2012-08-28 08:08:08 UTC

Here are my apps list for the reference:
User Info
=========
Namespace: accep5

Login: jinzhang+6

Namespace accep5's Applications
===============================
myscapp1
    Framework: jbossas-7
     Creation: 2012-08-28T01:18:29-04:00
         UUID: 1b0c09429af2440f93a6df20687c7ced
      Git URL: ssh://1b0c09429af2440f93a6df20687c7ced.rhcloud.com/~/git/myscapp1.git/
   Public URL: http://myscapp1-accep5.dev.rhcloud.com/
   Cartridges:
       haproxy-1.4
       mongodb-2.0 - mongodb://e97b99a172-accep5.dev.rhcloud.com:35581/

myscapp2
    Framework: perl-5.10
     Creation: 2012-08-28T01:22:10-04:00
         UUID: a272700d1eb646ef90298140d16111a2
      Git URL: ssh://a272700d1eb646ef90298140d16111a2.rhcloud.com/~/git/myscapp2.git/
   Public URL: http://myscapp2-accep5.dev.rhcloud.com/
   Cartridges:
       haproxy-1.4
       mongodb-2.0 - mongodb://96e1746617-accep5.dev.rhcloud.com:35671/

myscapp3
    Framework: php-5.3
     Creation: 2012-08-28T01:26:39-04:00
         UUID: c890b5592987445dabed5d0a382bea9b
      Git URL: ssh://c890b5592987445dabed5d0a382bea9b.rhcloud.com/~/git/myscapp3.git/
   Public URL: http://myscapp3-accep5.dev.rhcloud.com/
   Cartridges:
       postgresql-8.4 - postgresql://591716586b-accep5.dev.rhcloud.com:35596/
       haproxy-1.4

myscapp5
    Framework: nodejs-0.6
     Creation: 2012-08-28T01:35:55-04:00
         UUID: 16096e85a6954bd995adb90ec515e98a
      Git URL: ssh://16096e85a6954bd995adb90ec515e98a.rhcloud.com/~/git/myscapp5.git/
   Public URL: http://myscapp5-accep5.dev.rhcloud.com/
   Cartridges:
       postgresql-8.4 - postgresql://8628d935cb-accep5.dev.rhcloud.com:35641/
       haproxy-1.4
       mongodb-2.0 - mongodb://72c7211e7b-accep5.dev.rhcloud.com:35561/

myscapp6
    Framework: ruby-1.8
     Creation: 2012-08-28T01:39:16-04:00
         UUID: 33d4044ccfa44ecf90151dade7393bb7
      Git URL: ssh://33d4044ccfa44ecf90151dade7393bb7.rhcloud.com/~/git/myscapp6.git/
   Public URL: http://myscapp6-accep5.dev.rhcloud.com/
   Cartridges:
       postgresql-8.4 - postgresql://995fe09210-accep5.dev.rhcloud.com:35656/
       haproxy-1.4
       mongodb-2.0 - mongodb://d6e53c630a-accep5.dev.rhcloud.com:35681/

myscapp8
    Framework: ruby-1.9
     Creation: 2012-08-28T02:05:18-04:00
         UUID: c42963d4506c45fcb8891ccbc7f27807
      Git URL: ssh://c42963d4506c45fcb8891ccbc7f27807.rhcloud.com/~/git/myscapp8.git/
   Public URL: http://myscapp8-accep5.dev.rhcloud.com/
   Cartridges:
       postgresql-8.4 - postgresql://2219e14d1d-accep5.dev.rhcloud.com:35661/
       jenkins-client-1.4 - https://jenk-accep5.dev.rhcloud.com/job/myscapp8-build/
       haproxy-1.4
       mongodb-2.0 - mongodb://2961cec604-accep5.dev.rhcloud.com:35626/

myscapp9
    Framework: jbosseap-6.0
     Creation: 2012-08-28T02:09:19-04:00
         UUID: 045c12388e1a47939d5118c01af3e6d7
      Git URL: ssh://045c12388e1a47939d5118c01af3e6d7.rhcloud.com/~/git/myscapp9.git/
   Public URL: http://myscapp9-accep5.dev.rhcloud.com/
   Cartridges:
       postgresql-8.4 - postgresql://525ea25869-accep5.dev.rhcloud.com:35696/
       haproxy-1.4

jenk
    Framework: jenkins-1.4
     Creation: 2012-08-28T02:41:05-04:00
         UUID: ca8f2fa502584be49b90c85ed304a88a
      Git URL: ssh://ca8f2fa502584be49b90c85ed304a88a.rhcloud.com/~/git/jenk.git/
   Public URL: http://jenk-accep5.dev.rhcloud.com/
   Cartridges:
       None

Comment 2 joycezhang 2012-08-28 08:11:19 UTC

Update domain will be successful if updating domain with a new name instead of the name which was failed in step2. So set this bug as low serverity.

Comment 3 joycezhang 2012-09-03 10:45:13 UTC

This bug is still reproduced on devenv_2108.

All app info are lost after updating domain time out.

# rhc domain update joyceaccp3
Password: *


Updating domain 'accep2' to namespace 'joyceaccp3'

RESULT:
Connection to server timed out. It is possible the operation finished without being able to report success. Use 'rhc domain show' or 'rhc app status' to check the status of your applications.

# rhc domain show --trace
Password: *


User Info
=========
Namespace: accep2

Login: jinzhang+5

Namespace accep2's Applications
===============================
/usr/local/share/gems/gems/rhc-0.98.6/lib/rhc/rest.rb:186:in `block in process_error_response': Domain 'accep2' not found (RHC::Rest::ResourceNotFoundException)
	from /usr/local/share/gems/gems/rhc-0.98.6/lib/rhc/rest.rb:184:in `each'
	from /usr/local/share/gems/gems/rhc-0.98.6/lib/rhc/rest.rb:184:in `process_error_response'
	from /usr/local/share/gems/gems/rhc-0.98.6/lib/rhc/rest.rb:160:in `rescue in request'
	from /usr/local/share/gems/gems/rhc-0.98.6/lib/rhc/rest.rb:147:in `request'
	from /usr/local/share/gems/gems/rhc-0.98.6/lib/rhc/rest/domain.rb:39:in `applications'
	from /usr/local/share/gems/gems/rhc-0.98.6/lib/rhc/commands/domain.rb:71:in `block (2 levels) in show'
	from /usr/local/share/gems/gems/rhc-0.98.6/lib/rhc/helpers.rb:168:in `call'
	from /usr/local/share/gems/gems/rhc-0.98.6/lib/rhc/helpers.rb:168:in `section'
	from /usr/local/share/gems/gems/rhc-0.98.6/lib/rhc/helpers.rb:185:in `paragraph'
	from /usr/local/share/gems/gems/rhc-0.98.6/lib/rhc/commands/domain.rb:67:in `block in show'
	from /usr/local/share/gems/gems/rhc-0.98.6/lib/rhc/commands/domain.rb:66:in `each'
	from /usr/local/share/gems/gems/rhc-0.98.6/lib/rhc/commands/domain.rb:66:in `show'
	from /usr/local/share/gems/gems/rhc-0.98.6/lib/rhc/commands.rb:160:in `block (3 levels) in to_commander'
	from /usr/local/share/gems/gems/commander-4.1.2/lib/commander/command.rb:180:in `call'
	from /usr/local/share/gems/gems/commander-4.1.2/lib/commander/command.rb:180:in `call'
	from /usr/local/share/gems/gems/commander-4.1.2/lib/commander/command.rb:155:in `run'
	from /usr/local/share/gems/gems/commander-4.1.2/lib/commander/runner.rb:402:in `run_active_command'
	from /usr/local/share/gems/gems/rhc-0.98.6/lib/rhc/commands.rb:73:in `run!'
	from /usr/local/share/gems/gems/commander-4.1.2/lib/commander/delegates.rb:11:in `run!'
	from /usr/local/share/gems/gems/rhc-0.98.6/lib/rhc/cli.rb:37:in `start'
	from /usr/local/share/gems/gems/rhc-0.98.6/bin/rhc:62:in `<top (required)>'
	from /usr/local/bin/rhc:23:in `load'
	from /usr/local/bin/rhc:23:in `<main>'

Comment 4 John (J5) Palmieri 2012-10-04 15:14:50 UTC

The rest api needs to be able to handle disconnects during updates.  We can not judge how long this is going to take and a timeout will happen in the most extreme cases.  Once an update is issued it should complete on the server even if we disconnect.

Comment 5 Ravi Sankar 2012-10-16 18:55:43 UTC

https://github.com/openshift/li/pull/484
Fixed and merged.

Comment 6 Ravi Sankar 2012-10-16 19:32:26 UTC

li pull request is merged but the crankcase pull request https://github.com/openshift/origin-server/pull/676 is yet to be merged. Will put ON_QA status once it is merged.

Comment 7 joycezhang 2012-10-17 08:04:35 UTC

This bug has been verified and fixed on devenv_2340. All apps are shown with old domain name if domain update failed. Mark it as Verified. Thanks.