Created attachment 914971 [details]
Cannot delete application or deallocate gears

Description of problem:
I created an application with --scaling flag and minimum of 2 nodes. I later tried to delete it, and received error code 504. I tried deleting it again, received "Another operation is already running." Eventually 2 of the 3 gears were deleted, but the last gear is not deleteable: Unable to complete the requested operation due to: The node to create the gear on has not yet been identified. Reference ID: 877e49c3d4352647a2c34da021bcd380

It seems like this might be a race condition where the reference to the gear was somehow lost, and therefore the app cannot be deleted? The web UI (https://openshift.redhat.com/app/console/application/53b8fbd24382ecd207001541-server) was giving me the same error:

Unable to complete the requested operation due to: The node to create the gear on has not yet been identified Reference ID: ff993ac61293fb9cfcf9f353b60ae697

Now it just says: "We appear to be having technical difficulties". The Application page (https://openshift.redhat.com/app/console/applications) shows 3/3 gears used, but only 1 gear is allocated to the app.


Version-Release number of selected component (if applicable):
~/bin/rhc --version
rhc 1.26.9


How reproducible:
Getting error on each deletion attempt


Steps to Reproduce:
$ ~/bin/rhc app delete server
This is a non-reversible action! Your application code and data will be permanently deleted if you continue!

Are you sure you want to delete the application 'server'? (yes|no): yes

Deleting application 'server' ... 
Server returned an unexpected error code: 504

$ ~/bin/rhc app delete server
This is a non-reversible action! Your application code and data will be permanently deleted if you continue!

Are you sure you want to delete the application 'server'? (yes|no): y

Deleting application 'server' ... Unable to perform action on app object. Another operation is already running.

$ rhc app delete --confirm server -d
DEBUG: Using config file /home/akovacs/.openshift/express.conf
DEBUG: Git config 'git config --get rhc.app-id' returned '53b8fbd24382ecd207001541'
DEBUG: Git config 'git config --get rhc.app-name' returned 'server'
DEBUG: Git config 'git config --get rhc.domain-name' returned 'zapchart'
DEBUG: Authenticating with RHC::Auth::Token
DEBUG: Connecting to https://openshift.redhat.com/broker/rest/api
DEBUG: Client supports API versions 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7
DEBUG: Using token authentication
DEBUG: Created new httpclient
DEBUG: Request GET https://openshift.redhat.com/broker/rest/api
DEBUG:    code 200  595 ms
DEBUG: Server supports API versions 1.0, 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7
DEBUG:    Using API version 1.7
DEBUG: Client API version 1.7 is not current. Refetching API
DEBUG: Using token authentication
DEBUG: Request GET https://openshift.redhat.com/broker/rest/api
DEBUG:    code 200  145 ms
DEBUG: Using token authentication
DEBUG: Request GET https://openshift.redhat.com/broker/rest/domain/zapchart/application/server
DEBUG:    code 200  177 ms
Deleting application 'server' ... DEBUG: Deleting application server
DEBUG: Using token authentication
DEBUG: Request DELETE https://openshift.redhat.com/broker/rest/application/53b8fbd24382ecd207001541
DEBUG:    code 500  472 ms

Unable to complete the requested operation due to: The node to create the gear on has not yet been identified
Reference ID: 28a79653e78ccd0e48f58fbc1d3a555a


Expected results:
App should be deleted, gears should be deallocated. If this is a rare occurrence, then please feel free to manually delete/deallocate the gears. Thank you for your help.