Description of problem: create scalable jbossas app on stage always get failure. Version-Release number of selected component (if applicable): stg.openshift.redhat.com How reproducible: always Steps to Reproduce: 1.rhc app create -a scalejboss -t jbossas-7 -s 2. 3. Actual results: Submitting form: debug: true rhlogin: bmeng+1 Contacting https://stg.openshift.redhat.com Creating application: scalejb in bmeng1s /Library/Ruby/Gems/1.8/gems/rhc-0.93.18/lib/rhc-common.rb:514:in `create_app': undefined method `uuid' for []:Array (NoMethodError) from /Library/Ruby/Gems/1.8/gems/rhc-0.93.18/bin/rhc-app:227:in `create_app' from /Library/Ruby/Gems/1.8/gems/rhc-0.93.18/bin/rhc-app:499 from /usr/bin/rhc-app:23:in `load' from /usr/bin/rhc-app:23 Expected results: scalable jbossas app create successful. Additional info: this issue does not exist on devenv-stage_200 and INT env
after creation failure, app info can be listed in 'rhc domain show' but cannot be accessed via browser.
$ grep libra_server ~/.openshift/express.conf libra_server=stg.openshift.redhat.com $ rhc app create -p [hidden] -a scalejboss -t jbossas-7 -s Creating application: scalejboss in rmstg Now your new domain name is being propagated worldwide (this might take a minute)... retry # 4 - Waiting for DNS: scalejboss-rmstg.stg.rhcloud.com Warning: Permanently added 'scalejboss-rmstg.stg.rhcloud.com,107.21.77.136' (RSA) to the list of known hosts. Confirming application 'scalejboss' is available: Success! scalejboss published: http://scalejboss-rmstg.stg.rhcloud.com/ git url: ssh://1110d509273349e6ba12dfa7d6bac7e3.rhcloud.com/~/git/scalejboss.git/ Successfully created application: scalejboss Accessing haproxy shows the gear is up: http://scalejboss-rmstg.stg.rhcloud.com/haproxy-status/ Accessing the app shows the welcome page: http://scalejboss-rmstg.stg.rhcloud.com/ Suspect this is due to hitting one of the numerous timeouts we have baked into the system while stg was under a heavy load.
Stg has a broker watchdog that seems to be killing processes while they are creating jboss applications leaving the system in a weird state. The watchdog timeout was increased to 5 minutes and we specifically observed that it will wait out jboss app creation.
verified on today's stage, bug fixed.