Description of problem: Use 2 different users to create 2 scalable jbossas app at the same time. Sometimes the creation will fail because of "Node execution failure (invalid exit code from node)." Version-Release number of selected component (if applicable): devenv_1931 How reproducible: Sometimes Steps to Reproduce: 1.Create 2 scalable jbossas app at the same time $ rhc app create -a app1 -t jbossas-7 -s -l <rhlogin1> $ rhc app create -a app2 -t jbossas-7 -s -l <rhlogin2> 2.Check development.log on the server 3. Actual results: I've attached the development.log Sometimes the creation fails. <--------------client output---------------> Contacting https://ec2-23-22-239-132.compute-1.amazonaws.com Creating application: app2 in jizhao70 Problem reported from server. Response code was 500. MESSAGES: Failed to create application app2 due to: Node execution failure (invalid exit code from node). If the problem persists please contact Red Hat support. <------------------------------------------> Expected results: App should be created successfully. Additional info:
Created attachment 601709 [details] development.log
Unable to reproduce the problem after 10 tries; however, the 500 error was generated by an exception raised from inside the code due to the failure of app_ctl.sh #<NoMethodError: undefined method `cart_commands' for [{:args=>[], :command=>"SYSTEM_SSH_KEY_REMOVE"}]:Array> /usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.14.14/lib/stickshift-controller/app/models/application.rb:1349
Pull request to resolve the exception. https://github.com/openshift/crankcase/pull/314
Additional bug fixes were required. The exception should be fixed in devenv_1933.
I am still seeing this on devenv1933 and I saw it the first time itself. Attaching development.log 1.Create 2 scalable jbossas app at the same time $ rhc app create -a app1 -t jbossas-7 -s -l <rhlogin1> $ rhc app create -a app2 -t jbossas-7 -s -l <rhlogin2> 2.Check development.log on the server
Created attachment 601846 [details] development_new.log
It looks like the haproxy configure hook is failing for some reason: DEBUG: server results: SSH_KEY_ADD: AAAAB3NzaC1yc2EAAAABIwAAAQEAy1vdhhL/tt2gabawSEvil95SSqxDg2JD94MqlZ A70Y/eY9hYshmzzNsrIgQzxvA+DKqBGp3ehUwz1RIHuBUb6NHoD2e9lZJGBEWYMU39D10GQv+lFKYUyCHI9OY6w+m3Mc/JmcL2kjX4 zq0g68Y/RtO8Co9eBdLcn0DkDf3egxAjI55G/47bnhTuoak7g2ArkVESLlBrtceN7LMgUDbshJlCgjuk4rspCQ2pnR9fgvBaMIWb4O ArEGncdUbmNy0txP2JgDEwx/TaHvZFjKFSXHvRJWpL0k60g5iz+Ja+yb00A9DWZhij64tGHZ2gVycHAEIPPBMa9JElazaIAzRg4w== DEBUG: server results: BROKER_AUTH_KEY_ADD: DEBUG: server results: Failed to start /var/lib/stickshift/3f69f4c7ac564c12b5336b0e2d0ed5b4/jbossas-7/jb1_ctl.sh DEBUG: Cartridge command embedded/haproxy-1.4::configure exitcode = 121 Will keep trying to reproduce.
Not seeing this anymore.. 1.Create 2 scalable jbossas app at the same time $ rhc app create -a app1 -t jbossas-7 -s -l cgole $ rhc app create -a app2 -t jbossas-7 -s -l cgole+1 2.Check development.log on the server
Marking low due to the difficulty reproducing.
We've not heard of any reports of this happening in prod either. Closing bug for cleanup, please re-open if the issue still exists.