Description of problem: Our automation creates and then destroys applications to test the broker and verify that it is running properly. One of the applications was stuck and not able to be destroyed. Here is the mcollective.log from the server. I, [2013-10-06T07:11:08.675968 #1305] INFO -- : openshift.rb:180:in `execute_parallel_action' execute_parallel_action call / action: execute_parallel, agent=openshift, data={"ex-std-node234.prod.rhcloud.com"=> [{:tag=>"", :gear=>"524dee49e0b8cdf32e000001", :job=> {:cartridge=>"openshift-origin-node", :action=>"app-state-show", :args=> {"--with-app-uuid"=>"524dee49e0b8cdf32e000001", "--with-app-name"=>"chkexsrv1", "--with-container-uuid"=>"524dee49e0b8cdf32e000001", "--with-container-name"=>"chkexsrv1", "--with-namespace"=>"openshiftnagios", "--with-uid"=>6479, "--with-request-id"=>"969099581a9a5708db6d39f4e71712af"}}, :result_stdout=>"", :result_stderr=>"", :result_exit_code=>""}], :process_results=>true} I, [2013-10-06T07:11:08.676286 #1305] INFO -- : openshift.rb:131:in `execute_action' Executing action [app-state-show] using method oo_app_state_show with args [{"--with-app-uuid"=>"524dee49e0b8cdf32e000001", "--with-app-name"=>"chkexsrv1", "--with-container-uuid"=>"524dee49e0b8cdf32e000001", "--with-container-name"=>"chkexsrv1", "--with-namespace"=>"openshiftnagios", "--with-uid"=>6479, "--with-request-id"=>"969099581a9a5708db6d39f4e71712af"}] I, [2013-10-06T07:11:08.688036 #1305] INFO -- : openshift.rb:146:in `execute_action' Finished executing action [app-state-show] (0) I, [2013-10-06T07:11:08.720790 #1305] INFO -- : openshift.rb:201:in `execute_parallel_action' execute_parallel_action call - [{:tag=>"", :gear=>"524dee49e0b8cdf32e000001", :job=>{:cartridge=>"openshift-origin-node", :action=>"app-state-show", :args=>{"--with-app-uuid"=>"524dee49e0b8cdf32e000001", "--with-app-name"=>"chkexsrv1", "--with-container-uuid"=>"524dee49e0b8cdf32e000001", "--with-container-name"=>"chkexsrv1", "--with-namespace"=>"openshiftnagios", "--with-uid"=>6479, "--with-request-id"=>"969099581a9a5708db6d39f4e71712af"}}, :result_stdout=>"started", :result_stderr=>"", :result_exit_code=>0}] I, [2013-10-06T07:11:36.537698 #1305] INFO -- : openshift.rb:91:in `cartridge_do_action' cartridge_do_action call / action: cartridge_do, agent=openshift, data={:cartridge=>"openshift-origin-node", :action=>"app-destroy", :args=> {"--with-app-uuid"=>"524dee49e0b8cdf32e000001", "--with-app-name"=>"chkexsrv1", "--with-container-uuid"=>"524dee49e0b8cdf32e000001", "--with-container-name"=>"chkexsrv1", "--with-namespace"=>"openshiftnagios", "--with-uid"=>6479, "--with-request-id"=>"62a618a77651bbbb2d37f52e63c84b71", "--cart-name"=>"openshift-origin-node"}, :process_results=>true} I, [2013-10-06T07:11:36.537931 #1305] INFO -- : openshift.rb:92:in `cartridge_do_action' cartridge_do_action validation = openshift-origin-node app-destroy {"--with-app-uuid"=>"524dee49e0b8cdf32e000001", "--with-app-name"=>"chkexsrv1", "--with-container-uuid"=>"524dee49e0b8cdf32e000001", "--with-container-name"=>"chkexsrv1", "--with-namespace"=>"openshiftnagios", "--with-uid"=>6479, "--with-request-id"=>"62a618a77651bbbb2d37f52e63c84b71", "--cart-name"=>"openshift-origin-node"} I, [2013-10-06T07:11:36.538262 #1305] INFO -- : openshift.rb:131:in `execute_action' Executing action [app-destroy] using method oo_app_destroy with args [{"--with-app-uuid"=>"524dee49e0b8cdf32e000001", "--with-app-name"=>"chkexsrv1", "--with-container-uuid"=>"524dee49e0b8cdf32e000001", "--with-container-name"=>"chkexsrv1", "--with-namespace"=>"openshiftnagios", "--with-uid"=>6479, "--with-request-id"=>"62a618a77651bbbb2d37f52e63c84b71", "--cart-name"=>"openshift-origin-node"}] I, [2013-10-06T07:11:36.558267 #1305] INFO -- : openshift.rb:330:in `rescue in oo_app_destroy' '' is not a legal cartridge identifier I, [2013-10-06T07:11:36.558551 #1305] INFO -- : openshift.rb:331:in `rescue in oo_app_destroy' ["/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-common-1.14.1/lib/openshift-origin-common/models/manifest.rb:322:in `parse_ident'", "/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.14.7/lib/openshift-origin-node/model/v2_cart_model.rb:164:in `get_cartridge_from_directory'", "/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.14.7/lib/openshift-origin-node/model/v2_cart_model.rb:77:in `block in each_cartridge'", "/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.14.7/lib/openshift-origin-node/model/v2_cart_model.rb:975:in `block in process_cartridges'", "/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.14.7/lib/openshift-origin-node/model/v2_cart_model.rb:973:in `each'", "/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.14.7/lib/openshift-origin-node/model/v2_cart_model.rb:973:in `process_cartridges'", "/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.14.7/lib/openshift-origin-node/model/v2_cart_model.rb:76:in `each_cartridge'", "/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.14.7/lib/openshift-origin-node/model/application_container.rb:237:in `block in destroy'", "/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-common-1.14.1/lib/openshift-origin-common/utils/file_needs_sync.rb:38:in `block in open'", "/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-common-1.14.1/lib/openshift-origin-common/utils/file_needs_sync.rb:36:in `open'", "/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-common-1.14.1/lib/openshift-origin-common/utils/file_needs_sync.rb:36:in `open'", "/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.14.7/lib/openshift-origin-node/model/application_container.rb:233:in `destroy'", "/opt/rh/ruby193/root/usr/libexec/mcollective/mcollective/agent/openshift.rb:328:in `oo_app_destroy'", "/opt/rh/ruby193/root/usr/libexec/mcollective/mcollective/agent/openshift.rb:136:in `execute_action'", "/opt/rh/ruby193/root/usr/libexec/mcollective/mcollective/agent/openshift.rb:103:in `cartridge_do_action'", "/opt/rh/ruby193/root/usr/share/ruby/mcollective/rpc/agent.rb:86:in `handlemsg'", "/opt/rh/ruby193/root/usr/share/ruby/mcollective/agents.rb:126:in `block (2 levels) in dispatch'", "/opt/rh/ruby193/root/usr/share/ruby/timeout.rb:69:in `timeout'", "/opt/rh/ruby193/root/usr/share/ruby/mcollective/agents.rb:125:in `block in dispatch'"] I, [2013-10-06T07:11:36.558681 #1305] INFO -- : openshift.rb:146:in `execute_action' Finished executing action [app-destroy] (-1) I, [2013-10-06T07:11:36.581485 #1305] INFO -- : openshift.rb:112:in `cartridge_do_action' cartridge_do_action failed (-1) ------ '' is not a legal cartridge identifier ------) Version-Release number of selected component (if applicable): Current How reproducible: I don't know. Steps to Reproduce: 1. 2. 3. Actual results: Calling destroy fails. Expected results: Should succeed and remove the app. Additional info:
'mcollective' was restarted in the middle of configure. Strange part is that the new process sent the response back to what was initiated by the previous process. Half of what was logged in the active queue, but with exitcode '0'. This is a behaviour issue with mcollective.
*** Bug 1015960 has been marked as a duplicate of this bug. ***