Bug 806115

Summary: Site unit tests failing because broker has 500 on deletion
Product: OKD Reporter: Clayton Coleman <ccoleman>
Component: PodAssignee: Rob Millner <rmillner>
Status: CLOSED CURRENTRELEASE QA Contact: libra bugs <libra-bugs>
Severity: high Docs Contact:
Priority: unspecified    
Version: 1.xCC: mfisher, rchopra, rmillner, yujzhang
Target Milestone: ---Keywords: Triaged
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-04-13 18:34:10 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Clayton Coleman 2012-03-22 22:50:40 UTC 
Site tests are failing because we can't delete the domain, which fails cause it can't delete an app.

libra_check development log has it

https://ci.dev.openshift.redhat.com/jenkins/job/libra_check/1477/artifact/rhc/log/development.log

Started DELETE "/broker/rest/domains/1332451436d1ba55.json?force=true" for 127.0.0.1 at Thu Mar 22 17:52:57 -0400 2012
  Processing by DomainsController#destroy as JSON
  Parameters: {"id"=>"1332451436d1ba55", "force"=>"true"}
MongoDataStore.find(CloudUser, app_test1, app_test1)

DEBUG: rpc_exec: rpc_client=#<MCollective::RPC::Client:0x7f7ac6938b70>
Node execution failure (invalid exit code from node).  If the problem persists please contact Red Hat support.
#<StickShift::NodeException: Node execution failure (invalid exit code from node).  If the problem persists please contact Red Hat support.>
["/var/www/stickshift/broker/lib/express/broker/application_container_proxy.rb:890:in `run_cartridge_command'", "/var/www/stickshift/broker/lib/express/broker/application_container_proxy.rb:684:in `remove_component'", "/var/www/stickshift/broker/lib/express/broker/application_container_proxy.rb:202:in `deconfigure_cartridge'", "/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.7.1/lib/stickshift-controller/app/models/gear.rb:62:in `deconfigure'", "/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.7.1/lib/stickshift-controller/app/models/application.rb:506:in `deconfigure_dependencies'", "/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.7.1/lib/stickshift-controller/app/models/application.rb:1305:in `call'", "/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.7.1/lib/stickshift-controller/app/models/application.rb:1305:in `run_on_gears'", "/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.7.1/lib/stickshift-controller/app/models/application.rb:1303:in `each'", "/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.7.1/lib/stickshift-controller/app/models/application.rb:1303:in `run_on_gears'", "/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.7.1/lib/stickshift-controller/app/models/application.rb:505:in `deconfigure_dependencies'", "/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.7.1/lib/stickshift-controller/app/models/application.rb:500:in `each'", "/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.7.1/lib/stickshift-controller/app/models/application.rb:500:in `deconfigure_dependencies'", "/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.7.1/lib/stickshift-controller/app/models/application.rb:233:in `cleanup_and_delete'", "/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.7.1/lib/stickshift-controller/app/controllers/domains_controller.rb:178:in `destroy'", "/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.7.1/lib/stickshift-controller/app/controllers/domains_controller.rb:177:in `each'", "/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.7.1/lib/stickshift-controller/app/controllers/domains_controller.rb:177:in `destroy'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal/implicit_render.rb:4:in `send_action'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal/implicit_render.rb:4:in `send_action'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/abstract_controller/base.rb:150:in `process_action'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal/rendering.rb:11:in `process_action'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/abstract_controller/callbacks.rb:18:in `process_action'", "/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/callbacks.rb:436:in `_run__750238963__process_action__2084551337__callbacks'", "/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/callbacks.rb:410:in `send'", "/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/callbacks.rb:410:in `_run_process_action_callbacks'", "/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/callbacks.rb:94:in `send'", "/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/callbacks.rb:94:in `run_callbacks'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/abstract_controller/callbacks.rb:17:in `process_action'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal/rescue.rb:17:in `process_action'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal/instrumentation.rb:30:in `process_action'", "/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/notifications.rb:52:in `instrument'", "/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/notifications/instrumenter.rb:21:in `instrument'", "/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/notifications.rb:52:in `instrument'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal/instrumentation.rb:29:in `process_action'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/abstract_controller/base.rb:119:in `process'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/abstract_controller/rendering.rb:41:in `process'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal.rb:138:in `dispatch'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal/rack_delegation.rb:14:in `dispatch'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_controller/metal.rb:178:in `action'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/routing/route_set.rb:62:in `call'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/routing/route_set.rb:62:in `dispatch'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/routing/route_set.rb:27:in `call'", "/usr/lib/ruby/gems/1.8/gems/rack-mount-0.7.1/lib/rack/mount/route_set.rb:150:in `call'", "/usr/lib/ruby/gems/1.8/gems/rack-mount-0.7.1/lib/rack/mount/code_generation.rb:93:in `recognize'", "/usr/lib/ruby/gems/1.8/gems/rack-mount-0.7.1/lib/rack/mount/code_generation.rb:82:in `optimized_each'", "/usr/lib/ruby/gems/1.8/gems/rack-mount-0.7.1/lib/rack/mount/code_generation.rb:92:in `recognize'", "/usr/lib/ruby/gems/1.8/gems/rack-mount-0.7.1/lib/rack/mount/route_set.rb:141:in `call'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/routing/route_set.rb:493:in `call'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/middleware/best_standards_support.rb:17:in `call'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/middleware/head.rb:14:in `call'", "/usr/lib/ruby/gems/1.8/gems/rack-1.3.0/lib/rack/methodoverride.rb:24:in `call'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/middleware/params_parser.rb:21:in `call'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/middleware/flash.rb:182:in `call'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/middleware/session/abstract_store.rb:149:in `call'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/middleware/cookies.rb:302:in `call'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/middleware/callbacks.rb:46:in `call'", "/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.10/lib/active_support/callbacks.rb:416:in `_run_call_callbacks'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/middleware/callbacks.rb:44:in `call'", "/usr/lib/ruby/gems/1.8/gems/rack-1.3.0/lib/rack/sendfile.rb:102:in `call'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/middleware/remote_ip.rb:48:in `call'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/middleware/show_exceptions.rb:47:in `call'", "/usr/lib/ruby/gems/1.8/gems/railties-3.0.10/lib/rails/rack/logger.rb:13:in `call'", "/usr/lib/ruby/gems/1.8/gems/rack-1.3.0/lib/rack/runtime.rb:17:in `call'", "/usr/lib/ruby/gems/1.8/gems/rack-1.3.0/lib/rack/lock.rb:34:in `call'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.10/lib/action_dispatch/middleware/static.rb:30:in `call'", "/usr/lib/ruby/gems/1.8/gems/railties-3.0.10/lib/rails/application.rb:168:in `call'", "/usr/lib/ruby/gems/1.8/gems/railties-3.0.10/lib/rails/application.rb:77:in `send'", "/usr/lib/ruby/gems/1.8/gems/railties-3.0.10/lib/rails/application.rb:77:in `method_missing'", "/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/rack/request_handler.rb:96:in `process_request'", "/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/abstract_request_handler.rb:514:in `accept_and_process_next_request'", "/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/abstract_request_handler.rb:274:in `main_loop'", "/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/rack/application_spawner.rb:205:in `start_request_handler'", "/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/rack/application_spawner.rb:78:in `spawn_application'", "/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/utils.rb:479:in `safe_fork'", "/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/rack/application_spawner.rb:63:in `spawn_application'", "/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/spawn_manager.rb:264:in `spawn_rack_application'", "/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/spawn_manager.rb:137:in `spawn_application'", "/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/spawn_manager.rb:275:in `handle_spawn_application'", "/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/abstract_server.rb:357:in `__send__'", "/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/abstract_server.rb:357:in `server_main_loop'", "/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/abstract_server.rb:206:in `start_synchronously'", "/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/helper-scripts/passenger-spawn-server:102"]
MongoDataStore.save(Application, app_test1, test1, #hidden)

Completed 500 Internal Server Error in 4982ms

Exception (exceptionNode execution failure (invalid exit code from node).  If the problem persists please contact Red Hat support.successfulfailedexceptionNode execution failure (invalid exit code from node).  If the problem persists please contact Red Hat support.gear#<Gear:0x7f7ac638bb28>):
Comment 1 Rajat Chopra 2012-03-23 02:30:44 UTC 
Fixed with rev# 6202f4538cbb333c465afa32b6e06ea6cb3f0baa
Comment 2 Yujie Zhang 2012-03-26 10:25:49 UTC 
Verify this bug on devenv_1672, it has been fixed now.