Attachment 604666 Details for Bug 848500 – broker log

broker log

broker.log (text/x-log), 34.51 KB, created by Kenny Woodson on 2012-08-15 19:05:17 UTC

(hide)

Description:

Filename:

MIME Type:

Creator: Kenny Woodson

Created: 2012-08-15 19:05:17 UTC

Size: 34.51 KB

patch

obsolete

>Started POST "/broker/cartridge" for 127.0.0.1 at Wed Aug 15 14:00:24 -0400 2012
>  Processing by LegacyBrokerController#cartridge_post as 
>  Parameters: {"password"=>"[FILTERED]", "json_data"=>"{\"node_profile\":\"small\",\"debug\":true,\"cartridge\":\"php-5.3\",\"api\":\"1.1.3\",\"app_name\":\"chkexsrv1\",\"action\":\"configure\",\"rhlogin\":\"mmcgrath+nagios@redhat.com\"}"}
>User-Agent = 'rhc/0.96.9 (ruby 1.8.7; x86_64-linux)'
>Response from Streamline took (/wapps/streamline/login.html): 296.172 ms
>POST Response code = 200
>Current login = mmcgrath+nagios@redhat.com / authenticated for mmcgrath+nagios@redhat.com
>MongoDataStore.find(CloudUser, mmcgrath+nagios@redhat.com, mmcgrath+nagios@redhat.com)
>
>Checking to see if application name is black listed
>Creating application chkexsrv1
>Auto-merging group @@app/cart-php-5.3 into @@app
>Creating gears
>DEBUG: find_available_impl: district_uuid: 48295095e12d4a93b1ae419250580720
>DEBUG: rpc_get_fact: fact=active_capacity
>DEBUG: rpc_exec: rpc_client=#<MCollective::RPC::Client:0x7f5582126bf8>
>Current server: ex-std-node54.prod.rhcloud.com active capacity: 58.75
>CURRENT SERVER: ex-std-node54.prod.rhcloud.com
>DEBUG: find_available_impl: current_server: ex-std-node54.prod.rhcloud.com: 58.75
>MongoDataStore.reserve_district_uid(48295095e12d4a93b1ae419250580720)
>
>MongoDataStore.create(Application, mmcgrath+nagios@redhat.com, chkexsrv1, #hidden)
>
>DEBUG: rpc_exec_direct: rpc_client=#<MCollective::RPC::Client:0x7f5581e98d28>
>DEBUG: rpc_client.custom_request('cartridge_do', {:cartridge=>"stickshift-node", :action=>"app-create", :args=>{"--with-container-name"=>"chkexsrv1", "--with-uid"=>6829, "--with-app-name"=>"chkexsrv1", "--with-namespace"=>"openshiftnagios", "--with-container-uuid"=>"94679084a13a4272ba0de6a2b38490bd", "--with-app-uuid"=>"94679084a13a4272ba0de6a2b38490bd"}}, @id, {'identity' => @id})
>DEBUG: []
>Node execution failure (error getting result from node).  If the problem persists please contact Red Hat support.
>/usr/lib/ruby/gems/1.8/gems/gearchanger-mcollective-plugin-0.1.7/lib/gearchanger-mcollective-plugin/gearchanger/mcollective_application_container_proxy.rb:1054:in `parse_result'
>/usr/lib/ruby/gems/1.8/gems/gearchanger-mcollective-plugin-0.1.7/lib/gearchanger-mcollective-plugin/gearchanger/mcollective_application_container_proxy.rb:183:in `create'
>/usr/lib/ruby/gems/1.8/gems/gearchanger-mcollective-plugin-0.1.7/lib/gearchanger-mcollective-plugin/gearchanger/mcollective_application_container_proxy.rb:172:in `each'
>/usr/lib/ruby/gems/1.8/gems/gearchanger-mcollective-plugin-0.1.7/lib/gearchanger-mcollective-plugin/gearchanger/mcollective_application_container_proxy.rb:172:in `create'
>/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.14.17/lib/stickshift-controller/app/models/gear.rb:34:in `create'
>/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.14.17/lib/stickshift-controller/app/models/group_instance.rb:73:in `add_gear'
>/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.14.17/lib/stickshift-controller/app/models/application.rb:293:in `create'
>/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.14.17/lib/stickshift-controller/app/models/application.rb:292:in `each'
>/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.14.17/lib/stickshift-controller/app/models/application.rb:292:in `create'
>/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.14.17/lib/stickshift-controller/app/controllers/legacy_broker_controller.rb:223:in `cartridge_post'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_controller/metal/implicit_render.rb:4:in `send_action'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_controller/metal/implicit_render.rb:4:in `send_action'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/abstract_controller/base.rb:150:in `process_action'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_controller/metal/rendering.rb:11:in `process_action'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/abstract_controller/callbacks.rb:18:in `process_action'
>/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.13/lib/active_support/callbacks.rb:461:in `_run__1961107264__process_action__1366870675__callbacks'
>/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.13/lib/active_support/callbacks.rb:410:in `send'
>/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.13/lib/active_support/callbacks.rb:410:in `_run_process_action_callbacks'
>/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.13/lib/active_support/callbacks.rb:94:in `send'
>/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.13/lib/active_support/callbacks.rb:94:in `run_callbacks'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/abstract_controller/callbacks.rb:17:in `process_action'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_controller/metal/rescue.rb:17:in `process_action'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_controller/metal/instrumentation.rb:30:in `process_action'
>/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.13/lib/active_support/notifications.rb:52:in `instrument'
>/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.13/lib/active_support/notifications/instrumenter.rb:21:in `instrument'
>/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.13/lib/active_support/notifications.rb:52:in `instrument'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_controller/metal/instrumentation.rb:29:in `process_action'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/abstract_controller/base.rb:119:in `process'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/abstract_controller/rendering.rb:41:in `process'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_controller/metal.rb:138:in `dispatch'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_controller/metal/rack_delegation.rb:14:in `dispatch'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_controller/metal.rb:178:in `action'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_dispatch/routing/route_set.rb:68:in `call'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_dispatch/routing/route_set.rb:68:in `dispatch'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_dispatch/routing/route_set.rb:33:in `call'
>/usr/lib/ruby/gems/1.8/gems/rack-mount-0.7.1/lib/rack/mount/route_set.rb:150:in `call'
>/usr/lib/ruby/gems/1.8/gems/rack-mount-0.7.1/lib/rack/mount/code_generation.rb:93:in `recognize'
>/usr/lib/ruby/gems/1.8/gems/rack-mount-0.7.1/lib/rack/mount/code_generation.rb:68:in `optimized_each'
>/usr/lib/ruby/gems/1.8/gems/rack-mount-0.7.1/lib/rack/mount/code_generation.rb:92:in `recognize'
>/usr/lib/ruby/gems/1.8/gems/rack-mount-0.7.1/lib/rack/mount/route_set.rb:141:in `call'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_dispatch/routing/route_set.rb:499:in `call'
>/usr/lib/ruby/gems/1.8/gems/mongo_mapper-0.11.1/lib/mongo_mapper/middleware/identity_map.rb:10:in `call'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_dispatch/middleware/best_standards_support.rb:17:in `call'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_dispatch/middleware/head.rb:14:in `call'
>/usr/lib/ruby/gems/1.8/gems/rack-1.3.0/lib/rack/methodoverride.rb:24:in `call'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_dispatch/middleware/params_parser.rb:21:in `call'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_dispatch/middleware/flash.rb:182:in `call'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_dispatch/middleware/session/abstract_store.rb:149:in `call'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_dispatch/middleware/cookies.rb:302:in `call'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_dispatch/middleware/callbacks.rb:46:in `call'
>/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.13/lib/active_support/callbacks.rb:416:in `_run_call_callbacks'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_dispatch/middleware/callbacks.rb:44:in `call'
>/usr/lib/ruby/gems/1.8/gems/rack-1.3.0/lib/rack/sendfile.rb:102:in `call'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_dispatch/middleware/remote_ip.rb:48:in `call'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_dispatch/middleware/show_exceptions.rb:47:in `call'
>/usr/lib/ruby/gems/1.8/gems/railties-3.0.13/lib/rails/rack/logger.rb:13:in `call'
>/usr/lib/ruby/gems/1.8/gems/rack-1.3.0/lib/rack/runtime.rb:17:in `call'
>/usr/lib/ruby/gems/1.8/gems/rack-1.3.0/lib/rack/lock.rb:34:in `call'
>/usr/lib/ruby/gems/1.8/gems/railties-3.0.13/lib/rails/application.rb:168:in `call'
>/usr/lib/ruby/gems/1.8/gems/railties-3.0.13/lib/rails/application.rb:77:in `send'
>/usr/lib/ruby/gems/1.8/gems/railties-3.0.13/lib/rails/application.rb:77:in `method_missing'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/rack/request_handler.rb:96:in `process_request'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/abstract_request_handler.rb:514:in `accept_and_process_next_request'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/abstract_request_handler.rb:274:in `main_loop'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/classic_rails/application_spawner.rb:321:in `start_request_handler'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/classic_rails/application_spawner.rb:275:in `send'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/classic_rails/application_spawner.rb:275:in `handle_spawn_application'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/utils.rb:479:in `safe_fork'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/classic_rails/application_spawner.rb:270:in `handle_spawn_application'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/abstract_server.rb:357:in `__send__'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/abstract_server.rb:357:in `server_main_loop'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/abstract_server.rb:206:in `start_synchronously'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/abstract_server.rb:180:in `start'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/classic_rails/application_spawner.rb:149:in `start'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/spawn_manager.rb:219:in `spawn_rails_application'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/abstract_server_collection.rb:132:in `lookup_or_add'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/spawn_manager.rb:214:in `spawn_rails_application'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/abstract_server_collection.rb:82:in `synchronize'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/abstract_server_collection.rb:79:in `synchronize'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/spawn_manager.rb:213:in `spawn_rails_application'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/spawn_manager.rb:132:in `spawn_application'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/spawn_manager.rb:275:in `handle_spawn_application'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/abstract_server.rb:357:in `__send__'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/abstract_server.rb:357:in `server_main_loop'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/abstract_server.rb:206:in `start_synchronously'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/helper-scripts/passenger-spawn-server:102
>DEBUG: rpc_exec_direct: rpc_client=#<MCollective::RPC::Client:0x7f5581cab7e0>
>DEBUG: rpc_client.custom_request('cartridge_do', {:cartridge=>"stickshift-node", :action=>"app-destroy", :args=>{"--with-container-name"=>"chkexsrv1", "--with-app-name"=>"chkexsrv1", "--with-namespace"=>"openshiftnagios", "--with-container-uuid"=>"94679084a13a4272ba0de6a2b38490bd", "--with-app-uuid"=>"94679084a13a4272ba0de6a2b38490bd"}}, @id, {'identity' => @id})
>MongoDataStore.save(Application, mmcgrath+nagios@redhat.com, chkexsrv1, #hidden)
>
>Unable to create gear on node
>/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.14.17/lib/stickshift-controller/app/models/gear.rb:53:in `create'
>/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.14.17/lib/stickshift-controller/app/models/group_instance.rb:73:in `add_gear'
>/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.14.17/lib/stickshift-controller/app/models/application.rb:293:in `create'
>/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.14.17/lib/stickshift-controller/app/models/application.rb:292:in `each'
>/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.14.17/lib/stickshift-controller/app/models/application.rb:292:in `create'
>/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.14.17/lib/stickshift-controller/app/controllers/legacy_broker_controller.rb:223:in `cartridge_post'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_controller/metal/implicit_render.rb:4:in `send_action'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_controller/metal/implicit_render.rb:4:in `send_action'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/abstract_controller/base.rb:150:in `process_action'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_controller/metal/rendering.rb:11:in `process_action'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/abstract_controller/callbacks.rb:18:in `process_action'
>/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.13/lib/active_support/callbacks.rb:461:in `_run__1961107264__process_action__1366870675__callbacks'
>/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.13/lib/active_support/callbacks.rb:410:in `send'
>/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.13/lib/active_support/callbacks.rb:410:in `_run_process_action_callbacks'
>/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.13/lib/active_support/callbacks.rb:94:in `send'
>/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.13/lib/active_support/callbacks.rb:94:in `run_callbacks'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/abstract_controller/callbacks.rb:17:in `process_action'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_controller/metal/rescue.rb:17:in `process_action'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_controller/metal/instrumentation.rb:30:in `process_action'
>/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.13/lib/active_support/notifications.rb:52:in `instrument'
>/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.13/lib/active_support/notifications/instrumenter.rb:21:in `instrument'
>/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.13/lib/active_support/notifications.rb:52:in `instrument'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_controller/metal/instrumentation.rb:29:in `process_action'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/abstract_controller/base.rb:119:in `process'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/abstract_controller/rendering.rb:41:in `process'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_controller/metal.rb:138:in `dispatch'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_controller/metal/rack_delegation.rb:14:in `dispatch'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_controller/metal.rb:178:in `action'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_dispatch/routing/route_set.rb:68:in `call'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_dispatch/routing/route_set.rb:68:in `dispatch'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_dispatch/routing/route_set.rb:33:in `call'
>/usr/lib/ruby/gems/1.8/gems/rack-mount-0.7.1/lib/rack/mount/route_set.rb:150:in `call'
>/usr/lib/ruby/gems/1.8/gems/rack-mount-0.7.1/lib/rack/mount/code_generation.rb:93:in `recognize'
>/usr/lib/ruby/gems/1.8/gems/rack-mount-0.7.1/lib/rack/mount/code_generation.rb:68:in `optimized_each'
>/usr/lib/ruby/gems/1.8/gems/rack-mount-0.7.1/lib/rack/mount/code_generation.rb:92:in `recognize'
>/usr/lib/ruby/gems/1.8/gems/rack-mount-0.7.1/lib/rack/mount/route_set.rb:141:in `call'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_dispatch/routing/route_set.rb:499:in `call'
>/usr/lib/ruby/gems/1.8/gems/mongo_mapper-0.11.1/lib/mongo_mapper/middleware/identity_map.rb:10:in `call'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_dispatch/middleware/best_standards_support.rb:17:in `call'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_dispatch/middleware/head.rb:14:in `call'
>/usr/lib/ruby/gems/1.8/gems/rack-1.3.0/lib/rack/methodoverride.rb:24:in `call'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_dispatch/middleware/params_parser.rb:21:in `call'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_dispatch/middleware/flash.rb:182:in `call'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_dispatch/middleware/session/abstract_store.rb:149:in `call'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_dispatch/middleware/cookies.rb:302:in `call'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_dispatch/middleware/callbacks.rb:46:in `call'
>/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.13/lib/active_support/callbacks.rb:416:in `_run_call_callbacks'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_dispatch/middleware/callbacks.rb:44:in `call'
>/usr/lib/ruby/gems/1.8/gems/rack-1.3.0/lib/rack/sendfile.rb:102:in `call'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_dispatch/middleware/remote_ip.rb:48:in `call'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_dispatch/middleware/show_exceptions.rb:47:in `call'
>/usr/lib/ruby/gems/1.8/gems/railties-3.0.13/lib/rails/rack/logger.rb:13:in `call'
>/usr/lib/ruby/gems/1.8/gems/rack-1.3.0/lib/rack/runtime.rb:17:in `call'
>/usr/lib/ruby/gems/1.8/gems/rack-1.3.0/lib/rack/lock.rb:34:in `call'
>/usr/lib/ruby/gems/1.8/gems/railties-3.0.13/lib/rails/application.rb:168:in `call'
>/usr/lib/ruby/gems/1.8/gems/railties-3.0.13/lib/rails/application.rb:77:in `send'
>/usr/lib/ruby/gems/1.8/gems/railties-3.0.13/lib/rails/application.rb:77:in `method_missing'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/rack/request_handler.rb:96:in `process_request'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/abstract_request_handler.rb:514:in `accept_and_process_next_request'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/abstract_request_handler.rb:274:in `main_loop'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/classic_rails/application_spawner.rb:321:in `start_request_handler'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/classic_rails/application_spawner.rb:275:in `send'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/classic_rails/application_spawner.rb:275:in `handle_spawn_application'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/utils.rb:479:in `safe_fork'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/classic_rails/application_spawner.rb:270:in `handle_spawn_application'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/abstract_server.rb:357:in `__send__'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/abstract_server.rb:357:in `server_main_loop'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/abstract_server.rb:206:in `start_synchronously'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/abstract_server.rb:180:in `start'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/classic_rails/application_spawner.rb:149:in `start'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/spawn_manager.rb:219:in `spawn_rails_application'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/abstract_server_collection.rb:132:in `lookup_or_add'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/spawn_manager.rb:214:in `spawn_rails_application'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/abstract_server_collection.rb:82:in `synchronize'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/abstract_server_collection.rb:79:in `synchronize'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/spawn_manager.rb:213:in `spawn_rails_application'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/spawn_manager.rb:132:in `spawn_application'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/spawn_manager.rb:275:in `handle_spawn_application'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/abstract_server.rb:357:in `__send__'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/abstract_server.rb:357:in `server_main_loop'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/lib/phusion_passenger/abstract_server.rb:206:in `start_synchronously'
>/usr/lib/ruby/gems/1.8/gems/passenger-3.0.4/helper-scripts/passenger-spawn-server:102
>Rolling back application gear creation
>MongoDataStore.save(Application, mmcgrath+nagios@redhat.com, chkexsrv1, #hidden)
>
>MongoDataStore.save(Application, mmcgrath+nagios@redhat.com, chkexsrv1, #hidden)
>
>[REQ_ID=81a06ab95b8d48828c3f672c6cf9ad04] ACTION=LEGACY_CREATE_APP Failed to create application chkexsrv1: Unable to create gear on node
>MongoDataStore.save(Application, mmcgrath+nagios@redhat.com, chkexsrv1, #hidden)
>
>MongoDataStore.delete(Application, mmcgrath+nagios@redhat.com, chkexsrv1)
>
>[REQ_ID=81a06ab95b8d48828c3f672c6cf9ad04] ACTION=LEGACY_BROKER StickShift::NodeException for /broker/cartridge: Unable to create gear on node
>/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.14.17/lib/stickshift-controller/app/models/gear.rb:53:in `create'
>/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.14.17/lib/stickshift-controller/app/models/group_instance.rb:73:in `add_gear'
>/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.14.17/lib/stickshift-controller/app/models/application.rb:293:in `create'
>/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.14.17/lib/stickshift-controller/app/models/application.rb:292:in `each'
>/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.14.17/lib/stickshift-controller/app/models/application.rb:292:in `create'
>/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.14.17/lib/stickshift-controller/app/controllers/legacy_broker_controller.rb:223:in `cartridge_post'
>--DEBUG--
>
>--RESULT--
>
>--MESSAGE--
>
>--ERROR--
>Node execution failure (error getting result from node).  If the problem persists please contact Red Hat support.
>--APP INFO--
>
>--CART COMMANDS--
>
>--CART PROPERTIES--
>{}
>--DATA--
>
>--EXIT CODE--
>5
>Completed 500 Internal Server Error in 339242ms (Views: 1.8ms)
>
>
>
>Started POST "/broker/cartridge" for 127.0.0.1 at Wed Aug 15 14:03:41 -0400 2012
>  Processing by LegacyBrokerController#cartridge_post as 
>  Parameters: {"password"=>"[FILTERED]", "json_data"=>"{\"app_name\":\"chkexsrv1\",\"action\":\"deconfigure\",\"debug\":true,\"api\":\"1.1.3\",\"rhlogin\":\"mmcgrath+nagios@redhat.com\"}"}
>User-Agent = 'rhc/0.96.9 (ruby 1.8.7; x86_64-linux)'
>Response from Streamline took (/wapps/streamline/login.html): 288.956 ms
>POST Response code = 200
>Current login = mmcgrath+nagios@redhat.com / authenticated for mmcgrath+nagios@redhat.com
>MongoDataStore.find(CloudUser, mmcgrath+nagios@redhat.com, mmcgrath+nagios@redhat.com)
>
>DEBUG: DYNECT Login with path: /REST/Session/
>DEBUG: DYNECT Response: {"status": "success", "data": {"token": "gIan2rRK6jEKmQ4wMx+NtUxltxnsPB2lvqvbu8cODvTtrPS4At5uHOKSGYsYeCnPVBrXK5a7ZLWCFXbC4jVbPJkLpMDoLxXPelb4dZZkoTwvQYXQoidVbv/Qoqnxlt5gY5z3uPBOOz2KhztP56A4grZyxfiPala1", "version": "3.3.5"}, "job_id": 157404829, "msgs": [{"INFO": "login: Login successful", "SOURCE": "BLL", "ERR_CD": null, "LVL": "INFO"}]}
>DEBUG: DYNECT Response data: tokengIan2rRK6jEKmQ4wMx+NtUxltxnsPB2lvqvbu8cODvTtrPS4At5uHOKSGYsYeCnPVBrXK5a7ZLWCFXbC4jVbPJkLpMDoLxXPelb4dZZkoTwvQYXQoidVbv/Qoqnxlt5gY5z3uPBOOz2KhztP56A4grZyxfiPala1version3.3.5
>DEBUG: DYNECT delete with path: /REST/CNAMERecord/rhcloud.com/chkexsrv1-openshiftnagios.rhcloud.com/ and headers: {"Auth-Token"=>"[hidden]", "Content-Type"=>"application/json"}
>DEBUG: DYNECT: Could not find /REST/CNAMERecord/rhcloud.com/chkexsrv1-openshiftnagios.rhcloud.com/ to delete
>DEBUG: DYNECT put/post with path: /REST/Zone/rhcloud.com/ json data: {"publish":"true"} and headers: {"Auth-Token"=>"[hidden]", "Content-Type"=>"application/json"}
>DEBUG: DYNECT Response: {"status": "success", "data": {"zone_type": "Primary", "serial_style": "increment", "serial": 1806587, "zone": "rhcloud.com"}, "job_id": 157404835, "msgs": [{"INFO": "changeset: No changes to apply.", "SOURCE": "BLL", "ERR_CD": null, "LVL": "INFO"}, {"INFO": "publish: Could not publish rhcloud.com. No changes to apply.", "SOURCE": "BLL", "ERR_CD": null, "LVL": "INFO"}]}
>DEBUG: DYNECT Response data: zonerhcloud.comserial1806587zone_typePrimaryserial_styleincrement
>DEBUG: DYNECT delete with path: /REST/Session/ and headers: {"Auth-Token"=>"[hidden]", "Content-Type"=>"application/json"}
>DEBUG: DYNECT Response: {"status": "success", "data": {}, "job_id": 157404851, "msgs": [{"INFO": "logout: Logout successful", "SOURCE": "BLL", "ERR_CD": null, "LVL": "INFO"}]}
>DEBUG: DYNECT Response data: 
>DEBUG: rpc_exec_direct: rpc_client=#<MCollective::RPC::Client:0x7f5581cc98d0>
>DEBUG: rpc_client.custom_request('cartridge_do', {:cartridge=>"stickshift-node", :action=>"app-destroy", :args=>{"--with-container-name"=>"chkexsrv1", "--with-app-name"=>"chkexsrv1", "--with-namespace"=>"openshiftnagios", "--with-container-uuid"=>"94679084a13a4272ba0de6a2b38490bd", "--with-app-uuid"=>"94679084a13a4272ba0de6a2b38490bd"}}, @id, {'identity' => @id})
>DEBUG: []
>Node execution failure (error getting result from node).  If the problem persists please contact Red Hat support.
>#<StickShift::NodeException: Node execution failure (error getting result from node).  If the problem persists please contact Red Hat support.>
>["/usr/lib/ruby/gems/1.8/gems/gearchanger-mcollective-plugin-0.1.7/lib/gearchanger-mcollective-plugin/gearchanger/mcollective_application_container_proxy.rb:1054:in `parse_result'", "/usr/lib/ruby/gems/1.8/gems/gearchanger-mcollective-plugin-0.1.7/lib/gearchanger-mcollective-plugin/gearchanger/mcollective_application_container_proxy.rb:205:in `destroy'", "/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.14.17/lib/stickshift-controller/app/models/gear.rb:60:in `destroy'", "/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.14.17/lib/stickshift-controller/app/models/group_instance.rb:91:in `remove_gear'", "/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.14.17/lib/stickshift-controller/app/models/application.rb:342:in `destroy'", "/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.14.17/lib/stickshift-controller/app/models/application.rb:1514:in `call'", "/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.14.17/lib/stickshift-controller/app/models/application.rb:1514:in `run_on_gears'", "/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.14.17/lib/stickshift-controller/app/models/application.rb:1512:in `each'", "/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.14.17/lib/stickshift-controller/app/models/application.rb:1512:in `run_on_gears'", "/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.14.17/lib/stickshift-controller/app/models/application.rb:341:in `destroy'", "/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.14.17/lib/stickshift-controller/app/models/application.rb:340:in `each'", "/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.14.17/lib/stickshift-controller/app/models/application.rb:340:in `destroy'", "/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.14.17/lib/stickshift-controller/app/models/application.rb:317:in `cleanup_and_delete'", "/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.14.17/lib/stickshift-controller/app/controllers/legacy_broker_controller.rb:265:in `cartridge_post'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_controller/metal/implicit_render.rb:4:in `send_action'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_controller/metal/implicit_render.rb:4:in `send_action'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/abstract_controller/base.rb:150:in `process_action'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_controller/metal/rendering.rb:11:in `process_action'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/abstract_controller/callbacks.rb:18:in `process_action'", "/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.13/lib/active_support/callbacks.rb:461:in `_run__1961107264__process_action__1366870675__callbacks'", "/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.13/lib/active_support/callbacks.rb:410:in `send'", "/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.13/lib/active_support/callbacks.rb:410:in `_run_process_action_callbacks'", "/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.13/lib/active_support/callbacks.rb:94:in `send'", "/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.13/lib/active_support/callbacks.rb:94:in `run_callbacks'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/abstract_controller/callbacks.rb:17:in `process_action'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_controller/metal/rescue.rb:17:in `process_action'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_controller/metal/instrumentation.rb:30:in `process_action'", "/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.13/lib/active_support/notifications.rb:52:in `instrument'", "/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.13/lib/active_support/notifications/instrumenter.rb:21:in `instrument'", "/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.13/lib/active_support/notifications.rb:52:in `instrument'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_controller/metal/instrumentation.rb:29:in `process_action'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/abstract_controller/base.rb:119:in `process'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/abstract_controller/rendering.rb:41:in `process'", "/usr/lib/ruby/gem
>MongoDataStore.save(Application, mmcgrath+nagios@redhat.com, chkexsrv1, #hidden)
>
>Unable to clean up application on gear #<Gear:0x7f5581cfccf8> due to exception Node execution failure (error getting result from node).  If the problem persists please contact Red Hat support.
>["/usr/lib/ruby/gems/1.8/gems/gearchanger-mcollective-plugin-0.1.7/lib/gearchanger-mcollective-plugin/gearchanger/mcollective_application_container_proxy.rb:1054:in `parse_result'", "/usr/lib/ruby/gems/1.8/gems/gearchanger-mcollective-plugin-0.1.7/lib/gearchanger-mcollective-plugin/gearchanger/mcollective_application_container_proxy.rb:205:in `destroy'", "/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.14.17/lib/stickshift-controller/app/models/gear.rb:60:in `destroy'", "/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.14.17/lib/stickshift-controller/app/models/group_instance.rb:91:in `remove_gear'", "/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.14.17/lib/stickshift-controller/app/models/application.rb:342:in `destroy'", "/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.14.17/lib/stickshift-controller/app/models/application.rb:1514:in `call'", "/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.14.17/lib/stickshift-controller/app/models/application.rb:1514:in `run_on_gears'", "/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.14.17/lib/stickshift-controller/app/models/application.rb:1512:in `each'", "/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.14.17/lib/stickshift-controller/app/models/application.rb:1512:in `run_on_gears'", "/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.14.17/lib/stickshift-controller/app/models/application.rb:341:in `destroy'", "/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.14.17/lib/stickshift-controller/app/models/application.rb:340:in `each'", "/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.14.17/lib/stickshift-controller/app/models/application.rb:340:in `destroy'", "/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.14.17/lib/stickshift-controller/app/models/application.rb:317:in `cleanup_and_delete'", "/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.14.17/lib/stickshift-controller/app/controllers/legacy_broker_controller.rb:265:in `cartridge_post'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_controller/metal/implicit_render.rb:4:in `send_action'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_controller/metal/implicit_render.rb:4:in `send_action'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/abstract_controller/base.rb:150:in `process_action'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_controller/metal/rendering.rb:11:in `process_action'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/abstract_controller/callbacks.rb:18:in `process_action'", "/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.13/lib/active_support/callbacks.rb:461:in `_run__1961107264__process_action__1366870675__callbacks'", "/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.13/lib/active_support/callbacks.rb:410:in `send'", "/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.13/lib/active_support/callbacks.rb:410:in `_run_process_action_callbacks'", "/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.13/lib/active_support/callbacks.rb:94:in `send'", "/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.13/lib/active_support/callbacks.rb:94:in `run_callbacks'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/abstract_controller/callbacks.rb:17:in `process_action'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_controller/metal/rescue.rb:17:in `process_action'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_controller/metal/instrumentation.rb:30:in `process_action'", "/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.13/lib/active_support/notifications.rb:52:in `instrument'", "/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.13/lib/active_support/notifications/instrumenter.rb:21:in `instrument'", "/usr/lib/ruby/gems/1.8/gems/activesupport-3.0.13/lib/active_support/notifications.rb:52:in `instrument'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_controller/metal/instrumentation.rb:29:in `process_action'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/abstract_controller/base.rb:119:in `process'", "/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/abstract_controller/rendering.rb:41:in `process'", "/usr/lib/ruby/gem
>[REQ_ID=7f9592e0c91c402aa8f7a1ea9546985c] ACTION=LEGACY_BROKER StickShift::NodeException for /broker/cartridge: Could not destroy all gears of application.
>/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.14.17/lib/stickshift-controller/app/models/application.rb:358:in `destroy'
>/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.14.17/lib/stickshift-controller/app/models/application.rb:317:in `cleanup_and_delete'
>/usr/lib/ruby/gems/1.8/gems/stickshift-controller-0.14.17/lib/stickshift-controller/app/controllers/legacy_broker_controller.rb:265:in `cartridge_post'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_controller/metal/implicit_render.rb:4:in `send_action'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_controller/metal/implicit_render.rb:4:in `send_action'
>/usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/abstract_controller/base.rb:150:in `process_action'
>--DEBUG--
>
>--RESULT--
>
>--MESSAGE--
>
>--ERROR--
>
>--APP INFO--
>
>--CART COMMANDS--
>
>--CART PROPERTIES--
>{}
>--DATA--
>
>--EXIT CODE--
>0
>Completed 500 Internal Server Error in 184145ms (Views: 2.9ms)
>

Actions: View

Attachments on bug 848500: 604666 | 604667 | 605781