Bug 1022985 - custom cartridge broke my account
Summary: custom cartridge broke my account
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: OpenShift Online
Classification: Red Hat
Component: Pod
Version: 2.x
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: ---
Assignee: Rajat Chopra
QA Contact: libra bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-10-24 12:03 UTC by Marek Mahut
Modified: 2015-05-15 00:22 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2014-01-24 03:26:34 UTC
Target Upstream Version:

Attachments (Terms of Use)
Add an attachment (proposed patch, testcase, etc.)

Description Marek Mahut 2013-10-24 12:03:54 UTC 
Description of problem:

I've been playing with a customer cartridge on my online account and something went wrong. Not sure how and if it is reproducible on a fresh account, but when I try to do something with my current account I'm getting this error:

$ rhc apps --debug
DEBUG: Using config file /home/mmahut/.openshift/express.conf
DEBUG: Authenticating with RHC::Auth::Token
DEBUG: Connecting to https://openshift.redhat.com/broker/rest/api
DEBUG: Client supports API versions 1.1, 1.2, 1.3, 1.4, 1.5
DEBUG: Using token authentication
DEBUG: Created new httpclient
DEBUG: Request GET https://openshift.redhat.com/broker/rest/api
DEBUG:    code 200  578 ms
DEBUG: Server supports API versions 1.0, 1.1, 1.2, 1.3, 1.4, 1.5, 1.6
DEBUG:    Using API version 1.5
DEBUG: Client API version 1.5 is not current. Refetching API
DEBUG: Using token authentication
DEBUG: Request GET https://openshift.redhat.com/broker/rest/api
DEBUG:    code 200  173 ms
DEBUG: Using token authentication
DEBUG: Request GET https://openshift.redhat.com/broker/rest/applications?include=cartridges
DEBUG:    code 500  232 ms
Unable to complete the requested operation due to: The application 'datadog' requires 'mmahut-datadog-1.0' but a matching cartridge could not be found.
Reference ID: ab8323607b857251cda5f85c247217fb
$


Logs from production are saying:

RESULT=SUCCESS STATUS=unknown TIMESTAMP=1382615936 DATE=2013-10-24 TIME=07:58:56 ACTION=AUTHENTICATE REQ_ID=ab8323607b857251cda5f85c247217fb USER_ID=511a7781f2cb83b780000698 LOGIN=mmahut IP=209.132.186.34 SCOPES=session Authenticated
RESULT=FAILURE STATUS=internal_server_error TIMESTAMP=1382615936 DATE=2013-10-24 TIME=07:58:56 ACTION=LIST_APPLICATION REQ_ID=ab8323607b857251cda5f85c247217fb USER_ID=511a7781f2cb83b780000698 LOGIN=mmahut DOMAIN= Unable to complete the requested operation due to: The application 'datadog' requires 'mmahut-datadog-1.0' but a matching cartridge could not be found.

And then:

2013-10-24 07:58:56.588 [ERROR] Reference ID: ab8323607b857251cda5f85c247217fb - The application 'datadog' requires 'mmahut-datadog-1.0' but a matching cartridge could not be found
  /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-controller-1.15.16/app/helpers/cartridge_cache.rb:83:in `find_cartridge_or_raise_exception'
  /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-controller-1.15.16/app/rest_models/rest_application15.rb:81:in `block in initialize'
  /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-controller-1.15.16/app/rest_models/rest_application15.rb:80:in `each'
  /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-controller-1.15.16/app/rest_models/rest_application15.rb:80:in `initialize'
  /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-controller-1.15.16/app/helpers/rest_model_helper.rb:35:in `new'
  /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-controller-1.15.16/app/helpers/rest_model_helper.rb:35:in `get_rest_application'
  /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-controller-1.15.16/app/controllers/applications_controller.rb:21:in `block in index'
  /opt/rh/ruby193/root/usr/share/gems/gems/mongoid-3.0.21/lib/mongoid/contextual/mongo.rb:577:in `yield_document'
  /opt/rh/ruby193/root/usr/share/gems/gems/mongoid-3.0.21/lib/mongoid/contextual/mongo.rb:133:in `block (2 levels) in each'
  /opt/rh/ruby193/root/usr/share/gems/gems/mongoid-3.0.21/lib/mongoid/contextual/mongo.rb:132:in `each'
  /opt/rh/ruby193/root/usr/share/gems/gems/mongoid-3.0.21/lib/mongoid/contextual/mongo.rb:132:in `block in each'
  /opt/rh/ruby193/root/usr/share/gems/gems/mongoid-3.0.21/lib/mongoid/contextual/mongo.rb:556:in `selecting'
  /opt/rh/ruby193/root/usr/share/gems/gems/mongoid-3.0.21/lib/mongoid/contextual/mongo.rb:131:in `each'
  /opt/rh/ruby193/root/usr/share/gems/gems/mongoid-3.0.21/lib/mongoid/contextual.rb:18:in `each'
  /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-controller-1.15.16/app/controllers/applications_controller.rb:21:in `map'
  /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-controller-1.15.16/app/controllers/applications_controller.rb:21:in `index'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_controller/metal/implicit_render.rb:4:in `send_action'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/abstract_controller/base.rb:167:in `process_action'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_controller/metal/rendering.rb:10:in `process_action'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/abstract_controller/callbacks.rb:18:in `block in process_action'
  /opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:470:in `block in _run__4337711153245528513__process_action__369112843280187972__callbacks'
  /opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:215:in `block in _conditional_callback_around_397'
  /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-controller-1.15.16/lib/openshift/controller/action_log.rb:80:in `set_logged_request'
  /opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:214:in `_conditional_callback_around_397'
  /opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:403:in `_run__4337711153245528513__process_action__369112843280187972__callbacks'
  /opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:405:in `__run_callback'
  /opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:385:in `_run_process_action_callbacks'
  /opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:81:in `run_callbacks'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/abstract_controller/callbacks.rb:17:in `process_action'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_controller/metal/rescue.rb:29:in `process_action'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_controller/metal/instrumentation.rb:30:in `block in process_action'
  /opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/notifications.rb:123:in `block in instrument'
  /opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
  /opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/notifications.rb:123:in `instrument'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_controller/metal/instrumentation.rb:29:in `process_action'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_controller/metal/params_wrapper.rb:207:in `process_action'
  /opt/rh/ruby193/root/usr/share/gems/gems/newrelic_rpm-3.6.6.147/lib/new_relic/agent/instrumentation/rails3/action_controller.rb:38:in `block in process_action'
  /opt/rh/ruby193/root/usr/share/gems/gems/newrelic_rpm-3.6.6.147/lib/new_relic/agent/instrumentation/controller_instrumentation.rb:318:in `perform_action_with_newrelic_trace'
  /opt/rh/ruby193/root/usr/share/gems/gems/newrelic_rpm-3.6.6.147/lib/new_relic/agent/instrumentation/rails3/action_controller.rb:37:in `process_action'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/abstract_controller/base.rb:121:in `process'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/abstract_controller/rendering.rb:45:in `process'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_controller/metal.rb:203:in `dispatch'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_controller/metal/rack_delegation.rb:14:in `dispatch'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_controller/metal.rb:246:in `block in action'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_dispatch/routing/route_set.rb:73:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_dispatch/routing/route_set.rb:73:in `dispatch'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_dispatch/routing/route_set.rb:36:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/journey-1.0.4/lib/journey/router.rb:68:in `block in call'
  /opt/rh/ruby193/root/usr/share/gems/gems/journey-1.0.4/lib/journey/router.rb:56:in `each'
  /opt/rh/ruby193/root/usr/share/gems/gems/journey-1.0.4/lib/journey/router.rb:56:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_dispatch/routing/route_set.rb:600:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/newrelic_rpm-3.6.6.147/lib/new_relic/rack/error_collector.rb:43:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/newrelic_rpm-3.6.6.147/lib/new_relic/rack/agent_hooks.rb:22:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/newrelic_rpm-3.6.6.147/lib/new_relic/rack/browser_monitoring.rb:16:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/mongo_mapper-0.11.1/lib/mongo_mapper/middleware/identity_map.rb:10:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/mongoid-3.0.21/lib/rack/mongoid/middleware/identity_map.rb:34:in `block in call'
  /opt/rh/ruby193/root/usr/share/gems/gems/mongoid-3.0.21/lib/mongoid/unit_of_work.rb:39:in `unit_of_work'
  /opt/rh/ruby193/root/usr/share/gems/gems/mongoid-3.0.21/lib/rack/mongoid/middleware/identity_map.rb:34:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_dispatch/middleware/best_standards_support.rb:17:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/rack-1.4.1/lib/rack/etag.rb:23:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/rack-1.4.1/lib/rack/conditionalget.rb:25:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_dispatch/middleware/head.rb:14:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_dispatch/middleware/params_parser.rb:21:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_dispatch/middleware/flash.rb:242:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/rack-1.4.1/lib/rack/session/abstract/id.rb:205:in `context'
  /opt/rh/ruby193/root/usr/share/gems/gems/rack-1.4.1/lib/rack/session/abstract/id.rb:200:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_dispatch/middleware/cookies.rb:339:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_dispatch/middleware/callbacks.rb:28:in `block in call'
  /opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:405:in `_run__1231030782650992792__call__272527061529642639__callbacks'
  /opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:405:in `__run_callback'
  /opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:385:in `_run_call_callbacks'
  /opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:81:in `run_callbacks'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_dispatch/middleware/callbacks.rb:27:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_dispatch/middleware/remote_ip.rb:31:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_dispatch/middleware/debug_exceptions.rb:16:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_dispatch/middleware/show_exceptions.rb:56:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/railties-3.2.8/lib/rails/rack/logger.rb:26:in `call_app'
  /opt/rh/ruby193/root/usr/share/gems/gems/railties-3.2.8/lib/rails/rack/logger.rb:16:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_dispatch/middleware/request_id.rb:22:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/rack-1.4.1/lib/rack/methodoverride.rb:21:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/rack-1.4.1/lib/rack/runtime.rb:17:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/cache/strategy/local_cache.rb:72:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/rack-1.4.1/lib/rack/lock.rb:15:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_dispatch/middleware/static.rb:62:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/rack-cache-1.2/lib/rack/cache/context.rb:136:in `forward'
  /opt/rh/ruby193/root/usr/share/gems/gems/rack-cache-1.2/lib/rack/cache/context.rb:245:in `fetch'
  /opt/rh/ruby193/root/usr/share/gems/gems/rack-cache-1.2/lib/rack/cache/context.rb:185:in `lookup'
  /opt/rh/ruby193/root/usr/share/gems/gems/rack-cache-1.2/lib/rack/cache/context.rb:66:in `call!'
  /opt/rh/ruby193/root/usr/share/gems/gems/rack-cache-1.2/lib/rack/cache/context.rb:51:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/railties-3.2.8/lib/rails/engine.rb:479:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/railties-3.2.8/lib/rails/application.rb:223:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/railties-3.2.8/lib/rails/railtie/configurable.rb:30:in `method_missing'
  /opt/rh/ruby193/root/usr/share/gems/gems/passenger-3.0.21/lib/phusion_passenger/rack/request_handler.rb:97:in `process_request'
  /opt/rh/ruby193/root/usr/share/gems/gems/passenger-3.0.21/lib/phusion_passenger/abstract_request_handler.rb:521:in `accept_and_process_next_request'
  /opt/rh/ruby193/root/usr/share/gems/gems/passenger-3.0.21/lib/phusion_passenger/abstract_request_handler.rb:274:in `main_loop'
  /opt/rh/ruby193/root/usr/share/gems/gems/passenger-3.0.21/lib/phusion_passenger/rack/application_spawner.rb:206:in `start_request_handler'
  /opt/rh/ruby193/root/usr/share/gems/gems/passenger-3.0.21/lib/phusion_passenger/rack/application_spawner.rb:79:in `block in spawn_application'
  /opt/rh/ruby193/root/usr/share/gems/gems/passenger-3.0.21/lib/phusion_passenger/utils.rb:470:in `safe_fork'
  /opt/rh/ruby193/root/usr/share/gems/gems/passenger-3.0.21/lib/phusion_passenger/rack/application_spawner.rb:64:in `spawn_application'
  /opt/rh/ruby193/root/usr/share/gems/gems/passenger-3.0.21/lib/phusion_passenger/spawn_manager.rb:264:in `spawn_rack_application'
  /opt/rh/ruby193/root/usr/share/gems/gems/passenger-3.0.21/lib/phusion_passenger/spawn_manager.rb:137:in `spawn_application'
  /opt/rh/ruby193/root/usr/share/gems/gems/passenger-3.0.21/lib/phusion_passenger/spawn_manager.rb:275:in `handle_spawn_application'
  /opt/rh/ruby193/root/usr/share/gems/gems/passenger-3.0.21/lib/phusion_passenger/abstract_server.rb:357:in `server_main_loop'
  /opt/rh/ruby193/root/usr/share/gems/gems/passenger-3.0.21/lib/phusion_passenger/abstract_server.rb:206:in `start_synchronously'
  /opt/rh/ruby193/root/usr/share/gems/gems/passenger-3.0.21/helper-scripts/passenger-spawn-server:102:in `<main>' (pid:21325)
Comment 1 Marek Mahut 2013-10-24 12:23:31 UTC 
Odd, I see it on the broker but cannot find it by uuid on the mongo hosts.

================================================================================

      Login:         mmahut
      Plan:          free (ACTIVE)

      App Name:      datadog
      App UUID:      5268fc05e0b8cdad4700020f
      Creation Time: 2013-10-24 10:52:53 AM
      URL:           http://datadog-mmahut.rhcloud.com
    
      Group Instance[0]:
          Components: 
              Cartridge Name: diy-0.1
              Component Name: diy-0.1 
           Gear[0]
              Server Identity: ex-std-node264.prod.rhcloud.com
              Gear UUID:       5268fc05e0b8cdad4700020f
              Gear UID:        4725
          
      Current DNS
      -----------
     datadog-mmahut.rhcloud.com is an alias for ex-std-node264.prod.rhcloud.com.
     ex-std-node264.prod.rhcloud.com is an alias for ec2-174-129-54-118.compute-1.amazonaws.com.
     ec2-174-129-54-118.compute-1.amazonaws.com has address 10.118.57.234
    
================================================================================


libra:PRIMARY> db.applications.find( { "uuid" : "5268fc05e0b8cdad4700020f" } ).pretty()
libra:PRIMARY>
Comment 2 Rajat Chopra 2013-10-25 21:17:39 UTC 
Got the application's dump from mongo using 
db.applications.find( { "_id" : ObjectId("5268fc05e0b8cdad4700020f")}).pretty()


I imported this into my devenv's mongo and I could not see any problem. The mongo dump also looks quite alright and nothing seems to mention about the downloadable cartridge.
Attached is the mongo dump.

Seeking that the problem be rechecked if it still exists.
Comment 4 Marek Mahut 2013-10-26 16:30:10 UTC 
This is working now. Any idea what could have been the problem?
Comment 5 Xiaoli Tian 2013-10-28 10:24:30 UTC 
Move it to verified according to comment 4
Note You need to log in before you can comment on or make changes to this bug.