Bug 1022985 - custom cartridge broke my account
custom cartridge broke my account
Status: CLOSED CURRENTRELEASE
Product: OpenShift Online
Classification: Red Hat
Component: Kubernetes (Show other bugs)
2.x
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Rajat Chopra
libra bugs
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-10-24 08:03 EDT by Marek Mahut
Modified: 2015-05-14 20:22 EDT (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2014-01-23 22:26:34 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Marek Mahut 2013-10-24 08:03:54 EDT
Description of problem:

I've been playing with a customer cartridge on my online account and something went wrong. Not sure how and if it is reproducible on a fresh account, but when I try to do something with my current account I'm getting this error:

$ rhc apps --debug
DEBUG: Using config file /home/mmahut/.openshift/express.conf
DEBUG: Authenticating with RHC::Auth::Token
DEBUG: Connecting to https://openshift.redhat.com/broker/rest/api
DEBUG: Client supports API versions 1.1, 1.2, 1.3, 1.4, 1.5
DEBUG: Using token authentication
DEBUG: Created new httpclient
DEBUG: Request GET https://openshift.redhat.com/broker/rest/api
DEBUG:    code 200  578 ms
DEBUG: Server supports API versions 1.0, 1.1, 1.2, 1.3, 1.4, 1.5, 1.6
DEBUG:    Using API version 1.5
DEBUG: Client API version 1.5 is not current. Refetching API
DEBUG: Using token authentication
DEBUG: Request GET https://openshift.redhat.com/broker/rest/api
DEBUG:    code 200  173 ms
DEBUG: Using token authentication
DEBUG: Request GET https://openshift.redhat.com/broker/rest/applications?include=cartridges
DEBUG:    code 500  232 ms
Unable to complete the requested operation due to: The application 'datadog' requires 'mmahut-datadog-1.0' but a matching cartridge could not be found.
Reference ID: ab8323607b857251cda5f85c247217fb
$


Logs from production are saying:

RESULT=SUCCESS STATUS=unknown TIMESTAMP=1382615936 DATE=2013-10-24 TIME=07:58:56 ACTION=AUTHENTICATE REQ_ID=ab8323607b857251cda5f85c247217fb USER_ID=511a7781f2cb83b780000698 LOGIN=mmahut IP=209.132.186.34 SCOPES=session Authenticated
RESULT=FAILURE STATUS=internal_server_error TIMESTAMP=1382615936 DATE=2013-10-24 TIME=07:58:56 ACTION=LIST_APPLICATION REQ_ID=ab8323607b857251cda5f85c247217fb USER_ID=511a7781f2cb83b780000698 LOGIN=mmahut DOMAIN= Unable to complete the requested operation due to: The application 'datadog' requires 'mmahut-datadog-1.0' but a matching cartridge could not be found.

And then:

2013-10-24 07:58:56.588 [ERROR] Reference ID: ab8323607b857251cda5f85c247217fb - The application 'datadog' requires 'mmahut-datadog-1.0' but a matching cartridge could not be found
  /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-controller-1.15.16/app/helpers/cartridge_cache.rb:83:in `find_cartridge_or_raise_exception'
  /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-controller-1.15.16/app/rest_models/rest_application15.rb:81:in `block in initialize'
  /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-controller-1.15.16/app/rest_models/rest_application15.rb:80:in `each'
  /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-controller-1.15.16/app/rest_models/rest_application15.rb:80:in `initialize'
  /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-controller-1.15.16/app/helpers/rest_model_helper.rb:35:in `new'
  /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-controller-1.15.16/app/helpers/rest_model_helper.rb:35:in `get_rest_application'
  /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-controller-1.15.16/app/controllers/applications_controller.rb:21:in `block in index'
  /opt/rh/ruby193/root/usr/share/gems/gems/mongoid-3.0.21/lib/mongoid/contextual/mongo.rb:577:in `yield_document'
  /opt/rh/ruby193/root/usr/share/gems/gems/mongoid-3.0.21/lib/mongoid/contextual/mongo.rb:133:in `block (2 levels) in each'
  /opt/rh/ruby193/root/usr/share/gems/gems/mongoid-3.0.21/lib/mongoid/contextual/mongo.rb:132:in `each'
  /opt/rh/ruby193/root/usr/share/gems/gems/mongoid-3.0.21/lib/mongoid/contextual/mongo.rb:132:in `block in each'
  /opt/rh/ruby193/root/usr/share/gems/gems/mongoid-3.0.21/lib/mongoid/contextual/mongo.rb:556:in `selecting'
  /opt/rh/ruby193/root/usr/share/gems/gems/mongoid-3.0.21/lib/mongoid/contextual/mongo.rb:131:in `each'
  /opt/rh/ruby193/root/usr/share/gems/gems/mongoid-3.0.21/lib/mongoid/contextual.rb:18:in `each'
  /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-controller-1.15.16/app/controllers/applications_controller.rb:21:in `map'
  /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-controller-1.15.16/app/controllers/applications_controller.rb:21:in `index'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_controller/metal/implicit_render.rb:4:in `send_action'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/abstract_controller/base.rb:167:in `process_action'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_controller/metal/rendering.rb:10:in `process_action'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/abstract_controller/callbacks.rb:18:in `block in process_action'
  /opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:470:in `block in _run__4337711153245528513__process_action__369112843280187972__callbacks'
  /opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:215:in `block in _conditional_callback_around_397'
  /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-controller-1.15.16/lib/openshift/controller/action_log.rb:80:in `set_logged_request'
  /opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:214:in `_conditional_callback_around_397'
  /opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:403:in `_run__4337711153245528513__process_action__369112843280187972__callbacks'
  /opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:405:in `__run_callback'
  /opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:385:in `_run_process_action_callbacks'
  /opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:81:in `run_callbacks'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/abstract_controller/callbacks.rb:17:in `process_action'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_controller/metal/rescue.rb:29:in `process_action'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_controller/metal/instrumentation.rb:30:in `block in process_action'
  /opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/notifications.rb:123:in `block in instrument'
  /opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
  /opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/notifications.rb:123:in `instrument'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_controller/metal/instrumentation.rb:29:in `process_action'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_controller/metal/params_wrapper.rb:207:in `process_action'
  /opt/rh/ruby193/root/usr/share/gems/gems/newrelic_rpm-3.6.6.147/lib/new_relic/agent/instrumentation/rails3/action_controller.rb:38:in `block in process_action'
  /opt/rh/ruby193/root/usr/share/gems/gems/newrelic_rpm-3.6.6.147/lib/new_relic/agent/instrumentation/controller_instrumentation.rb:318:in `perform_action_with_newrelic_trace'
  /opt/rh/ruby193/root/usr/share/gems/gems/newrelic_rpm-3.6.6.147/lib/new_relic/agent/instrumentation/rails3/action_controller.rb:37:in `process_action'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/abstract_controller/base.rb:121:in `process'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/abstract_controller/rendering.rb:45:in `process'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_controller/metal.rb:203:in `dispatch'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_controller/metal/rack_delegation.rb:14:in `dispatch'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_controller/metal.rb:246:in `block in action'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_dispatch/routing/route_set.rb:73:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_dispatch/routing/route_set.rb:73:in `dispatch'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_dispatch/routing/route_set.rb:36:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/journey-1.0.4/lib/journey/router.rb:68:in `block in call'
  /opt/rh/ruby193/root/usr/share/gems/gems/journey-1.0.4/lib/journey/router.rb:56:in `each'
  /opt/rh/ruby193/root/usr/share/gems/gems/journey-1.0.4/lib/journey/router.rb:56:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_dispatch/routing/route_set.rb:600:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/newrelic_rpm-3.6.6.147/lib/new_relic/rack/error_collector.rb:43:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/newrelic_rpm-3.6.6.147/lib/new_relic/rack/agent_hooks.rb:22:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/newrelic_rpm-3.6.6.147/lib/new_relic/rack/browser_monitoring.rb:16:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/mongo_mapper-0.11.1/lib/mongo_mapper/middleware/identity_map.rb:10:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/mongoid-3.0.21/lib/rack/mongoid/middleware/identity_map.rb:34:in `block in call'
  /opt/rh/ruby193/root/usr/share/gems/gems/mongoid-3.0.21/lib/mongoid/unit_of_work.rb:39:in `unit_of_work'
  /opt/rh/ruby193/root/usr/share/gems/gems/mongoid-3.0.21/lib/rack/mongoid/middleware/identity_map.rb:34:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_dispatch/middleware/best_standards_support.rb:17:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/rack-1.4.1/lib/rack/etag.rb:23:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/rack-1.4.1/lib/rack/conditionalget.rb:25:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_dispatch/middleware/head.rb:14:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_dispatch/middleware/params_parser.rb:21:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_dispatch/middleware/flash.rb:242:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/rack-1.4.1/lib/rack/session/abstract/id.rb:205:in `context'
  /opt/rh/ruby193/root/usr/share/gems/gems/rack-1.4.1/lib/rack/session/abstract/id.rb:200:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_dispatch/middleware/cookies.rb:339:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_dispatch/middleware/callbacks.rb:28:in `block in call'
  /opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:405:in `_run__1231030782650992792__call__272527061529642639__callbacks'
  /opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:405:in `__run_callback'
  /opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:385:in `_run_call_callbacks'
  /opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/callbacks.rb:81:in `run_callbacks'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_dispatch/middleware/callbacks.rb:27:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_dispatch/middleware/remote_ip.rb:31:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_dispatch/middleware/debug_exceptions.rb:16:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_dispatch/middleware/show_exceptions.rb:56:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/railties-3.2.8/lib/rails/rack/logger.rb:26:in `call_app'
  /opt/rh/ruby193/root/usr/share/gems/gems/railties-3.2.8/lib/rails/rack/logger.rb:16:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_dispatch/middleware/request_id.rb:22:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/rack-1.4.1/lib/rack/methodoverride.rb:21:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/rack-1.4.1/lib/rack/runtime.rb:17:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/activesupport-3.2.8/lib/active_support/cache/strategy/local_cache.rb:72:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/rack-1.4.1/lib/rack/lock.rb:15:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/actionpack-3.2.8/lib/action_dispatch/middleware/static.rb:62:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/rack-cache-1.2/lib/rack/cache/context.rb:136:in `forward'
  /opt/rh/ruby193/root/usr/share/gems/gems/rack-cache-1.2/lib/rack/cache/context.rb:245:in `fetch'
  /opt/rh/ruby193/root/usr/share/gems/gems/rack-cache-1.2/lib/rack/cache/context.rb:185:in `lookup'
  /opt/rh/ruby193/root/usr/share/gems/gems/rack-cache-1.2/lib/rack/cache/context.rb:66:in `call!'
  /opt/rh/ruby193/root/usr/share/gems/gems/rack-cache-1.2/lib/rack/cache/context.rb:51:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/railties-3.2.8/lib/rails/engine.rb:479:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/railties-3.2.8/lib/rails/application.rb:223:in `call'
  /opt/rh/ruby193/root/usr/share/gems/gems/railties-3.2.8/lib/rails/railtie/configurable.rb:30:in `method_missing'
  /opt/rh/ruby193/root/usr/share/gems/gems/passenger-3.0.21/lib/phusion_passenger/rack/request_handler.rb:97:in `process_request'
  /opt/rh/ruby193/root/usr/share/gems/gems/passenger-3.0.21/lib/phusion_passenger/abstract_request_handler.rb:521:in `accept_and_process_next_request'
  /opt/rh/ruby193/root/usr/share/gems/gems/passenger-3.0.21/lib/phusion_passenger/abstract_request_handler.rb:274:in `main_loop'
  /opt/rh/ruby193/root/usr/share/gems/gems/passenger-3.0.21/lib/phusion_passenger/rack/application_spawner.rb:206:in `start_request_handler'
  /opt/rh/ruby193/root/usr/share/gems/gems/passenger-3.0.21/lib/phusion_passenger/rack/application_spawner.rb:79:in `block in spawn_application'
  /opt/rh/ruby193/root/usr/share/gems/gems/passenger-3.0.21/lib/phusion_passenger/utils.rb:470:in `safe_fork'
  /opt/rh/ruby193/root/usr/share/gems/gems/passenger-3.0.21/lib/phusion_passenger/rack/application_spawner.rb:64:in `spawn_application'
  /opt/rh/ruby193/root/usr/share/gems/gems/passenger-3.0.21/lib/phusion_passenger/spawn_manager.rb:264:in `spawn_rack_application'
  /opt/rh/ruby193/root/usr/share/gems/gems/passenger-3.0.21/lib/phusion_passenger/spawn_manager.rb:137:in `spawn_application'
  /opt/rh/ruby193/root/usr/share/gems/gems/passenger-3.0.21/lib/phusion_passenger/spawn_manager.rb:275:in `handle_spawn_application'
  /opt/rh/ruby193/root/usr/share/gems/gems/passenger-3.0.21/lib/phusion_passenger/abstract_server.rb:357:in `server_main_loop'
  /opt/rh/ruby193/root/usr/share/gems/gems/passenger-3.0.21/lib/phusion_passenger/abstract_server.rb:206:in `start_synchronously'
  /opt/rh/ruby193/root/usr/share/gems/gems/passenger-3.0.21/helper-scripts/passenger-spawn-server:102:in `<main>' (pid:21325)
Comment 1 Marek Mahut 2013-10-24 08:23:31 EDT
Odd, I see it on the broker but cannot find it by uuid on the mongo hosts.

================================================================================

      Login:         mmahut
      Plan:          free (ACTIVE)

      App Name:      datadog
      App UUID:      5268fc05e0b8cdad4700020f
      Creation Time: 2013-10-24 10:52:53 AM
      URL:           http://datadog-mmahut.rhcloud.com
    
      Group Instance[0]:
          Components: 
              Cartridge Name: diy-0.1
              Component Name: diy-0.1 
           Gear[0]
              Server Identity: ex-std-node264.prod.rhcloud.com
              Gear UUID:       5268fc05e0b8cdad4700020f
              Gear UID:        4725
          
      Current DNS
      -----------
     datadog-mmahut.rhcloud.com is an alias for ex-std-node264.prod.rhcloud.com.
     ex-std-node264.prod.rhcloud.com is an alias for ec2-174-129-54-118.compute-1.amazonaws.com.
     ec2-174-129-54-118.compute-1.amazonaws.com has address 10.118.57.234
    
================================================================================


libra:PRIMARY> db.applications.find( { "uuid" : "5268fc05e0b8cdad4700020f" } ).pretty()
libra:PRIMARY>
Comment 2 Rajat Chopra 2013-10-25 17:17:39 EDT
Got the application's dump from mongo using 
db.applications.find( { "_id" : ObjectId("5268fc05e0b8cdad4700020f")}).pretty()


I imported this into my devenv's mongo and I could not see any problem. The mongo dump also looks quite alright and nothing seems to mention about the downloadable cartridge.
Attached is the mongo dump.

Seeking that the problem be rechecked if it still exists.
Comment 4 Marek Mahut 2013-10-26 12:30:10 EDT
This is working now. Any idea what could have been the problem?
Comment 5 Xiaoli Tian 2013-10-28 06:24:30 EDT
Move it to verified according to comment 4

Note You need to log in before you can comment on or make changes to this bug.