Bug 985921 - "rhc domain show" errors with "The server did not respond correctly."
"rhc domain show" errors with "The server did not respond correctly."
Status: CLOSED DUPLICATE of bug 988255
Product: OpenShift Online
Classification: Red Hat
Component: Kubernetes (Show other bugs)
2.x
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Lili Nader
libra bugs
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-07-18 09:52 EDT by Thomas Wiest
Modified: 2015-05-14 20:19 EDT (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-07-29 13:18:20 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Thomas Wiest 2013-07-18 09:52:54 EDT
Description of problem:
Our monitoring in INT and STG is calling "rhc domain show" which is erroring with this:
$ /usr/bin/rhc domain show -d -k --timeout 185 -p "$pass"
DEBUG: Using config file /home/nagios_monitor/.openshift/express.conf
DEBUG: Authenticating with RHC::Auth::Basic
DEBUG: Connecting to https://localhost/broker/rest/api
DEBUG: Getting all domains
DEBUG: Client supports API versions 1.1, 1.2, 1.3, 1.4, 1.5
DEBUG: Request GET https://localhost/broker/rest/api
DEBUG: SSL Verification failed -- Using self signed cert
DEBUG:    code 200  110 ms
DEBUG: Server supports API versions 1.0, 1.1, 1.2, 1.3, 1.4, 1.5
DEBUG: Request GET https://localhost/broker/rest/domains
DEBUG:    code 200  598 ms
DEBUG: Getting all applications for domain nagiosmonitor
DEBUG: Request GET https://localhost/broker/rest/domains/nagiosmonitor/applications?include=cartridges
DEBUG:    code 500  590 ms
DEBUG: Response did not include a message from server: invalid character at "<!DOCTYPE "
The server did not respond correctly. This may be an issue with the server configuration or with your connection to
the server (such as a Web proxy or firewall). Please verify that you can access the OpenShift server
https://localhost/broker/rest/domains/nagiosmonitor/applications




The broker production.log shows this:
2013-07-18 09:44:57.964 [INFO ] Started GET "/broker/rest/api" for 127.0.0.1 at 2013-07-18 09:44:57 -0400 (pid:29171)
2013-07-18 09:44:57.965 [INFO ] Processing by ApiController#show as JSON (pid:29171)
2013-07-18 09:44:57.985 [INFO ] Completed 200 OK in 20ms (Views: 16.8ms) (pid:29171)
2013-07-18 09:44:58.471 [INFO ] Started GET "/broker/rest/domains" for 127.0.0.1 at 2013-07-18 09:44:58 -0400 (pid:29171)
2013-07-18 09:44:58.481 [INFO ] Processing by DomainsController#index as JSON (pid:29171)
2013-07-18 09:44:58.624 [DEBUG] Response from Streamline took (/wapps/streamline/login.html): 575.072771 ms (pid:29171)
2013-07-18 09:44:58.624 [DEBUG] POST Response code = 200 (pid:29171)
2013-07-18 09:44:58.624 [DEBUG] Current login = REDACTED / authenticated for REDACTED (pid:29171)
2013-07-18 09:44:58.627 [DEBUG] MOPED: 10.70.145.129:27017 QUERY        database=openshift_broker collection=cloud_users selector={"$query"=>{"login"=>"REDACTED"}, "$orderby"=>{:_id=>1}} flags=[] limit=-1 skip=0 batch_size=nil fields=nil (2.3167ms) (pid:29171)
2013-07-18 09:44:58.628 [DEBUG] Getting domains for user REDACTED (pid:29171)
2013-07-18 09:44:58.630 [DEBUG] MOPED: 10.70.145.129:27017 QUERY        database=openshift_broker collection=domains selector={"owner_id"=>"51142f66dbd93ce16a000e28"} flags=[] limit=0 skip=0 batch_size=nil fields=nil (1.9188ms) (pid:29171)
2013-07-18 09:44:58.639 [INFO ] Completed 200 OK in 591ms (Views: 2.8ms) (pid:29171)
2013-07-18 09:44:58.681 [INFO ] Started GET "/broker/rest/domains/nagiosmonitor/applications?include=cartridges" for 127.0.0.1 at 2013-07-18 09:44:58 -0400 (pid:29171)
2013-07-18 09:44:58.683 [INFO ] Processing by ApplicationsController#index as JSON (pid:29171)
2013-07-18 09:44:58.683 [INFO ] Parameters: {"include"=>"cartridges", "domain_id"=>"nagiosmonitor"} (pid:29171)
2013-07-18 09:44:59.229 [DEBUG] Response from Streamline took (/wapps/streamline/login.html): 545.22655 ms (pid:29171)
2013-07-18 09:44:59.229 [DEBUG] POST Response code = 200 (pid:29171)
2013-07-18 09:44:59.229 [DEBUG] Current login = REDACTED / authenticated for REDACTED (pid:29171)
2013-07-18 09:44:59.232 [DEBUG] MOPED: 10.70.145.129:27017 QUERY        database=openshift_broker collection=cloud_users selector={"$query"=>{"login"=>"REDACTED"}, "$orderby"=>{:_id=>1}} flags=[] limit=-1 skip=0 batch_size=nil fields=nil (2.5609ms) (pid:29171)
2013-07-18 09:44:59.235 [DEBUG] MOPED: 10.70.145.129:27017 QUERY        database=openshift_broker collection=domains selector={"$query"=>{"owner_id"=>"51142f66dbd93ce16a000e28", "canonical_namespace"=>"nagiosmonitor"}, "$orderby"=>{:_id=>1}} flags=[] limit=-1 skip=0 batch_size=nil fields=nil (1.9128ms) (pid:29171)
2013-07-18 09:44:59.248 [DEBUG] MOPED: 10.70.145.129:27017 QUERY        database=openshift_broker collection=applications selector={"domain_id"=>"51142f66dbd93ce16a000e2c"} flags=[] limit=0 skip=0 batch_size=nil fields=nil (12.6989ms) (pid:29171)
2013-07-18 09:44:59.260 [INFO ] Completed 500 Internal Server Error in 577ms (pid:29171)
2013-07-18 09:44:59.265 [FATAL] NoMethodError (undefined method `categories' for nil:NilClass):
  openshift-origin-controller (1.11.8) app/rest_models/rest_application.rb:82:in `block in initialize'
  openshift-origin-controller (1.11.8) app/rest_models/rest_application.rb:80:in `each'
  openshift-origin-controller (1.11.8) app/rest_models/rest_application.rb:80:in `initialize'
  openshift-origin-controller (1.11.8) app/helpers/rest_model_helper.rb:31:in `new'
  openshift-origin-controller (1.11.8) app/helpers/rest_model_helper.rb:31:in `get_rest_application'
  openshift-origin-controller (1.11.8) app/controllers/applications_controller.rb:19:in `block in index'
  mongoid (3.0.21) lib/mongoid/relations/targets/enumerable.rb:169:in `block in each'
  mongoid (3.0.21) lib/mongoid/contextual/mongo.rb:577:in `yield_document'
  mongoid (3.0.21) lib/mongoid/contextual/mongo.rb:133:in `block (2 levels) in each'
  moped (1.3.2) lib/moped/query.rb:78:in `block in each'
  moped (1.3.2) lib/moped/cursor.rb:26:in `block in each'
  moped (1.3.2) lib/moped/cursor.rb:26:in `each'
  moped (1.3.2) lib/moped/cursor.rb:26:in `each'
  moped (1.3.2) lib/moped/query.rb:77:in `each'
  moped (1.3.2) lib/moped/query.rb:77:in `each'
  mongoid (3.0.21) lib/mongoid/contextual/mongo.rb:132:in `block in each'
  mongoid (3.0.21) lib/mongoid/contextual/mongo.rb:556:in `selecting'
  mongoid (3.0.21) lib/mongoid/contextual/mongo.rb:131:in `each'
  mongoid (3.0.21) lib/mongoid/contextual.rb:18:in `each'
  mongoid (3.0.21) lib/mongoid/relations/targets/enumerable.rb:166:in `each'
  mongoid (3.0.21) lib/mongoid/relations/referenced/many.rb:395:in `map'
  mongoid (3.0.21) lib/mongoid/relations/referenced/many.rb:395:in `method_missing'
  openshift-origin-controller (1.11.8) app/controllers/applications_controller.rb:19:in `index'
  actionpack (3.2.8) lib/action_controller/metal/implicit_render.rb:4:in `send_action'
  actionpack (3.2.8) lib/abstract_controller/base.rb:167:in `process_action'
  actionpack (3.2.8) lib/action_controller/metal/rendering.rb:10:in `process_action'
  actionpack (3.2.8) lib/abstract_controller/callbacks.rb:18:in `block in process_action'
  activesupport (3.2.8) lib/active_support/callbacks.rb:481:in `block in _run__2475025031870956118__process_action__2004240631298587419__callbacks'
  activesupport (3.2.8) lib/active_support/callbacks.rb:215:in `block in _conditional_callback_around_359'
  openshift-origin-controller (1.11.8) lib/openshift/controller/action_log.rb:34:in `set_logged_request'
  activesupport (3.2.8) lib/active_support/callbacks.rb:214:in `_conditional_callback_around_359'
  activesupport (3.2.8) lib/active_support/callbacks.rb:403:in `_run__2475025031870956118__process_action__2004240631298587419__callbacks'
  activesupport (3.2.8) lib/active_support/callbacks.rb:405:in `__run_callback'
  activesupport (3.2.8) lib/active_support/callbacks.rb:385:in `_run_process_action_callbacks'
  activesupport (3.2.8) lib/active_support/callbacks.rb:81:in `run_callbacks'
  actionpack (3.2.8) lib/abstract_controller/callbacks.rb:17:in `process_action'
  actionpack (3.2.8) lib/action_controller/metal/rescue.rb:29:in `process_action'
  actionpack (3.2.8) lib/action_controller/metal/instrumentation.rb:30:in `block in process_action'
  activesupport (3.2.8) lib/active_support/notifications.rb:123:in `block in instrument'
  activesupport (3.2.8) lib/active_support/notifications/instrumenter.rb:20:in `instrument'
  activesupport (3.2.8) lib/active_support/notifications.rb:123:in `instrument'
  actionpack (3.2.8) lib/action_controller/metal/instrumentation.rb:29:in `process_action'
  actionpack (3.2.8) lib/action_controller/metal/params_wrapper.rb:207:in `process_action'
  actionpack (3.2.8) lib/abstract_controller/base.rb:121:in `process'
  actionpack (3.2.8) lib/abstract_controller/rendering.rb:45:in `process'
  actionpack (3.2.8) lib/action_controller/metal.rb:203:in `dispatch'
  actionpack (3.2.8) lib/action_controller/metal/rack_delegation.rb:14:in `dispatch'
  actionpack (3.2.8) lib/action_controller/metal.rb:246:in `block in action'
  actionpack (3.2.8) lib/action_dispatch/routing/route_set.rb:73:in `call'
  actionpack (3.2.8) lib/action_dispatch/routing/route_set.rb:73:in `dispatch'
  actionpack (3.2.8) lib/action_dispatch/routing/route_set.rb:36:in `call'
  journey (1.0.4) lib/journey/router.rb:68:in `block in call'
  journey (1.0.4) lib/journey/router.rb:56:in `each'
  journey (1.0.4) lib/journey/router.rb:56:in `call'
  actionpack (3.2.8) lib/action_dispatch/routing/route_set.rb:600:in `call'
  mongo_mapper (0.11.1) lib/mongo_mapper/middleware/identity_map.rb:10:in `call'
  mongoid (3.0.21) lib/rack/mongoid/middleware/identity_map.rb:34:in `block in call'
  mongoid (3.0.21) lib/mongoid/unit_of_work.rb:39:in `unit_of_work'
  mongoid (3.0.21) lib/rack/mongoid/middleware/identity_map.rb:34:in `call'
  actionpack (3.2.8) lib/action_dispatch/middleware/best_standards_support.rb:17:in `call'
  rack (1.4.1) lib/rack/etag.rb:23:in `call'
  rack (1.4.1) lib/rack/conditionalget.rb:25:in `call'
  actionpack (3.2.8) lib/action_dispatch/middleware/head.rb:14:in `call'
  actionpack (3.2.8) lib/action_dispatch/middleware/params_parser.rb:21:in `call'
  actionpack (3.2.8) lib/action_dispatch/middleware/flash.rb:242:in `call'
  rack (1.4.1) lib/rack/session/abstract/id.rb:205:in `context'
  rack (1.4.1) lib/rack/session/abstract/id.rb:200:in `call'
  actionpack (3.2.8) lib/action_dispatch/middleware/cookies.rb:339:in `call'
  actionpack (3.2.8) lib/action_dispatch/middleware/callbacks.rb:28:in `block in call'
  activesupport (3.2.8) lib/active_support/callbacks.rb:405:in `_run__724529480576098724__call__1495730240980561763__callbacks'
  activesupport (3.2.8) lib/active_support/callbacks.rb:405:in `__run_callback'
  activesupport (3.2.8) lib/active_support/callbacks.rb:385:in `_run_call_callbacks'
  activesupport (3.2.8) lib/active_support/callbacks.rb:81:in `run_callbacks'
  actionpack (3.2.8) lib/action_dispatch/middleware/callbacks.rb:27:in `call'
  actionpack (3.2.8) lib/action_dispatch/middleware/remote_ip.rb:31:in `call'
  actionpack (3.2.8) lib/action_dispatch/middleware/debug_exceptions.rb:16:in `call'
  actionpack (3.2.8) lib/action_dispatch/middleware/show_exceptions.rb:56:in `call'
  railties (3.2.8) lib/rails/rack/logger.rb:26:in `call_app'
  railties (3.2.8) lib/rails/rack/logger.rb:16:in `call'
  actionpack (3.2.8) lib/action_dispatch/middleware/request_id.rb:22:in `call'
  rack (1.4.1) lib/rack/methodoverride.rb:21:in `call'
  rack (1.4.1) lib/rack/runtime.rb:17:in `call'
  activesupport (3.2.8) lib/active_support/cache/strategy/local_cache.rb:72:in `call'
  rack (1.4.1) lib/rack/lock.rb:15:in `call'
  rack-cache (1.2) lib/rack/cache/context.rb:136:in `forward'
  rack-cache (1.2) lib/rack/cache/context.rb:245:in `fetch'
  rack-cache (1.2) lib/rack/cache/context.rb:185:in `lookup'
  rack-cache (1.2) lib/rack/cache/context.rb:66:in `call!'
  rack-cache (1.2) lib/rack/cache/context.rb:51:in `call'
  railties (3.2.8) lib/rails/engine.rb:479:in `call'
  railties (3.2.8) lib/rails/application.rb:223:in `call'
  railties (3.2.8) lib/rails/railtie/configurable.rb:30:in `method_missing'
  passenger (3.0.17) lib/phusion_passenger/rack/request_handler.rb:96:in `process_request'
  passenger (3.0.17) lib/phusion_passenger/abstract_request_handler.rb:517:in `accept_and_process_next_request'
  passenger (3.0.17) lib/phusion_passenger/abstract_request_handler.rb:274:in `main_loop'
  passenger (3.0.17) lib/phusion_passenger/rack/application_spawner.rb:206:in `start_request_handler'
  passenger (3.0.17) lib/phusion_passenger/rack/application_spawner.rb:79:in `block in spawn_application'
  passenger (3.0.17) lib/phusion_passenger/utils.rb:470:in `safe_fork'
  passenger (3.0.17) lib/phusion_passenger/rack/application_spawner.rb:64:in `spawn_application'
  passenger (3.0.17) lib/phusion_passenger/spawn_manager.rb:264:in `spawn_rack_application'
  passenger (3.0.17) lib/phusion_passenger/spawn_manager.rb:137:in `spawn_application'
  passenger (3.0.17) lib/phusion_passenger/spawn_manager.rb:275:in `handle_spawn_application'
  passenger (3.0.17) lib/phusion_passenger/abstract_server.rb:357:in `server_main_loop'
  passenger (3.0.17) lib/phusion_passenger/abstract_server.rb:206:in `start_synchronously'
  passenger (3.0.17) helper-scripts/passenger-spawn-server:102:in `<main>' (pid:29171)




Version-Release number of selected component (if applicable):
rhc-broker-1.11.6-1.el6oso.noarch


How reproducible:
Very with the account we're using for monitoring.


Steps to Reproduce:
1. Unknown, just started happening


Actual results:
Errors with "The server did not respond correctly."


Expected results:
Should show our domain.
Comment 1 Xiaoli Tian 2013-07-18 10:27:32 EDT
It starts to happen for my account as well:
[test@dhcp-9-116 ~]$ rhc domain show -l xtian+int1@redhat.com
The server did not respond correctly. This may be an issue with the server
configuration or with your connection to the server (such as a Web proxy or
firewall). Please verify that you can access the OpenShift server
https://stg.openshift.redhat.com/broker/rest/domains/xtian1/applications

Which still worked this afternoon, not sure if it's related to the security updates which was happening several hours ago
Comment 2 Dan McPherson 2013-07-18 10:40:14 EDT
Even if this is a config issue we need to fix the error message.
Comment 3 Lili Nader 2013-07-29 13:18:20 EDT

*** This bug has been marked as a duplicate of bug 988255 ***

Note You need to log in before you can comment on or make changes to this bug.