Red Hat Bugzilla – Bug 864921
Exception is seen upon creating domain when no cartridge is installed in node.
Last modified: 2017-03-08 12:34 EST
Description of problem: There is two vms, one is broker, another is node. On broker, install openshift-origin-broker, rubygem-openshift-origin-msg-broker-mcollective, rubygem-openshift-origin-auth-remote-user and rubygem-openshift-origin-dns-bind. On node, install rubygem-openshift-origin-node and openshift-origin-msg-node-mcollective. Then create domain via rest api. # curl -k -X POST -H 'Accept: application/xml' --data-urlencode id=jialiu --user jialiu@redhat.com:passwd https://10.66.11.161/broker/rest/domains The following error is seen: <--snip--> <title>Action Controller: Exception caught</title> <--snip--> <h1> JSON::ParserError in DomainsController#create </h1> <pre>A JSON text must at least contain two octets!</pre> <--snip--> Check development.log, found broker was trying to get cartridge list when create domain for the first time. # tailf /var/www/openshift/broker/log/development.log Started POST "/broker/rest/domains" for 10.66.65.182 at Wed Oct 10 19:37:35 +0800 2012 Processing by DomainsController#create as XML Parameters: {"id"=>"jialiu"} MongoDataStore.find(CloudUser, jialiu@redhat.com, jialiu@redhat.com) Creating domain with namespace jialiu Checking to see if namesspace jialiu is available Creating domain 5cd519f81a934f75abd18b46430fa79b with namespace jialiu for user jialiu@redhat.com Attempting to add namespace 'jialiu' notifying the domain observer of domain create done notifying the domain observer MongoDataStore.create(Domain, jialiu@redhat.com, 5cd519f81a934f75abd18b46430fa79b, #hidden) CURRENT SERVER: node0 DEBUG: find_one_impl: current_server: node0 DEBUG: rpc_exec_direct: rpc_client=#<MCollective::RPC::Client:0x7fb550e198f0> DEBUG: rpc_client.custom_request('cartridge_do', {:args=>{"--with-descriptors"=>true, "--porcelain"=>true}, :cartridge=>"openshift-origin-node", :action=>"cartridge-list"}, @id, {'identity' => @id}) DEBUG: server results: No such file or directory - /usr/libexec/openshift/cartridges Completed 500 Internal Server Error in 5325ms JSON::ParserError (A JSON text must at least contain two octets!): Rendered /usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_dispatch/middleware/templates/rescues/_trace.erb (1.4ms) Rendered /usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_dispatch/middleware/templates/rescues/_request_and_response.erb (5.1ms) Rendered /usr/lib/ruby/gems/1.8/gems/actionpack-3.0.13/lib/action_dispatch/middleware/templates/rescues/diagnostics.erb within rescues/layout (11.2ms) Actually domain is created successfully, I think it is not necessary to get cartridge list when creating domain. Version-Release number of selected component (if applicable): http://buildvm-devops.usersys.redhat.com/puddle/build/OpenShiftEnterprise/Beta/2012-10-09.4/DevOps/x86_64/os/ Following https://openshift.redhat.com/community/wiki/build-your-own to setup onpremise env using the above repo How reproducible: Always Steps to Reproduce: 1. 2. 3. Actual results: Expected results: During creating domain, it should retrieve cartridge list, they should be separated. Additional info:
We're investigating this sprint how to have rhc properly handle 500 errors from the server.
The actual problem reported has gone away at some point. When the node has an empty cart list (or none), that's what's reported via the API. You can still get an error in the same way, though, by having no nodes, or none that respond. The error bubbles up when doing a domain operation, including rhc setup (because a cartridge list is requested for reporting to the user; I don't think that's a problem). rhc is being modified to improve its handling, but I still think it's improper for the broker to give a 500 error just because it can't reach nodes. Particularly during the early stages of an install, before the user has tried to do anything requiring a node. Working on the fix.
The exception is now caught and just logged, with an empty cartridge list returned. The empty list isn't cached, so once any node shows up, the user should get a fresh cartridge list. So to test: 1. Set up a new broker & node 2. service mcollective stop on the node 3. (clear broker cache if it has been populated) 4. rhc setup a user (it shouldn't get any errors but have empty cartridge list) 5. service mcollective start on the node 6. rhc setup a user again (it should report the correct list of cartridges) Fix is in rubygem-openshift-origin-controller-1.0.9-1.el6op
[jason@jasontest ~]$ rhc setup OpenShift Client Tools (RHC) Setup Wizard This wizard will help you upload your SSH keys, set your application namespace, and check that other programs like Git are properly installed. Login to broker.example.com: |jia@redhat.com| Password: *** Configuration file /home/jason/.openshift/express.conf already exists, backing up to /home/jason/.openshift/express.conf.bak Created local config file: /home/jason/.openshift/express.conf The express.conf file contains user configuration, and can be transferred to different computers. We will now check to see if you have the necessary client tools installed. Checking for git ... found Checking for your namespace ... found namespace: jia Checking for applications ... none found Run 'rhc app create' to create your first application. Below is a list of the types of application you can create: Thank you for setting up your system. You can rerun this at any time by calling 'rhc setup'. [jason@jasontest ~]$ Version: OpenShiftEnterprise/1.1.x/2012-12-18.1
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHSA-2013-0220.html