Hide Forgot
Description of problem: Checked the old existing applications on INT, visited the appilcation url and checked the application details page, the gear cost of app is "0 started / 0" and the status value is empty, the "Gears" info is "None", and "Source Code" info is like "ssh:///~/git/jbosseap.git/", "Remote Access" info is "ssh". Check existing app using "rhc app show --gears" or other rhc commands will also meet errors, details are as following. Version-Release number of selected component (if applicable): INT(devenv_4101) How reproducible: always Steps to Reproduce: 1.Login to INT 2.Visit old existing application 3.Check the application details page 4.Do rhc app show --gears , rhc setup, rhc ssh Actual results: The status info is empty and "Gears" info is "None", gear cost is all 0, and "Source Code" info is like "ssh:///~/git/jbosseap.git/", "Remote Access" info is "ssh".Check existing app using "rhc app show --gears" or other commands will also meet errors, details are as following: $ rhc setup Checking common problems . An SSH connection could not be established to kitchensink-lxiamigrate.int.rhcloud.com. Your SSH configuration may not be correct, or the application may not be responding. bad URI(absolute but no path): ssh:// (URI::InvalidURIError) $ rhc ssh jbossas70s Connecting to ssh:// ... ssh: Could not resolve hostname ssh://: Name or service not known [root@Daphne ~]# rhc app show n10map --state Cartridge nodejs-0.10, mongodb-2.2 is 0/0 started $ rhc app show jbossas70s --gears /usr/local/lib/ruby/gems/2.0.0/gems/rhc-1.18.0/lib/rhc/highline_extensions.rb:363:in `block (2 levels) in column_widths': undefined method `min' for nil:NilClass (NoMethodError) from /usr/local/lib/ruby/gems/2.0.0/gems/rhc-1.18.0/lib/rhc/highline_extensions.rb:359:in `each' from /usr/local/lib/ruby/gems/2.0.0/gems/rhc-1.18.0/lib/rhc/highline_extensions.rb:359:in `each_with_index' from /usr/local/lib/ruby/gems/2.0.0/gems/rhc-1.18.0/lib/rhc/highline_extensions.rb:359:in `block in column_widths' from /usr/local/lib/ruby/gems/2.0.0/gems/rhc-1.18.0/lib/rhc/highline_extensions.rb:358:in `each' from /usr/local/lib/ruby/gems/2.0.0/gems/rhc-1.18.0/lib/rhc/highline_extensions.rb:358:in `column_widths' from /usr/local/lib/ruby/gems/2.0.0/gems/rhc-1.18.0/lib/rhc/highline_extensions.rb:375:in `allocate_widths_for' from /usr/local/lib/ruby/gems/2.0.0/gems/rhc-1.18.0/lib/rhc/highline_extensions.rb:433:in `widths' from /usr/local/lib/ruby/gems/2.0.0/gems/rhc-1.18.0/lib/rhc/highline_extensions.rb:446:in `header_rows' from /usr/local/lib/ruby/gems/2.0.0/gems/rhc-1.18.0/lib/rhc/highline_extensions.rb:453:in `rows' from /usr/local/lib/ruby/gems/2.0.0/gems/rhc-1.18.0/lib/rhc/highline_extensions.rb:82:in `say' from /usr/local/lib/ruby/gems/2.0.0/gems/rhc-1.18.0/lib/rhc/commands/app.rb:347:in `show' from /usr/local/lib/ruby/gems/2.0.0/gems/rhc-1.18.0/lib/rhc/commands.rb:285:in `execute' from /usr/local/lib/ruby/gems/2.0.0/gems/rhc-1.18.0/lib/rhc/commands.rb:276:in `block (3 levels) in to_commander' from /usr/local/lib/ruby/gems/2.0.0/gems/commander-4.1.4/lib/commander/command.rb:180:in `call' from /usr/local/lib/ruby/gems/2.0.0/gems/commander-4.1.4/lib/commander/command.rb:180:in `call' from /usr/local/lib/ruby/gems/2.0.0/gems/commander-4.1.4/lib/commander/command.rb:155:in `run' from /usr/local/lib/ruby/gems/2.0.0/gems/commander-4.1.4/lib/commander/runner.rb:400:in `run_active_command' from /usr/local/lib/ruby/gems/2.0.0/gems/rhc-1.18.0/lib/rhc/command_runner.rb:73:in `run!' from /usr/local/lib/ruby/gems/2.0.0/gems/commander-4.1.4/lib/commander/delegates.rb:7:in `run!' from /usr/local/lib/ruby/gems/2.0.0/gems/rhc-1.18.0/lib/rhc/cli.rb:37:in `start' from /usr/local/lib/ruby/gems/2.0.0/gems/rhc-1.18.0/bin/rhc:20:in `<top (required)>' from /usr/local/bin/rhc:23:in `load' from /usr/local/bin/rhc:23:in `<main>' Expected results: Should display correct gears, status and ssh related info for all old applications. Additional info:
Besides above issues, adding alias and cartridges also does not work, result is as following: [root@dhcp-8-252 home]# rhc alias add www.nodejsalias.com -a nodejs06787 -d DEBUG: Using config file /root/.openshift/express.conf DEBUG: Git config 'git config --get rhc.app-id' returned '' DEBUG: Git config 'git config --get rhc.app-name' returned '' DEBUG: Git config 'git config --get rhc.domain-name' returned '' DEBUG: Authenticating with RHC::Auth::Basic DEBUG: Connecting to https://int.openshift.redhat.com/broker/rest/api DEBUG: Getting all domains DEBUG: Client supports API versions 1.1, 1.2, 1.3, 1.4, 1.5 DEBUG: Created new httpclient DEBUG: Request GET https://int.openshift.redhat.com/broker/rest/api DEBUG: code 200 1319 ms DEBUG: Server supports API versions 1.0, 1.1, 1.2, 1.3, 1.4, 1.5, 1.6 DEBUG: Using API version 1.5 DEBUG: Client API version 1.5 is not current. Refetching API DEBUG: Request GET https://int.openshift.redhat.com/broker/rest/api DEBUG: code 200 321 ms DEBUG: Request GET https://int.openshift.redhat.com/broker/rest/domains DEBUG: code 200 1159 ms DEBUG: Request GET https://int.openshift.redhat.com/broker/rest/domains/stg999test/applications/nodejs06787 DEBUG: code 200 869 ms DEBUG: Running add_alias for nodejs06787 DEBUG: Request POST https://int.openshift.redhat.com/broker/rest/domain/stg999test/application/nodejs06787/aliases DEBUG: code 404 1201 ms Alias 'www.nodejsalias.com' not found. [root@dhcp-8-252 home]# rhc cartridge add mysql-5.1 -a nodejs06787 -d DEBUG: Using config file /root/.openshift/express.conf DEBUG: Git config 'git config --get rhc.app-id' returned '' DEBUG: Git config 'git config --get rhc.app-name' returned '' DEBUG: Git config 'git config --get rhc.domain-name' returned '' DEBUG: Authenticating with RHC::Auth::Basic DEBUG: Connecting to https://int.openshift.redhat.com/broker/rest/api DEBUG: Getting all cartridges DEBUG: Client supports API versions 1.1, 1.2, 1.3, 1.4, 1.5 DEBUG: Created new httpclient DEBUG: Request GET https://int.openshift.redhat.com/broker/rest/api DEBUG: code 200 2413 ms DEBUG: Server supports API versions 1.0, 1.1, 1.2, 1.3, 1.4, 1.5, 1.6 DEBUG: Using API version 1.5 DEBUG: Client API version 1.5 is not current. Refetching API DEBUG: Request GET https://int.openshift.redhat.com/broker/rest/api DEBUG: code 200 357 ms DEBUG: Request GET https://int.openshift.redhat.com/broker/rest/cartridges DEBUG: code 200 937 ms Adding mysql-5.1 to application 'nodejs06787' ... DEBUG: Getting all domains DEBUG: Request GET https://int.openshift.redhat.com/broker/rest/domains DEBUG: code 200 1145 ms DEBUG: Request GET https://int.openshift.redhat.com/broker/rest/domains/stg999test/applications/nodejs06787?include=cartridges DEBUG: code 200 1142 ms DEBUG: Adding cartridge nodejs06787 DEBUG: Request POST https://int.openshift.redhat.com/broker/rest/domain/stg999test/application/nodejs06787/cartridges DEBUG: code 422 28476 ms Gear containing application dns not found
This issue can be reproduced on devenv_4102 as well. After upgrade the packages to the latest and restart the services, the gear ssh info will be cleared when check the app info. But app can be visited via web browser. # rhc app show jbeap1 Password: jbeap1 @ http://jbeap1-bmeng1.dev.rhcloud.com/ (uuid: 52a17f18baf5abbe55000001) ------------------------------------------------------------------------------- Domain: bmeng1 Created: 3:39 PM Gears: 0 (defaults to small) Git URL: ssh:///~/git/jbeap1.git/ SSH: ssh:// Deployment: auto (on git push) jbosseap-6 (JBoss Enterprise Application Platform 6) ---------------------------------------------------- Gears: Located with jenkins-client-1 This gear costs an additional $0.03 per gear after the first 3 gears. jenkins-client-1 (Jenkins Client) --------------------------------- Gears: Located with jbosseap-6 Job URL: https://jenkins-bmeng1.dev.rhcloud.com/job/jbeap1-build/ And when trying to upgrade the node, the gears cannot be found from the node. # oo-admin-upgrade upgrade-node --version 2.0.37 --ignore-cartridge-version Upgrader started with options: {:version=>"2.0.37", :ignore_cartridge_version=>true, :target_server_identity=>nil, :upgrade_position=>1, :num_upgraders=>1, :max_threads=>12, :gear_whitelist=>[], :num_tries=>2} Building new upgrade queues and cluster metadata Getting all active gears... Getting all logins... Writing node queue to /tmp/oo-upgrade/node_queue Writing cluster metadata to /tmp/oo-upgrade/cluster_metadata Loading cluster metadata from /tmp/oo-upgrade/cluster_metadata Loading node queue from /tmp/oo-upgrade/node_queue Node queue is empty; there's nothing for the upgrader to do. Exiting.
On the devenv_4102 environment testing, the gears document in applications collection of rhc-datastore was empty, which caused the problem. Not sure why this happened since this problem does not happen in previous testings. <-------------------------> "gears" : [ ], "group_instances" : [ { "_id" : ObjectId("52a16034ab7615edd3000009") } ], <------------------------->
This problem is still on INT. Wasn't able to reproduce this problem after upgrading devenv-stage_601 to devenv-stage_604 and migrate.
INT is broken, data related to 'gears' field is lost during migration (migration script was not re-entrant) and we did not had a good backup to restore INT. So ignore INT and test it on STG/DEV/TEST environment.
There was a bug in the migration script that was fixed with --> https://github.com/openshift/li/pull/2216 The fix should have made it into devenv_4108. Please try again on the latest devenv or on STG once it is built/ready today. As far as INT is concerned, it is not in a state to allow for testing the migration. Refer email from Dan and me on Friday on this topic as well as the comment above from Ravi.
Tested on STG(devenv-stage_604), old applications can work correctly after the migration of STG, so verify this bug, thanks.