Bug 1183048
Summary: | oo-admin-repair does not fix stale env vars in domains with no apps | |||
---|---|---|---|---|
Product: | OpenShift Online | Reporter: | Andy Grimm <agrimm> | |
Component: | Pod | Assignee: | Abhishek Gupta <abhgupta> | |
Status: | CLOSED CURRENTRELEASE | QA Contact: | libra bugs <libra-bugs> | |
Severity: | unspecified | Docs Contact: | ||
Priority: | unspecified | |||
Version: | 1.x | CC: | jgoulding, jhou, jokerman, mmccomas, qizhao | |
Target Milestone: | --- | |||
Target Release: | --- | |||
Hardware: | Unspecified | |||
OS: | Unspecified | |||
Whiteboard: | ||||
Fixed In Version: | Doc Type: | Bug Fix | ||
Doc Text: | Story Points: | --- | ||
Clone Of: | ||||
: | 1202511 (view as bug list) | Environment: | ||
Last Closed: | 2015-03-05 19:56:58 UTC | Type: | Bug | |
Regression: | --- | Mount Type: | --- | |
Documentation: | --- | CRM: | ||
Verified Versions: | Category: | --- | ||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
Cloudforms Team: | --- | Target Upstream Version: | ||
Embargoed: | ||||
Bug Depends On: | ||||
Bug Blocks: | 1202511 |
Description
Andy Grimm
2015-01-16 14:51:03 UTC
Proposed fix: https://github.com/openshift/origin-server/pull/6066 Verified on devenv-5428 Steps: 1,I create some apps. when I run rhc apps the result likes: [root@local ~]# rhc apps d1 @ http://d1-qizhaotest.dev.rhcloud.com/ (uuid: 54db519abf14976fb5000062) --------------------------------------------------------------------------- Domain: qizhaotest Created: 8:56 PM Gears: 1 (defaults to small) Git URL: ssh://54db519abf14976fb5000062.rhcloud.com/~/git/d1.git/ SSH: 54db519abf14976fb5000062.rhcloud.com Deployment: auto (on git push) diy-0.1 (Do-It-Yourself 0.1) ---------------------------- Gears: Located with jenkins-client-1 jenkins-client-1 (Jenkins Client) --------------------------------- Gears: Located with diy-0.1 Job URL: https://jks-qizhaotest.dev.rhcloud.com/job/d1-build/ jks @ http://jks-qizhaotest.dev.rhcloud.com/ (uuid: 54db5112bf14976fb500002b) ----------------------------------------------------------------------------- Domain: qizhaotest Created: 8:54 PM Gears: 1 (defaults to small) Git URL: ssh://54db5112bf14976fb500002b.rhcloud.com/~/git/jks.git/ SSH: 54db5112bf14976fb500002b.rhcloud.com Deployment: auto (on git push) jenkins-1 (Jenkins Server) -------------------------- Gears: 1 small php @ http://php-qizhaotest.dev.rhcloud.com/ (uuid: 54db5097bf14976fb5000001) ----------------------------------------------------------------------------- Domain: qizhaotest Created: 8:52 PM Gears: 1 (defaults to small) Git URL: ssh://54db5097bf14976fb5000001.rhcloud.com/~/git/php.git/ SSH: 54db5097bf14976fb5000001.rhcloud.com Deployment: auto (on git push) php-5.3 (PHP 5.3) ----------------- Gears: 1 small You have access to 3 applications. 2, ssh to the instance as root user. [root@ip-10-180-44-88 ~]# mongo openshift_broker_dev MongoDB shell version: 2.4.12 connecting to: openshift_broker_dev Welcome to the MongoDB shell. For interactive help, type "help". For more comprehensive documentation, see http://docs.mongodb.org/ Questions? Try the support group http://groups.google.com/group/mongodb-user libra_rs:PRIMARY> db.applications.findOne({name:"jks"}) { "_id" : ObjectId("54db5112bf14976fb500002b"), "analytics" : { "user_agent" : "rhc/1.34.2 (ruby 2.0.0; x86_64-linux) (API [1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7]) (2.5.3.3, ruby 2.0.0 (2013-11-22))" }, "builder_id" : null, "canonical_name" : "jks", "component_instances" : [ { "_id" : ObjectId("54db5112bf14976fb500003d"), "cartridge_id" : ObjectId("54da9e66b494fef2dc000020"), "cartridge_name" : "jenkins-1", "cartridge_vendor" : "redhat", "component_name" : "jenkins-1", "component_properties" : { "username" : "system_builder", "password" : "T_9zsdsvjhab" }, "created_at" : ISODate("2015-02-11T12:54:42.876Z"), "group_instance_id" : ObjectId("54db5112bf14976fb500002d") } ], "config" : { "auto_deploy" : true, "deployment_branch" : "master", "keep_deployments" : 1, "deployment_type" : "git" }, "created_at" : ISODate("2015-02-11T12:54:42.877Z"), "default_gear_size" : "small", "deployments" : [ { "_id" : ObjectId("54db515abf14976fb5000046"), "activations" : [ 1423659298.050458 ], "artifact_url" : null, "created_at" : ISODate("2015-02-11T12:54:45.799Z"), "deployment_id" : "85acbac7", "force_clean_build" : false, "hot_deploy" : false, "ref" : "master", "sha1" : "a4727e6" } ], "domain_id" : ObjectId("54db5080bf149765fc000008"), "domain_namespace" : "qizhaotest", "gears" : [ { "_id" : ObjectId("54db5112bf14976fb500002b"), "app_dns" : true, "group_instance_id" : ObjectId("54db5112bf14976fb500002d"), "host_singletons" : true, "name" : "jks", "quarantined" : false, "server_identity" : "ip-10-180-44-88", "sparse_carts" : [ ], "uid" : null, "uuid" : "54db5112bf14976fb500002b" } ], "group_instances" : [ { "_id" : ObjectId("54db5112bf14976fb500002d"), "platform" : "linux", "addtl_fs_gb" : 0, "gear_size" : "small" } ], "group_overrides" : [ ], "ha" : false, "init_git_url" : null, "members" : [ { "t" : null, "n" : "qizhao", "r" : "admin", "f" : [ [ "domain", "admin" ] ], "e" : null, "_id" : ObjectId("54db506ebf149765fc000001") } ], "name" : "jks", "owner_id" : ObjectId("54db506ebf149765fc000001"), "pending_op_groups" : [ ], "scalable" : false, "secret_token" : "YSFHTa_wHVmzm0f6BW5RAOVtMyVpbMt6IdYMDBLHS3nXNJtGSSurWJR93qYKyRMiuY75dwOzZRnAhBXwG4HxJx-qya7ZmOAKMup-7RLzXRVY_KVIwGG0MON8W7wT_6rX", "updated_at" : ISODate("2015-02-11T12:55:54.998Z"), "uuid" : "25ad8748b1ed11e482330242ac110002" } libra_rs:PRIMARY> libra_rs:PRIMARY> db.applications.findOne({name:"jks"}) { "_id" : ObjectId("54db5112bf14976fb500002b"), "analytics" : { "user_agent" : "rhc/1.34.2 (ruby 2.0.0; x86_64-linux) (API [1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7]) (2.5.3.3, ruby 2.0.0 (2013-11-22))" }, "builder_id" : null, "canonical_name" : "jks", "component_instances" : [ { "_id" : ObjectId("54db5112bf14976fb500003d"), "cartridge_id" : ObjectId("54da9e66b494fef2dc000020"), "cartridge_name" : "jenkins-1", "cartridge_vendor" : "redhat", "component_name" : "jenkins-1", "component_properties" : { "username" : "system_builder", "password" : "T_9zsdsvjhab" }, "created_at" : ISODate("2015-02-11T12:54:42.876Z"), "group_instance_id" : ObjectId("54db5112bf14976fb500002d") } ], "config" : { "auto_deploy" : true, "deployment_branch" : "master", "keep_deployments" : 1, "deployment_type" : "git" }, "created_at" : ISODate("2015-02-11T12:54:42.877Z"), "default_gear_size" : "small", "deployments" : [ { "_id" : ObjectId("54db515abf14976fb5000046"), "activations" : [ 1423659298.050458 ], "artifact_url" : null, "created_at" : ISODate("2015-02-11T12:54:45.799Z"), "deployment_id" : "85acbac7", "force_clean_build" : false, "hot_deploy" : false, "ref" : "master", "sha1" : "a4727e6" } ], "domain_id" : ObjectId("54db5080bf149765fc000008"), "domain_namespace" : "qizhaotest", "gears" : [ { "_id" : ObjectId("54db5112bf14976fb500002b"), "app_dns" : true, "group_instance_id" : ObjectId("54db5112bf14976fb500002d"), "host_singletons" : true, "name" : "jks", "quarantined" : false, "server_identity" : "ip-10-180-44-88", "sparse_carts" : [ ], "uid" : null, "uuid" : "54db5112bf14976fb500002b" } ], "group_instances" : [ { "_id" : ObjectId("54db5112bf14976fb500002d"), "platform" : "linux", "addtl_fs_gb" : 0, "gear_size" : "small" } ], "group_overrides" : [ ], "ha" : false, "init_git_url" : null, "members" : [ { "t" : null, "n" : "qizhao", "r" : "admin", "f" : [ [ "domain", "admin" ] ], "e" : null, "_id" : ObjectId("54db506ebf149765fc000001") } ], "name" : "jks", "owner_id" : ObjectId("54db506ebf149765fc000001"), "pending_op_groups" : [ ], "scalable" : false, "secret_token" : "YSFHTa_wHVmzm0f6BW5RAOVtMyVpbMt6IdYMDBLHS3nXNJtGSSurWJR93qYKyRMiuY75dwOzZRnAhBXwG4HxJx-qya7ZmOAKMup-7RLzXRVY_KVIwGG0MON8W7wT_6rX", "updated_at" : ISODate("2015-02-11T12:55:54.998Z"), "uuid" : "25ad8748b1ed11e482330242ac110002" } libra_rs:PRIMARY> db.applications.remove({name:"jks"}) 3, Then I run "rhc apps" from the client like: [root@local ~]# rhc apps d1 @ http://d1-qizhaotest.dev.rhcloud.com/ (uuid: 54db519abf14976fb5000062) --------------------------------------------------------------------------- Domain: qizhaotest Created: 8:56 PM Gears: 1 (defaults to small) Git URL: ssh://54db519abf14976fb5000062.rhcloud.com/~/git/d1.git/ SSH: 54db519abf14976fb5000062.rhcloud.com Deployment: auto (on git push) diy-0.1 (Do-It-Yourself 0.1) ---------------------------- Gears: Located with jenkins-client-1 jenkins-client-1 (Jenkins Client) --------------------------------- Gears: Located with diy-0.1 Job URL: https://jks-qizhaotest.dev.rhcloud.com/job/d1-build/ php @ http://php-qizhaotest.dev.rhcloud.com/ (uuid: 54db5097bf14976fb5000001) ----------------------------------------------------------------------------- Domain: qizhaotest Created: 8:52 PM Gears: 1 (defaults to small) Git URL: ssh://54db5097bf14976fb5000001.rhcloud.com/~/git/php.git/ SSH: 54db5097bf14976fb5000001.rhcloud.com Deployment: auto (on git push) php-5.3 (PHP 5.3) ----------------- Gears: 1 small You have access to 2 applications. 4, I return to the ssh command line of the instance, then runs [root@ip-10-180-44-88 ~]# oo-broker oo-admin-repair -r --ssh-keys Started at: 2015-02-11 13:03:43 UTC Total gears found in mongo: 2 Gear '54db5097bf14976fb5000001' has a stale key 'domain-jks' in mongo with missing component/gear '54db5112bf14976fb500003d'. Gear '54db5097bf14976fb5000001' has a stale environment variable 'JENKINS_URL' in mongo with missing component/gear '54db5112bf14976fb500003d'. Gear '54db5097bf14976fb5000001' has a stale environment variable 'JENKINS_USERNAME' in mongo with missing component/gear '54db5112bf14976fb500003d'. Gear '54db5097bf14976fb5000001' has a stale environment variable 'JENKINS_PASSWORD' in mongo with missing component/gear '54db5112bf14976fb500003d'. Finished at: 2015-02-11 13:04:04 UTC Total time: 20.888s SUCCESS 5, from rhc client I delete all the applications under this domain: [root@local ~]# rhc app delete php This is a non-reversible action! Your application code and data will be permanently deleted if you continue! Are you sure you want to delete the application 'php'? (yes|no): yes Deleting application 'php' ... deleted [root@local ~]# rhc app delete d1 This is a non-reversible action! Your application code and data will be permanently deleted if you continue! Are you sure you want to delete the application 'd1'? (yes|no): yes Deleting application 'd1' ... deleted The corresponding job 'd1-build' in Jenkins has been disabled. You can re-enable or delete as desired. Job URL: https://jks-qizhaotest.dev.rhcloud.com/job/d1-build/ [root@local ~]# rhc apps No applications. Use 'rhc create-app'. 6, I return to the ssh command line of the instance, Re-run "oo-admin-repair -r --ssh-keys" and see the results: [root@ip-10-180-44-88 ~]# oo-broker oo-admin-repair -r --ssh-keys Started at: 2015-02-11 13:25:11 UTC Total gears found in mongo: 0 Domain 'qizhaotest' has a stale key 'domain-jks' in mongo with missing component/gear '54db5112bf14976fb500003d'. Domain 'qizhaotest' has a stale environment variable 'JENKINS_URL' in mongo with missing component/gear '54db5112bf14976fb500003d'. Domain 'qizhaotest' has a stale environment variable 'JENKINS_USERNAME' in mongo with missing component/gear '54db5112bf14976fb500003d'. Domain 'qizhaotest' has a stale environment variable 'JENKINS_PASSWORD' in mongo with missing component/gear '54db5112bf14976fb500003d'. Finished at: 2015-02-11 13:25:32 UTC Total time: 20.972s SUCCESS Result: 1 stale env vars should be detected in all domains, regardless of whether apps are present. 7, I repair the stale env vars from the ssh console: [root@ip-10-180-44-88 ~]# oo-broker oo-admin-repair --ssh-keys Started at: 2015-02-11 14:14:24 UTC Total gears found in mongo: 0 ...... Finished at: 2015-02-11 14:14:45 UTC Total time: 20.891s SUCCESS 8, I retest the error: [root@ip-10-180-44-88 ~]# oo-broker oo-admin-repair -r --ssh-keys Started at: 2015-02-11 14:15:16 UTC Total gears found in mongo: 0 Finished at: 2015-02-11 14:15:37 UTC Total time: 20.864s SUCCESS Result: 1 stale env vars error can be repaired. |