Description of problem: In INT, oo-admin-chk is reporting quite a few problems like this: Gear '5264ee9a6cec0e897f00001c' has key with hash '62cff379a9633286d55ec043e6c69413' and updated name 'domain-jenkins' in mongo but not on the node. I then run 'oo-admin-repair --ssh-keys', but it doesn't clean up The output at the end is this (no errors on the ssh key stuff): Fixed ssh key mismatches for 43 applications. Fixed consumed gears mismatches for 3 user. Failed to fix consumed gears mismatches for 1 applications. Fixed 6 unused UIDs across all districts. Version-Release number of selected component (if applicable): openshift-origin-broker-util-1.16.6-1.el6oso.noarch How reproducible: very with specific keys Steps to Reproduce: 1. unknown, found in INT Actual results: ssh key isn't able to be cleaned up Expected results: oo-admin-repair should be able to fix this
This is a case of bad data where the domain has two jenkins ssh keys. The previous key was not removed when the jenkins application was removed from the domain. The code has been fixed to ensure that even if an ssh key is left behind, then adding the jenkins app/server again will not add a second key but instead remove all the older keys and add just the new one. The cases of bad data will need to be fixed in production/stage.
We could add a check to oo-admin-chk and oo-admin-repair to weed out such ssh keys and environment variables that have been left behind after the component that was responsbible for creating them has been deleted/removed. We have a trello card for this already --> https://trello.com/c/tXKfaFjy