Bug 1045343 - [fork] oo-admin-chk -l 1 show 'FAILED' but did not detect any error actually after oo-admin-repair
Summary: [fork] oo-admin-chk -l 1 show 'FAILED' but did not detect any error actually ...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: OpenShift Online
Classification: Red Hat
Component: Pod
Version: 2.x
Hardware: All
OS: All
low
low
Target Milestone: ---
: ---
Assignee: Ravi Sankar
QA Contact: libra bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-12-20 08:30 UTC by zhaozhanqi
Modified: 2015-05-15 00:24 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2014-02-26 19:09:07 UTC
Target Upstream Version:


Attachments (Terms of Use)

Description zhaozhanqi 2013-12-20 08:30:49 UTC
Description of problem:
By oo-admin-repair --ssh-key to repair the ssh key issue, after that, oo-admin-chk -l1 will show 'FAILED', but in fact, it did not find some error.

Version-Release number of selected component (if applicable):
devenv_4161

How reproducible:
always

Steps to Reproduce:
1. Create an app 
2. Create ssh key
 ssh-keygen -t rsa -N '' -f testkey
3. Add new sshkey
  rhc sshkey add testkey testkey.pub
4. Remove the testkey in /var/lib/openshift/UUID/.ssh/authorized_keys 
5. Run 'oo-admin-chk -l 1'
6. Run 'oo-admin-repair --sshkey
7. Run 'oo-admin-chk -l 1' again

Actual results:

step 5:
oo-admin-chk -l1
Started at: 2013-12-20 06:59:19 UTC

Total gears found in mongo: 2
Total gears found on the nodes: 2
Total nodes that responded: 1

Finished at: 2013-12-20 07:00:01 UTC
Total time: 42.004s
Gear '52b3e86a92e613db8d000036' has key with hash '7a611e51642d8c0ba56a1e217532a5d9' and updated name '52b3e6c292e613db8d000001-testkey' in mongo but not on the node.

FAILED
Please refer to the oo-admin-repair tool to resolve some of these inconsistencies.

*****************************************************************
step 6:

oo-admin-repair --ssh-keys
Started at: 2013-12-20 02:07:21 -0500
Time to fetch mongo data: 0.021s
Total gears found in mongo: 2
Time to get all sshkeys for all gears from nodes: 20.732s
Total gears found on the nodes: 2
Total nodes that responded : 1
Check failed.
Gear '52b3e86a92e613db8d000036' has key with hash '7a611e51642d8c0ba56a1e217532a5d9' and updated name '52b3e6c292e613db8d000001-testkey' in mongo but not on the node.


Fixing ssh key inconsistencies for all affected applications:

Fixed ssh key mismatches for 1 applications.

Total time: 21.512s
Finished at: 2013-12-20 02:07:43 -0500

***********************************************************
step 7, oo-admin-chk -l 1 still show 'FAILED' but did not find any errors.

oo-admin-chk -l1 -v
Started at: 2013-12-20 07:15:16 UTC

Total gears found in mongo: 2
Total gears found on the nodes: 2
Total nodes that responded: 1
Checking application gears on corresponding nodes
Checking node gears in application database
Checking for user consumed gears and actual gears
Checking application gears and ssh keys on corresponding nodes
Checking for unreserved UIDs in the district
Checking for unused UIDs in the district
Checking for gears with the same UID
Checking gears available in applications collection but not in usage_records and viceversa: OK
Checking gears with additional storage in applications collection but not in usage_records and viceversa: OK
Checking gears with premium cartridge in applications collection but not in usage_records and viceversa: OK
Checking un-ended records in usage_records collection but not in usage collection and viceversa: OK
Checking usage_account_id validity for all billing users: OK
Checking plan_id validity for all billing users: OK
Checking plan_state validity for all billing users: OK

Finished at: 2013-12-20 07:15:58 UTC
Total time: 41.943s

FAILED
Please refer to the oo-admin-repair tool to resolve some of these inconsistencies.
 
Expected results:
Should show 'PASS'

Additional info:
Use 'oo-admin-repair --consumed-gears/ssh-keys/district-uids -r' to check, all of them show 'Success'

Comment 1 Ravi Sankar 2014-01-07 05:10:29 UTC
Fixed in fork_ami_broker_admin_repair_refactor_975

Comment 2 zhaozhanqi 2014-01-07 06:44:59 UTC
Tested this issue on fork_ami_broker_admin_repair_refactor_975, it has been fixed. 

After repair the sshkey by 'oo-admin-repair --ssh-key', run 'oo-admin-chk -l1 ' will show 'success'


oo-admin-repair --ssh-keys 
Started at: 2014-01-07 06:30:17 UTC
Total gears found in mongo: 1
Gear '52cb97d1a420637598000007' has key with hash 'b72fb83f7d4fd0dbaf810c5d350ca098' and updated name '52cb97bba420637598000001-testkey' in mongo but not on the node.

Finished at: 2014-01-07 06:30:39 UTC
Total time: 21.29s
SUCCESS
[root@ip-10-141-185-239 openshift]# oo-admin-chk -l 1
Started at: 2014-01-07 06:31:08 UTC

Total gears found in mongo: 1
Total gears found on the nodes: 1
Total nodes that responded: 1

Finished at: 2014-01-07 06:31:49 UTC
Total time: 40.72s
SUCCESS

Will mark it as 'verified' once it is merged to master

Comment 3 zhaozhanqi 2014-01-08 09:10:16 UTC
Mark this bug as 'verified' since it was merged.


Note You need to log in before you can comment on or make changes to this bug.