Description of problem: Add additional storage to an application as user with silver plan, and execute oo-admin-chk -l 1 from broker, checked failed with addtl storage not found for corresponding usage record. Version-Release number of selected component (if applicable): On devenv_4003 How reproducible: Always Steps to Reproduce: 1. Create a user with silver plan 2. Create an application, add 3G storage to it 3. Execute 'oo-admin-chk -l 1' Actual results: [root@ip-10-62-121-228 ~]# oo-admin-chk -l 1 Started at: 2013-11-08 03:25:40 -0500 Time to fetch mongo data: 0.035s Total gears found in mongo: 1 Time to get all gears from nodes: 20.71s Total gears found on the nodes: 1 Total nodes that responded : 1 Time to get all sshkeys for all gears from nodes: 20.061s Total gears found on the nodes: 1 Total nodes that responded : 1 Check failed. Found addtl storage for gear Id '527c9fb7fbcf4c622d00012c' but could not find corresponding usage record. Please refer to the oo-admin-repair tool to resolve some of these inconsistencies. Total time: 42.181s Finished at: 2013-11-08 03:26:22 -0500 Expected results: Should ignore the setmaxuntrackedstorage to make the checking pass. Additional info:
Fixed with --> https://github.com/openshift/origin-server/pull/4139
Updated the wrong bug, ignore comment #1
Tried couple of times but unable to reproduce the issue on latest devenv. Please verify once again and let me know exact steps for reproducing the issue.
Hi, @Ravi: here is the steps to reproduce this issue on devenv_4016 1. Create a scalable php application rhc create-app php1s php-5.3 -s 2. Upgrade current account to silver plan via restapi 3. Add 3G storage to the app rhc cartridge-storage --add 3 -c php-5.3 -a php1s 4. On broker run oo-admin-chk oo-admin-chk -l 1 Result: [root@ip-10-158-73-211 ~]# oo-admin-chk -l 1 Started at: 2013-11-10 21:43:25 -0500 Time to fetch mongo data: 0.282s Total gears found in mongo: 13 Time to get all gears from nodes: 21.454s Total gears found on the nodes: 13 Total nodes that responded : 1 Time to get all sshkeys for all gears from nodes: 20.049s Total gears found on the nodes: 13 Total nodes that responded : 1 Check failed. Found addtl storage for gear Id '528038e74937b87264000186' but could not find corresponding usage record. Please refer to the oo-admin-repair tool to resolve some of these inconsistencies. Total time: 44.281s Finished at: 2013-11-10 21:44:09 -0500 Re-check applications and usage_records collections: libra_rs:PRIMARY> db.applications.findOne({_id:ObjectId("528038e74937b87264000186")},{name:1,members:1,group_overrides:1}) { "_id" : ObjectId("528038e74937b87264000186"), "group_overrides" : [ { "components" : [ { "cart" : "php-5.3", "comp" : "php-5.3" }, { "cart" : "haproxy-1.4", "comp" : "web_proxy" } ], "min_gears" : 1, "additional_filesystem_gb" : 3, "max_gears" : -1, "gear_size" : "small" } ], "members" : [ { "_type" : null, "n" : "jhou", "r" : "admin", "f" : [ [ "domain", "admin" ] ], "e" : null, "_id" : ObjectId("528038d84937b8e9fb000595") } ], "name" : "php1s" } libra_rs:PRIMARY> db.usage_records.find({app_name:"php1s"}).pretty() { "_id" : ObjectId("528038f04937b872640001a3"), "event" : "begin", "time" : ISODate("2013-11-11T01:54:56.988Z"), "created_at" : ISODate("2013-11-11T01:54:56.988Z"), "gear_id" : ObjectId("528038e74937b87264000186"), "usage_type" : "GEAR_USAGE", "user_id" : ObjectId("528038d84937b8e9fb000595"), "app_name" : "php1s", "gear_size" : "small", "updated_at" : ISODate("2013-11-11T01:54:56.992Z") }
Here is my account detail for this testing { "api_version": 1.6, "data": { "capabilities": { "subaccounts": false, "gear_sizes": [ "small", "medium", "large" ], "plan_upgrade_enabled": true, "private_ssl_certificates": true, "max_storage_per_gear": 5 }, "consumed_gears": 1, "created_at": "2013-11-11T01:54:32Z", "id": "528038d84937b8e9fb000595", "links": { "ADD_KEY": { "href": "https://ec2-107-20-86-143.compute-1.amazonaws.com/broker/rest/user/keys", "method": "POST", "optional_params": [ ], "rel": "Add new SSH key", "required_params": [ { "description": "Name of the key", "invalid_options": [ ], "name": "name", "type": "string", "valid_options": [ ] }, { "description": "Type of Key", "invalid_options": [ ], "name": "type", "type": "string", "valid_options": [ "ssh-rsa", "ssh-dss", "ssh-rsa-cert-v01", "ssh-dss-cert-v01", "ssh-rsa-cert-v00", "ssh-dss-cert-v00", "krb5-principal" ] }, { "description": "The key portion of an ssh key (excluding key type and comment)", "invalid_options": [ ], "name": "content", "type": "string", "valid_options": [ ] } ] }, "LIST_KEYS": { "href": "https://ec2-107-20-86-143.compute-1.amazonaws.com/broker/rest/user/keys", "method": "GET", "optional_params": [ ], "rel": "List SSH keys", "required_params": [ ] } }, "login": "jhou", "max_domains": 3, "max_gears": 16, "plan_id": "silver", "plan_state": "ACTIVE", "usage_account_id": "1701043" }, "messages": [ ], "status": "ok", "supported_api_versions": [ 1.0, 1.1, 1.2, 1.3, 1.4, 1.5, 1.6 ], "type": "user", "version": "1.6" }
Fixed in https://github.com/openshift/origin-server/pull/4160
Above PR failed to merge, will verify it when the fix is in.
Commit pushed to master at https://github.com/openshift/origin-server https://github.com/openshift/origin-server/commit/19ca083e21bd4d19e6a124ea0a0a40aa8750a8db Bug 1028337 - Fix additional storage consistency check in oo-admin-chk
Verified on devenv_4027, the problem is fixed and oo-admin-chk passed