Bug 844490

I found the problem, when I looked at how many gears the user had.

User kevin.j.hooke:
      consumed gears: 5
           max gears: 3
          vip status: false

I'm not sure how it got into that state, but when I raised his gears to 6, then I was able to delete his applications.

I think that this is still a bug as the admin tools should always be able to remove an app.

Its failing the consistency check - (consumed_gears + ngears <= max_gears)
where, ngears is '-1' for a gear delete. The appropriate fix is coming with US2550, where gear and app deletion is separated. That also prevents the known cases where consumed_gears gets out of sync with actual gears consumed.

While the admin way should be to fix up 'consumed_gears' and then delete the app, because consumed_gears need to be fixed up anyway, should there be a method to 'force delete' the app? 


The 'force-delete' feature is implemented with pull request#147 in li.repo


Be aware that such a 'force delete' may result with gears being on the disk but not on mongo - the feature does list out the warnings indicating such slips, though.



How to test the feature?
1. Create some apps in a user account (on devenv)

2. Then force set the consumed gears count to reflect the situation in this bug. Use rhc-admin-ctl-user --setconsumedgears for that. Set it to a number much more than the max_gear limit.

3. Check mismatch between consumed_gears and max_gear limit by running rhc-admin-ctl-user -l <login>

4. Then try deleting the app, either as user, or by rhc-admin-ctl-app -a <app-name> -c destroy -l <login>

5. The deletion should fail.

6. Now try with the new feature rhc-admin-ctl-app -a <app-name> -c force-destroy -l <login>. The operation should succeed with some warnings on the screen.

expect to have fix today

Fix available with rev#2a968bbdfd024d7d3029ead211c6f15f4b90c4bc in li.repo and rev#1035699e2bf9d66255ad230f781552f21bfe36b6 in crankcase.repo

The consumed_gears count is not checked at gear deletion time now. With the new fix step#4 in comment#3 will succeed also.

Verififid this bug on devenv_1931, and PASSED.


1. Create some dummy data into mongodb, like this:
PRIMARY> db.user.find()
{ "_id" : "jialiu", "apps" : [ 	{ 	"categories" : [ ], 	"suggests_feature" : [ ], 	"creation_time" : "2012-08-01T05:32:13-04:00", 	"cart_data_def" : { 	 }, 	"help_topics" : { 	 }, 	"license" : "unknown", 	"name" : "phptest", 	"start_order" : [ 	"@@app/cart-php-5.3", 	"@@app" ], 	"conflicts_feature" : [ ], 	"conn_endpoints_list" : [ ], 	"vendor" : "unknown", 	"website" : "", 	"default_profile" : "default", 	"requires" : [ ], 	"group_instances" : [ 	{ 	"component_instances" : [ 	"@@app", 	"@@app/cart-php-5.3" ], 	"profile_name" : "default", 	"name" : "@@app/cart-php-5.3", 	"gears" : [ 	{ 	"configured_components" : [ 	"@@app/cart-php-5.3" ], 	"name" : "phptest", 	"group_instance_name" : "@@app/cart-php-5.3", 	"server_identity" : "ip-10-190-25-42", 	"uuid" : "5999fb93ea7743528f53036c43102acc", 	"uid" : null, 	"node_profile" : "small" } ], 	"group_name" : "default", 	"cart_name" : "php-5.3", 	"reused_by" : [ 	"@@app/cart-php-5.3", 	"@@app" ], 	"min" : 1, 	"max" : -1, 	"node_profile" : "small" } ], 	"requires_feature" : [ 	"php-5.3" ], 	"domain" : { 	"namespace" : "jialiu", 	"uuid" : "6eda7bcc625d43428c3411b4a47fcef5" }, 	"profiles" : [ 	{ 	"stop_order" : [ ], 	"name" : "default", 	"start_order" : [ ], 	"components" : [ 	{ 	"subscribes" : [ ], 	"depends_service" : [ ], "depends" : [ ], 	"name" : "default", 	"publishes" : [ ], 	"generated" : true } ], 	"group_overrides" : [ ], 	"groups" : [ 	{ 	"component_refs" : [ 	{ 	"name" : "default", 	"component" : "default" } ], 	"scaling" : { 	"min" : 1, 	"max" : -1 }, 	"name" : "default", 	"generated" : true } ], 	"connections" : [ ], 	"configure_order" : [ ], 	"generated" : true, 	"provides" : [ ] } ], 	"comp_instances" : [ 	{ 	"exec_order" : [ ], 	"parent_cart_group" : "default", 	"name" : "@@app/cart-php-5.3", 	"parent_component_name" : "default", 	"cart_properties" : { 	 }, 	"group_instance_name" : "@@app/cart-php-5.3", 	"parent_cart_profile" : "default", 	"cart_data" : [ 	null ], 	"parent_cart_name" : "php-5.3", 	"dependencies" : [ ] }, 	{ 	"exec_order" : [ 	"@@app/cart-php-5.3" ], 	"parent_cart_group" : "default", 	"name" : "@@app", 	"parent_component_name" : "default", 	"cart_properties" : { 	 }, 	"group_instance_name" : "@@app", 	"parent_cart_profile" : "default", 	"cart_data" : [ ], 	"parent_cart_name" : "phptest", 	"dependencies" : [ 	"@@app/cart-php-5.3" ] } ], 	"description" : "", 	"uuid" : "5999fb93ea7743528f53036c43102acc", 	"license_url" : "", 	"scalable" : false, 	"display_name" : "phptest-0.0-noarch", 	"provides_feature" : [ ], 	"configure_order" : [ 	"@@app/cart-php-5.3", 	"@@app" ], 	"node_profile" : "small", 	"architecture" : "noarch", 	"version" : "0.0" } ], "capabilities" : { "gear_sizes" : [ "small" ] }, "consumed_gears" : 5, "domains" : [ { "namespace" : "jialiu", "uuid" : "6eda7bcc625d43428c3411b4a47fcef5" } ], "env_vars" : null, "login" : "jialiu", "max_gears" : 3, "parent_user_login" : null, "ssh_keys" : { "default" : { "key" : "AAAAB3NzaC1yc2EAAAADAQABAAABAQDSv44aEPcObZkAN5VI8XHW23b7JL0wBftkPwtXwHF6ppxnvpIhQNyBy5crHWGrigEOGLsJWH7hmo/rfkELuhfpdaGIB582AAJ5Eeug+Fv7yQFQodCafALhh/piXXnJ7xsnFpy6Pz5OVuxC2nRoew8oqSIjKaHTdjzuSPNRviEKLTypcREtnQp7nCCTDm3NFjaM40tDA3/i9m708qViQHv5tqkdyrfLMu5Lq+oJMrzP911aCn3F0GTc+T/cUC/R2ay5wLhv9FT+eTrDSOsMFt9BZYFT+mfSyIJhaGuxB7OUQ3qf4RRMo+0hGINRSLldFtRaZiQUyqw6nKPR460MfgMN", "type" : "ssh-rsa" } }, "system_ssh_keys" : null, "usage_records" : [ 	{ 	"gear_size" : "small", 	"sync_time" : null, 	"usage_type" : "GEAR_USAGE", 	"gear_uuid" : "5999fb93ea7743528f53036c43102acc", 	"uuid" : "8d38a66f9d264762bcc327786312d34b", 	"time" : ISODate("2012-08-01T09:32:23.808Z"), 	"event" : "begin" } ], "uuid" : "0fd23545dce548529239c647df510e50" }


2. Check mismatch between consumed_gears and max_gear limit
# rhc-admin-ctl-user -l jialiu


User jialiu:
      consumed gears: 5
           max gears: 3
          gear sizes: small


3. Run rhc-admin-ctl-app with destroy option successfully, and check mongo to verify the dummp app data is also removed from mongo.
# rhc-admin-ctl-app -a phptest -c destroy -l jialiu 
  !!!! WARNING !!!! WARNING !!!! WARNING !!!!
  You are about to destroy the phptest application.

  This is NOT reversible, all remote data for this application will be removed.
Do you want to destroy this application (y/n): y
Successfully destroyed application: phptest

Check mongo, find the dummp app data is also removed from mongo.

4. Run rhc-admin-ctl-app with force-destroy option successfully, and check mongo to verify the dummp app data is also removed from mongo.
# rhc-admin-ctl-app -a phptest -c force-destroy -l jialiu
  !!!! WARNING !!!! WARNING !!!! WARNING !!!!
  You are about to destroy the phptest application.

  This is NOT reversible, all remote data for this application will be forcefully removed.
Do you want to forcefully destroy this application (y/n): y
Successfully destroyed application: phptest