Bug 1254882

Summary: Provisioning quota for CPU/Mem/Storage doesn't work for cloud providers
Product: Red Hat CloudForms Management Engine Reporter: Aziza Karol <akarol>
Component: AutomateAssignee: Tina Fitzgerald <tfitzger>
Status: CLOSED ERRATA QA Contact: Aziza Karol <akarol>
Severity: medium Docs Contact:
Priority: high    
Version: 5.4.0CC: gmccullo, jhardy, mkanoor, obarenbo, psavage, simaishi, tfitzger
Target Milestone: GA   
Target Release: 5.5.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: 5.5.0.11 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-12-08 13:27:51 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Aziza Karol 2015-08-19 07:49:58 UTC
Description of problem:


Version-Release number of selected component (if applicable):
100%

How reproducible:
5.4.1

Steps to Reproduce:
1.Tag the group "EvmGroup-super_administrator" with "Quota - Max Memory: 1GB" 
2.Navigate to cloud instance and provision instance by selecting the the flavors having memory greater then 1GB. (I selected m2.xlarge with memory 17GB.)

similarly repeat the steps for cpu and storage quota check by tagging.

Actual results:
Provision was successful.

Expected results:
Quota validation should be successful and message should be displayed in the request details

Additional info:

Comment 2 Greg McCullough 2015-08-19 12:39:15 UTC
Tina - The validate_quotas method for cloud is looking for memory/cpu settings from values that are only populated by the infra provisioning dialogs.  For cloud these values come from the "Instance Type" or flavor record.  The logic would need to be changed to lookup the selected flavor instance to determine the cpu/memory values.  Storage will need additional investigation as we do collect some storage information in the flavor but not size.

Comment 9 CFME Bot 2015-10-27 03:05:41 UTC
New commit detected on ManageIQ/manageiq/master:
https://github.com/ManageIQ/manageiq/commit/1250288dd7bc98b8975a4fa264f2fb12addadf66

commit 1250288dd7bc98b8975a4fa264f2fb12addadf66
Author:     Tina Fitzgerald <tfitzger>
AuthorDate: Mon Sep 14 11:35:52 2015 -0400
Commit:     Tina Fitzgerald <tfitzger>
CommitDate: Fri Oct 23 15:48:27 2015 -0400

    Consolidating quota code to help facilitate resolution for the
    following issues:
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1254211
     when quota exceeds Group Allocated Memory always shows "0.00GB"
     in last message of request details page (edit)
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1248747
     service :quota : Provisioning quota for CPU , Memory and Storage
    doesn't work (edit)
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1254882
    Provisioning quota for CPU/Mem/Storage doesn't work
    for cloud providers (edit)
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1126934
    [RFE] Quota enforcement based on used space (edit)
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1249772
    [RFE] Service :quota : Should be able to apply VM quota to service (edit)

 .../default.yaml                                   |   5 +-
 .../default.yaml                                   |   5 +-
 .../CommonMethods/Quota.class/__class__.yaml       | 173 ++++++++
 .../Quota.class/__methods__/validate_quota.rb      | 481 +++++++++++++++++++++
 .../Quota.class/__methods__/validate_quota.yaml    |  12 +
 .../CommonMethods/Quota.class/service_quota.yaml   |  26 ++
 .../System/CommonMethods/Quota.class/test.yaml     |  24 +
 .../System/CommonMethods/Quota.class/vm_quota.yaml |  26 ++
 8 files changed, 750 insertions(+), 2 deletions(-)
 create mode 100644 db/fixtures/ae_datastore/ManageIQ/System/CommonMethods/Quota.class/__class__.yaml
 create mode 100644 db/fixtures/ae_datastore/ManageIQ/System/CommonMethods/Quota.class/__methods__/validate_quota.rb
 create mode 100644 db/fixtures/ae_datastore/ManageIQ/System/CommonMethods/Quota.class/__methods__/validate_quota.yaml
 create mode 100644 db/fixtures/ae_datastore/ManageIQ/System/CommonMethods/Quota.class/service_quota.yaml
 create mode 100644 db/fixtures/ae_datastore/ManageIQ/System/CommonMethods/Quota.class/test.yaml
 create mode 100644 db/fixtures/ae_datastore/ManageIQ/System/CommonMethods/Quota.class/vm_quota.yaml

Comment 10 CFME Bot 2015-10-27 03:47:48 UTC
New commit detected on ManageIQ/manageiq/master:
https://github.com/ManageIQ/manageiq/commit/1250288dd7bc98b8975a4fa264f2fb12addadf66

commit 1250288dd7bc98b8975a4fa264f2fb12addadf66
Author:     Tina Fitzgerald <tfitzger>
AuthorDate: Mon Sep 14 11:35:52 2015 -0400
Commit:     Tina Fitzgerald <tfitzger>
CommitDate: Fri Oct 23 15:48:27 2015 -0400

    Consolidating quota code to help facilitate resolution for the
    following issues:
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1254211
     when quota exceeds Group Allocated Memory always shows "0.00GB"
     in last message of request details page (edit)
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1248747
     service :quota : Provisioning quota for CPU , Memory and Storage
    doesn't work (edit)
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1254882
    Provisioning quota for CPU/Mem/Storage doesn't work
    for cloud providers (edit)
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1126934
    [RFE] Quota enforcement based on used space (edit)
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1249772
    [RFE] Service :quota : Should be able to apply VM quota to service (edit)

 .../default.yaml                                   |   5 +-
 .../default.yaml                                   |   5 +-
 .../CommonMethods/Quota.class/__class__.yaml       | 173 ++++++++
 .../Quota.class/__methods__/validate_quota.rb      | 481 +++++++++++++++++++++
 .../Quota.class/__methods__/validate_quota.yaml    |  12 +
 .../CommonMethods/Quota.class/service_quota.yaml   |  26 ++
 .../System/CommonMethods/Quota.class/test.yaml     |  24 +
 .../System/CommonMethods/Quota.class/vm_quota.yaml |  26 ++
 8 files changed, 750 insertions(+), 2 deletions(-)
 create mode 100644 db/fixtures/ae_datastore/ManageIQ/System/CommonMethods/Quota.class/__class__.yaml
 create mode 100644 db/fixtures/ae_datastore/ManageIQ/System/CommonMethods/Quota.class/__methods__/validate_quota.rb
 create mode 100644 db/fixtures/ae_datastore/ManageIQ/System/CommonMethods/Quota.class/__methods__/validate_quota.yaml
 create mode 100644 db/fixtures/ae_datastore/ManageIQ/System/CommonMethods/Quota.class/service_quota.yaml
 create mode 100644 db/fixtures/ae_datastore/ManageIQ/System/CommonMethods/Quota.class/test.yaml
 create mode 100644 db/fixtures/ae_datastore/ManageIQ/System/CommonMethods/Quota.class/vm_quota.yaml

Comment 12 CFME Bot 2015-11-16 23:00:20 UTC
New commit detected on ManageIQ/manageiq/master:
https://github.com/ManageIQ/manageiq/commit/449991e3bd1a1b8faff3eec22c9953e3c15acf0c

commit 449991e3bd1a1b8faff3eec22c9953e3c15acf0c
Author:     Tina Fitzgerald <tfitzger>
AuthorDate: Mon Nov 2 11:08:20 2015 -0500
Commit:     Tina Fitzgerald <tfitzger>
CommitDate: Mon Nov 16 16:45:58 2015 -0500

    Fix cloud requested values and fixed spec test to reflect changes.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1254882

 .../QuotaMethods.class/__methods__/requested.rb    | 155 ++++++++++++++-------
 .../method_validation/calculate_requested_spec.rb  |   2 +-
 2 files changed, 109 insertions(+), 48 deletions(-)

Comment 13 CFME Bot 2015-11-17 03:03:19 UTC
New commit detected on cfme/5.5.z:
https://code.engineering.redhat.com/gerrit/gitweb?p=cfme.git;a=commitdiff;h=1923ca56e87ef5213b1910c6d8ea6d2adb639139

commit 1923ca56e87ef5213b1910c6d8ea6d2adb639139
Author:     Tina Fitzgerald <tfitzger>
AuthorDate: Mon Nov 2 11:08:20 2015 -0500
Commit:     Tina Fitzgerald <tfitzger>
CommitDate: Mon Nov 16 17:52:05 2015 -0500

    Fix cloud requested values and fixed spec test to reflect changes.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1254882

 .../QuotaMethods.class/__methods__/requested.rb    | 155 ++++++++++++++-------
 .../method_validation/calculate_requested_spec.rb  |   2 +-
 2 files changed, 109 insertions(+), 48 deletions(-)

Comment 14 CFME Bot 2015-11-17 03:03:26 UTC
New commit detected on cfme/5.5.z:
https://code.engineering.redhat.com/gerrit/gitweb?p=cfme.git;a=commitdiff;h=ddee05133f2bc193b3a05e735cddb9a25271b8b9

commit ddee05133f2bc193b3a05e735cddb9a25271b8b9
Merge: ca2014f 1923ca5
Author:     Greg McCullough <gmccullo>
AuthorDate: Mon Nov 16 21:57:28 2015 -0500
Commit:     Greg McCullough <gmccullo>
CommitDate: Mon Nov 16 21:57:28 2015 -0500

    Merge branch 'quota_fixes_cherry_pick' into '5.5.z'
    
    Fix cloud requested values and fixed spec test to reflect changes.
    
    https://github.com/ManageIQ/manageiq/pull/5267
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1254882
    
    Clean cherry-pick.
    
    Also resolves:
    https://bugzilla.redhat.com/show_bug.cgi?id=1248747
    https://bugzilla.redhat.com/show_bug.cgi?id=1254211
    https://bugzilla.redhat.com/show_bug.cgi?id=1126934
    https://bugzilla.redhat.com/show_bug.cgi?id=1273919
    https://bugzilla.redhat.com/show_bug.cgi?id=1263592
    
    See merge request !481

 .../QuotaMethods.class/__methods__/requested.rb    | 155 ++++++++++++++-------
 .../method_validation/calculate_requested_spec.rb  |   2 +-
 2 files changed, 109 insertions(+), 48 deletions(-)

Comment 15 Tina Fitzgerald 2015-11-30 20:35:16 UTC
General Quota Change Notes:

Old Quota:
Groups/users as the quota source.
Quota instance values specify max values for quota items.
Tags can be used to override instance max values.
Vm/Services had separate quota statemachines. 

New Quota:
Tenant is default source for quota. 
Has the ability to use groups/users with an automate method change.
Tenant quota values are not overridden by tag values.
Vm/Services use single common quota statemachine.

For testing purposes:
Setup tenant quota values: (configure/configuration/access control/tenants/tenant quota)
* Clearing all tenant values will cause quotas to pass.
** Can select one, or more quota values. (Allocated number of templates has values in tenant quota, but is not currently used in quota validation)
*** Once the tenant values are set, the tenant quota page will show values for total, in_use, allocated, available. This should help with setting test values for specific use cases.
Create Service items and bundles, if applicable. (both cloud and infrastructure service items/bundles)

Once the reported issue has been validated, we should revert to the old behavior of using group as the quota source to validate backward compatibility.

Using the automate explorer:
1) Copy the /System/CommonMethods/QuotaMethods/quota_source method to a writable enabled domain.

quota_source method.
1.  #
2.  # Description: Set Tenant as the default quota source.
3.  #
4.  
5.  # Sample code to enable group as the default quota source.
6.  # $evm.root['quota_source'] = @miq_request.requester.current_group
7.  # $evm.root['quota_source_type'] = 'group'
8.
9.  @miq_request = $evm.root['miq_request']
10. $evm.log(:info, "Request: #{@miq_request.description} id: #{@miq_request.id} ")
11.
12. $evm.root['quota_source'] = @miq_request.tenant
13. $evm.root['quota_source_type'] = 'tenant'
14.
15. $evm.log(:info, "Setting Quota Source #{$evm.root['quota_source'].inspect}")

2) Edit the copied quota_source method:
   a. Comment out lines 12 and 13 by placing a "#" at the beginning of the line.
   b. Copy lines 6 and 7 to line 11 and uncomment out lines.(remove the "#") 
3) Validate and save method.
4) Copy the /System/CommonMethods/QuotaStateMachine/Quota instance to a writable enabled domain.
5) Modify the copied Quota instance to set desired quota test values. 
6) Run test(Provision service and/or VMs).
7) Once the test has passed, add tag value(s) to override quota instance value(giving it a different value than the instance).
8) Run test(Provision service and/or VMs). Ensure the tag values have been used in the quota decision.

Comment 16 Aziza Karol 2015-12-02 06:32:57 UTC
Provisioning quota for CPU/Mem/Storage/Vms works for cloud providers.
Verified provisioning cloud service and service bundle

When quota exceeds quota validation messages is displayed in request details.

Verified version:5.5.0.12-rc2.20151124135609_653c0d4

The old behavior of using group as the quota source fails.I have open separate ticket and can be tracked in BZ https://bugzilla.redhat.com/show_bug.cgi?id=1286999

Comment 18 errata-xmlrpc 2015-12-08 13:27:51 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2015:2551

Comment 19 William Fitzgerald 2016-05-04 15:52:00 UTC
*** Bug 1253735 has been marked as a duplicate of this bug. ***