Bug 1456819 - Simultaneous service catalog request do not honour quotas
Summary: Simultaneous service catalog request do not honour quotas
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat CloudForms Management Engine
Classification: Red Hat
Component: Automate
Version: 5.7.0
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: GA
: 5.10.0
Assignee: Tina Fitzgerald
QA Contact: Ganesh Hubale
URL:
Whiteboard: quota:service
: 1449834 (view as bug list)
Depends On:
Blocks: 1489507 1536677
TreeView+ depends on / blocked
 
Reported: 2017-05-30 12:33 UTC by Sachin
Modified: 2019-02-11 13:54 UTC (History)
17 users (show)

Fixed In Version: 5.10.0.0
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1489507 1536677 (view as bug list)
Environment:
Last Closed: 2019-02-11 13:54:48 UTC
Category: ---
Cloudforms Team: ---
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 1581288 0 high CLOSED [RFE] Service Dialogs - Calculate Quota for instance_type dialog override. 2021-02-22 00:41:40 UTC

Internal Links: 1581288

Description Sachin 2017-05-30 12:33:21 UTC
Description of problem:

If two users from same group try to provision service catalog from different web-sessions, quotas are not honoured.

Created a service catalog with vm_name, instance_type & number_of_vms as fields. Set quotas threshold values for number_of_vms to 5.

If both user order 5 vms each, request is not denied by quota validation.


Version-Release number of selected component (if applicable):
5.7


How reproducible:
Always


Steps to Reproduce:
As above

Actual results:


Expected results:


Additional info:

Comment 2 ITD27M01 2017-05-30 13:28:48 UTC
Hi guys.

I should add that this problem is not only in case of simultaneous requests. Used resources are calculated only after the provisioning of a virtual machine or cloud instance. This means that all provisioning request executed in the time interval between sending the request and the end of the provisioning are met. Even if the total quota is negative in the end.

Comment 8 Tina Fitzgerald 2017-08-01 13:41:12 UTC
Hi Marianne,

Sorry for the delayed response.

I marked the PR as WIP because we realized there was more work to do for
 in-flight Service bundle and item provisions.
I added commits for the Service bundle and item calculations yesterday and will keep you posted on our progress.

Thanks,
Tina

Comment 9 CFME Bot 2017-08-21 20:06:33 UTC
New commit detected on ManageIQ/manageiq/master:
https://github.com/ManageIQ/manageiq/commit/fa19836f545fbeed012d3940455cfddf4cc18a95

commit fa19836f545fbeed012d3940455cfddf4cc18a95
Author:     Tina Fitzgerald <tfitzger@redhat.com>
AuthorDate: Mon Jun 26 17:41:27 2017 -0400
Commit:     Tina Fitzgerald <tfitzger@redhat.com>
CommitDate: Wed Aug 2 14:04:51 2017 -0400

    Modified active prov methods and added by_group and by_tenant active
    methods.  Added tests.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1456819

 app/models/mixins/miq_provision_quota_mixin.rb | 138 ++++++++++++++------
 spec/models/miq_provision_request_spec.rb      | 171 ++++++++++++++++++++++++-
 2 files changed, 271 insertions(+), 38 deletions(-)

Comment 13 Lucy Fu 2017-09-07 18:17:40 UTC
*** Bug 1449834 has been marked as a duplicate of this bug. ***

Comment 14 Vatsal Parekh 2017-10-10 08:33:31 UTC
As per comments on https://bugzilla.redhat.com/show_bug.cgi?id=1489507
If this was meant to be provisioning VMs using service catalog and selecting number of VMs from catalog, and not from dialog, then it works.

Comment 15 Vatsal Parekh 2017-10-10 10:33:29 UTC
Works with same and different users for simultaneous requests from simultaneous sessions.

Comment 16 CFME Bot 2017-10-13 14:16:55 UTC
New commit detected on ManageIQ/manageiq-content/master:
https://github.com/ManageIQ/manageiq-content/commit/5d4c745730b90ef9891cc76770f08a475bb2dab7

commit 5d4c745730b90ef9891cc76770f08a475bb2dab7
Author:     william fitzgerald <wfitzger@redhat.com>
AuthorDate: Tue Oct 10 09:29:54 2017 -0400
Commit:     william fitzgerald <wfitzger@redhat.com>
CommitDate: Wed Oct 11 12:05:08 2017 -0400

    Added active provisions to quota count.
    
    Modified used method to count active provisions for quota.
    Added log messages to method.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1456819
    
    Log messages display counts and type of quota.
    Sample log messages below:
    
    <AEMethod used> Quota Used: {:cpu=>373, :memory=>922780434432, :vms=>141, :storage=>9111136632832, :provisioned_storage=>10033917067264}
    
    <AEMethod used> Quota active_provisions_by_tenant: {:cpu=>1, :memory=>1073741824, :vms=>1, :storage=>44023414784, :provisioned_storage=>0}
    <AEMethod used> Quota Totals: {:cpu=>374, :memory=>923854176256, :vms=>142, :storage=>9155160047616, :provisioned_storage=>10033917067264}
    
    <AEMethod used> Quota Used: {:cpu=>1, :memory=>1073741824, :vms=>1, :storage=>8589934592, :provisioned_storage=>9663676416}
    <AEMethod used> Quota active_provisions_by_group: {:cpu=>1, :memory=>1073741824, :vms=>1, :storage=>44023414784, :provisioned_storage=>0}
    <AEMethod used> Quota Totals: {:cpu=>2, :memory=>2147483648, :vms=>2, :storage=>52613349376, :provisioned_storage=>9663676416}
    
    <AEMethod used> Quota Used: {:cpu=>1, :memory=>1073741824, :vms=>1, :storage=>8589934592, :provisioned_storage=>9663676416}
    <AEMethod used> Quota active_provisions_by_owner: {:cpu=>0, :memory=>0, :vms=>0, :storage=>0, :provisioned_storage=>0}
    <AEMethod used> Quota Totals: {:cpu=>1, :memory=>1073741824, :vms=>1, :storage=>8589934592, :provisioned_storage=>9663676416}

 .../QuotaMethods.class/__methods__/used.rb         | 29 +++++++++++++++++++-
 .../unit/method_validation/calculate_used_spec.rb  | 26 ------------------
 .../QuotaMethods.class/__methods__/used_spec.rb    | 32 +++++++++++++++++-----
 3 files changed, 53 insertions(+), 34 deletions(-)
 delete mode 100644 spec/automation/unit/method_validation/calculate_used_spec.rb

Comment 18 Tina Fitzgerald 2017-12-18 20:39:43 UTC
Hi Vatsal,

Our current Quota implementation does a best effort to calculate active provisions. 

The active provision code looks for MiqQueue entries with:

1. class_name: MiqProvisionRequest or ServiceTemplateProvisionRequest
   method_name: create_request_tasks
   state: dequeue
or
2. class_name: MiqAeEngine
   method_name: deliver
   tracking_label includes: "_provision_"

It is possible that when 2 requests are submitted at exactly the same time, one of the requests could have an MiqQueue entry that falls outside the above criteria.  As you saw when you staggered the requests, the active provisions were successfully calculated. 

We have plans for enhancing quota in the next major release, but the current MiqQueue calculations have this very small window where the request would not be included as you have described. 

The main issue reported here was no active provisions were included in quota at all. Since the code has been changed to calculate active provisions, can we validate this ticket?

Let me know if you have any questions.

Thanks,
Tina

Comment 19 Vatsal Parekh 2017-12-19 12:37:31 UTC
(In reply to Tina Fitzgerald from comment #18)
> Hi Vatsal,
> 
> Our current Quota implementation does a best effort to calculate active
> provisions. 
> 
> The active provision code looks for MiqQueue entries with:
> 
> 1. class_name: MiqProvisionRequest or ServiceTemplateProvisionRequest
>    method_name: create_request_tasks
>    state: dequeue
> or
> 2. class_name: MiqAeEngine
>    method_name: deliver
>    tracking_label includes: "_provision_"
> 
> It is possible that when 2 requests are submitted at exactly the same time,
> one of the requests could have an MiqQueue entry that falls outside the
> above criteria.  As you saw when you staggered the requests, the active
> provisions were successfully calculated. 
> 
> We have plans for enhancing quota in the next major release, but the current
> MiqQueue calculations have this very small window where the request would
> not be included as you have described. 
> 
> The main issue reported here was no active provisions were included in quota
> at all. Since the code has been changed to calculate active provisions, can
> we validate this ticket?
> 
> Let me know if you have any questions.
> 
> Thanks,
> Tina

Hey Tina,
According to the BZ reported here, the issue is not reported for requests being 1 minute apart, it says specifically for simultaneous requests from 2 different user at the same time.
And for that scenario, I still find quota failing.

Comment 20 Satoe Imaishi 2018-01-19 21:57:46 UTC
https://github.com/ManageIQ/manageiq/pull/16831

Comment 23 Marianne Feifer 2018-03-26 13:39:48 UTC
Hey, Satoe.  Can you please explain to Niladri why this went back to ASSIGNED from POST? PR was merged.

Comment 24 Satoe Imaishi 2018-03-26 14:55:37 UTC
I think I moved back because the fix didn't resolve the issue as per cfme-5.9 clone bug #1536677.

Comment 25 Tina Fitzgerald 2018-03-26 15:54:18 UTC
That is correct. The reason this is back in assigned is because the it involves the override of instance_type & number of vms in the Service dialog. We've resolved the simultaneous quota issue without the Service dialog overrides.

Comment 26 Tina Fitzgerald 2018-05-21 21:42:21 UTC
This issue is resolved with the exception of using the Service dialog override of instance_type which is not currently supported.

We can open a new ticket(RFE) to track the instance_type override.

Setting ticket to POST.

Comment 27 Tina Fitzgerald 2018-05-21 21:46:19 UTC
This PR is also required:
https://github.com/ManageIQ/manageiq-content/pull/203

Comment 28 Anurag 2018-06-25 07:37:47 UTC
Hi,

I have verified this on version: Version 5.10.0.1.20180619163011_900fdc4 and it's working fine. Thanks for resolving this issue, hence changing the status from on_qa to verified.

Thanks,
Anurag

Comment 29 Anurag 2018-06-25 07:38:38 UTC
Hi,

I have verified this on version: Version 5.10.0.1.20180619163011_900fdc4 and it's working fine. Thanks for resolving this issue, hence changing the status from on_qa to verified.

Thanks,
Anurag


Note You need to log in before you can comment on or make changes to this bug.