Bug 1455844 - Service template provisioning request do not honour quotas
Summary: Service template provisioning request do not honour quotas
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat CloudForms Management Engine
Classification: Red Hat
Component: Automate
Version: 5.7.0
Hardware: Unspecified
OS: Unspecified
medium
high
Target Milestone: GA
: 5.9.0
Assignee: William Fitzgerald
QA Contact: Vatsal Parekh
URL:
Whiteboard: quota:service
Depends On:
Blocks: 1465087 1465088
TreeView+ depends on / blocked
 
Reported: 2017-05-26 09:57 UTC by Sachin
Modified: 2018-03-06 15:04 UTC (History)
9 users (show)

Fixed In Version: 5.9.0.1
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1465087 1465088 (view as bug list)
Environment:
Last Closed: 2018-03-06 15:04:43 UTC
Category: ---
Cloudforms Team: ---
Target Upstream Version:


Attachments (Terms of Use)

Description Sachin 2017-05-26 09:57:55 UTC
Description of problem:

Create a service catalog with vm_name, instance_type & number_of_vms as fields. Set quotas threshold values for number_of_vms to 5 and provision service catalog with vm count as 10. Below is what I found in evm.logs

1. Using: Lifecycle -> Provision instances
~~~
[----] I, [2017-05-24T14:43:25.353134 #7375:d71130]  INFO -- : Q-task_id([miq_provision_request_21000000000010]) <AutomationEngine> <AEMethod [/ManageIQ/System/CommonMethods/QuotaMethods/validate_quota]> Starting 
[----] I, [2017-05-24T14:43:25.714563 #7375:d6f7a4]  INFO -- : Q-task_id([miq_provision_request_21000000000010]) <AutomationEngine> <AEMethod validate_quota> Request: Provision from [cirros] to [vm-###] id: 21000000000010 
[----] I, [2017-05-24T14:43:25.716793 #7375:d6f7a4]  INFO -- : Q-task_id([miq_provision_request_21000000000010]) <AutomationEngine> <AEMethod validate_quota> quota_warning: {:storage=>1073741824, :vms=>3, :cpu=>3, :memory=>1073741824}
[----] I, [2017-05-24T14:43:25.717771 #7375:d6f7a4]  INFO -- : Q-task_id([miq_provision_request_21000000000010]) <AutomationEngine> <AEMethod validate_quota> quota_limits: {:storage=>2147483648, :vms=>5, :cpu=>5, :memory=>2147483648}
[----] I, [2017-05-24T14:43:25.718700 #7375:d6f7a4]  INFO -- : Q-task_id([miq_provision_request_21000000000010]) <AutomationEngine> <AEMethod validate_quota> Item: storage Used: (0) Requested: (1073741824) Max: (2147483648) Warn: (1073741824)
[----] I, [2017-05-24T14:43:25.719602 #7375:d6f7a4]  INFO -- : Q-task_id([miq_provision_request_21000000000010]) <AutomationEngine> <AEMethod validate_quota> Item: vms Used: (0) Requested: (10) Max: (5) Warn: (3)
[----] I, [2017-05-24T14:43:25.720499 #7375:d6f7a4]  INFO -- : Q-task_id([miq_provision_request_21000000000010]) <AutomationEngine> <AEMethod validate_quota> Quota vms exceeded: Used(0) + Requested(10) > Quota(5)
[----] I, [2017-05-24T14:43:25.721406 #7375:d6f7a4]  INFO -- : Q-task_id([miq_provision_request_21000000000010]) <AutomationEngine> <AEMethod validate_quota> Quota maximum exceeded for key: vms_quota_exceeded reason: vms - Used: 0 plus requested: 10 exceeds quota: 5
[----] I, [2017-05-24T14:43:25.722296 #7375:d6f7a4]  INFO -- : Q-task_id([miq_provision_request_21000000000010]) <AutomationEngine> <AEMethod validate_quota> Item: cpu Used: (0) Requested: (1) Max: (5) Warn: (3)
[----] I, [2017-05-24T14:43:25.723255 #7375:d6f7a4]  INFO -- : Q-task_id([miq_provision_request_21000000000010]) <AutomationEngine> <AEMethod validate_quota> Item: memory Used: (0) Requested: (536870912) Max: (2147483648) Warn: (1073741824)
[----] I, [2017-05-24T14:43:25.794252 #7375:d71130]  INFO -- : Q-task_id([miq_provision_request_21000000000010]) <AutomationEngine> <AEMethod [/ManageIQ/System/CommonMethods/QuotaMethods/validate_quota]> Ending
[----] I, [2017-05-24T14:43:25.794990 #7375:d71130]  INFO -- : Q-task_id([miq_provision_request_21000000000010]) <AutomationEngine> Followed  Relationship [miqaedb:/System/CommonMethods/QuotaMethods/validate_quota#create]
[----] I, [2017-05-24T14:43:25.833977 #7375:d71130]  INFO -- : Q-task_id([miq_provision_request_21000000000010]) <AutomationEngine> Updated namespace [System/CommonMethods/QuotaStateMachine/rejected  ManageIQ/System/CommonMethods]
[----] I, [2017-05-24T14:43:25.850609 #7375:d71130]  INFO -- : Q-task_id([miq_provision_request_21000000000010]) <AutomationEngine> Invoking [inline] method [/ManageIQ/System/CommonMethods/QuotaStateMachine/rejected] with inputs [{}]
[----] I, [2017-05-24T14:43:25.851414 #7375:d71130]  INFO -- : Q-task_id([miq_provision_request_21000000000010]) <AutomationEngine> <AEMethod [/ManageIQ/System/CommonMethods/QuotaStateMachine/rejected]> Starting 
[----] I, [2017-05-24T14:43:26.164448 #7375:4f8ac88]  INFO -- : Q-task_id([miq_provision_request_21000000000010]) <AutomationEngine> <AEMethod rejected> Request denied because of Request exceeds maximum allowed for the following: (vms - Used: 0 plus requested: 10 exceeds quota: 5) 
[----] I, [2017-05-24T14:43:26.313724 #7375:d71130]  INFO -- : Q-task_id([miq_provision_request_21000000000010]) <AutomationEngine> <AEMethod [/ManageIQ/System/CommonMethods/QuotaStateMachine/rejected]> Ending
~~~

2. Using service catalog
~~~
[----] I, [2017-05-24T14:18:57.658155 #7470:d71130]  INFO -- : Q-task_id([service_template_provision_request_21000000000009]) <AutomationEngine> <AEMethod [/ManageIQ/System/CommonMethods/QuotaMethods/validate_quota]> Starting 
[----] I, [2017-05-24T14:18:58.017984 #7470:d6b424]  INFO -- : Q-task_id([service_template_provision_request_21000000000009]) <AutomationEngine> <AEMethod validate_quota> Request: Provisioning Service [Instance provision] from [Instance provision] id: 21000000000009 
[----] I, [2017-05-24T14:18:58.019195 #7470:d6b424]  INFO -- : Q-task_id([service_template_provision_request_21000000000009]) <AutomationEngine> <AEMethod validate_quota> quota_warning: {:storage=>1073741824, :vms=>3, :cpu=>3, :memory=>1073741824}
[----] I, [2017-05-24T14:18:58.019697 #7470:d6b424]  INFO -- : Q-task_id([service_template_provision_request_21000000000009]) <AutomationEngine> <AEMethod validate_quota> quota_limits: {:storage=>2147483648, :vms=>5, :cpu=>5, :memory=>2147483648}
[----] I, [2017-05-24T14:18:58.020212 #7470:d6b424]  INFO -- : Q-task_id([service_template_provision_request_21000000000009]) <AutomationEngine> <AEMethod validate_quota> Item: storage Used: (0) Requested: (1073741824) Max: (2147483648) Warn: (1073741824)
[----] I, [2017-05-24T14:18:58.020724 #7470:d6b424]  INFO -- : Q-task_id([service_template_provision_request_21000000000009]) <AutomationEngine> <AEMethod validate_quota> Item: vms Used: (0) Requested: (1) Max: (5) Warn: (3)
[----] I, [2017-05-24T14:18:58.021193 #7470:d6b424]  INFO -- : Q-task_id([service_template_provision_request_21000000000009]) <AutomationEngine> <AEMethod validate_quota> Item: cpu Used: (0) Requested: (1) Max: (5) Warn: (3)
[----] I, [2017-05-24T14:18:58.021762 #7470:d6b424]  INFO -- : Q-task_id([service_template_provision_request_21000000000009]) <AutomationEngine> <AEMethod validate_quota> Item: memory Used: (0) Requested: (536870912) Max: (2147483648) Warn: (1073741824)
[----] I, [2017-05-24T14:18:58.038831 #7470:d71130]  INFO -- : Q-task_id([service_template_provision_request_21000000000009]) <AutomationEngine> <AEMethod [/ManageIQ/System/CommonMethods/QuotaMethods/validate_quota]> Ending
~~~


Value of 'number_of_vms' turns out to be 1 when below lines where inserted within code.
(File: https://github.com/ManageIQ/manageiq/blob/euwe/db/fixtures/ae_datastore/ManageIQ/System/CommonMethods/QuotaMethods.class/__methods__/requested.rb)
~~~
--- requested.rb	2017-05-26 11:53:16.019079897 +0530
+++ requested_mod.rb	2017-05-26 14:39:52.521920486 +0530
@@ -5,6 +5,8 @@
     else
       next if @service_template.prov_type.starts_with?("generic")
       service_prov_option_value(prov_option, child_service_resource.resource, options_array)
+      $evm.log(:info, "CASE :number_of_vms #{child_service_resource.resource.get_option(:number_of_vms)}")
+      $evm.log(:info, "CASE :number_of_vms #{child_service_resource.resource.inspect}")
     end
   end
   options_array
~~~

Output:
~~~
[----] I, [2017-05-26T13:00:53.726396 #28601:ed5c10]  INFO -- : Q-task_id([service_template_provision_request_21000000000038]) <AutomationEngine> <AEMethod requested> CASE :number_of_vms 1
[----] I, [2017-05-26T13:00:53.728507 #28601:ed5c10]  INFO -- : Q-task_id([service_template_provision_request_21000000000038]) <AutomationEngine> <AEMethod requested> CASE :number_of_vms #<MiqAeServiceMiqProvisionRequestTemplate:0x5f963fc @object=#<MiqProvisionRequestTemplate id: 21000000000030, description: "Provision from [cirros] to [cfme###]", approval_state: "pending_approval", type: "MiqProvisionRequestTemplate", created_on: "2017-05-25 20:00:26", updated_on: "2017-05-25 20:30:11", fulfilled_on: nil, requester_id: 21000000000001, requester_name: "Administrator", request_type: "template", request_state: "pending", message: "VM Provisioning - Request Created", status: "Ok", options: {:initial_pass=>true, :service_template_request=>true, :miq_request_dialog_name=>"miq_provision_openstack_dialogs_template", :customize_enabled=>["enabled"], :requester_enabled=>["disabled"], :purpose_enabled=>["disabled"], :current_tab_key=>:customize, :owner_phone=>nil, :owner_country=>nil, :owner_phone_mobile=>nil, :owner_title=>nil, :owner_first_name=>nil, :owner_manager=>nil, :owner_address=>nil, :owner_company=>nil, :owner_last_name=>nil, :owner_manager_mail=>nil, :owner_city=>nil, :owner_department=>nil, :owner_load_ldap=>nil, :owner_manager_phone=>nil, :owner_state=>nil, :owner_office=>nil, :owner_zip=>nil, :owner_email=>nil, :request_notes=>nil, :vm_tags=>[], :placement_auto=>[false, 0], :placement_availability_zone=>[21000000000001, "nova"], :cloud_tenant=>[21000000000001, "admin"], :cloud_network=>[nil, nil], :security_groups=>[21000000000002, "default: Default security group"], :floating_ip_address=>[nil, nil], :number_of_vms=>[1, "1"], :vm_description=>nil, :vm_prefix=>nil, :src_vm_id=>[21000000000001, "cirros"], :vm_name=>"changeme", :schedule_type=>["immediately", "Immediately on Approval"], :schedule_time=>Fri, 26 May 2017 19:59:00 UTC +00:00, :retirement=>[2592000, "1 Month"], :retirement_warn=>[604800, "1 Week"], :instance_type=>[21000000000001, "m1.tiny"], :guest_access_key_pair=>[21000000000004, "psachin-key"], :name=>"Provision instances", :size=>nil, :delete_on_terminate=>false, :dns_servers=>nil, :dns_suffixes=>nil, :addr_mode=>["dhcp", "DHCP"], :linux_host_name=>nil, :gateway=>nil, :linux_domain_name=>nil, :subnet_mask=>nil, :customization_template_script=>"#cloud-config\nchpasswd:\n  list: |\n    root:<%= MiqPassword.decrypt(evm[:root_password]) %>\n  expire: False", :root_password=>"v2:{QqWAxpXPeS8HNk84LmMyzw==}", :hostname=>nil, :start_date=>"5/26/2017", :start_hour=>"19", :start_min=>"59", :description=>"Provision Instances", :long_description=>"", :provision_cost=>nil, :display=>true, :catalog_id=>21000000000001, :st_prov_type=>"openstack", :available_catalogs=>[["OpenStack", 21000000000001]], :retire_fqname=>"/Service/Retirement/StateMachines/ServiceRetirement/Default", :reconfigure_fqname=>"", :fqname=>"/Service/Provisioning/StateMachines/ServiceProvision_Template/CatalogItemInitialization", :available_dialogs=>{21000000000001=>"azure-single-vm-from-user-image", 21000000000002=>"OpenStack"}, :service_type=>"atomic", :dialog_id=>21000000000002, :src_vm_nics=>[], :src_vm_lans=>[], :src_ems_id=>[21000000000003, "RHOSP-11-quickvm"], :customization_template_id=>[21000000000007, "Basic root pass template"], :volumes=>[], :requester_group=>"EvmGroup-super_administrator", :delivered_on=>nil}, userid: "admin", source_id: 21000000000001, source_type: "VmOrTemplate", destination_id: nil, destination_type: nil, tenant_id: 21000000000001, service_order_id: nil, process: true>, @virtual_columns=["provision_type", "reason", "region_description", "region_number", "request_type_display", "resource_type", "stamped_on", "state", "v_approved_by", "v_approved_by_email"], @associations=["approvers", "destination", "eligible_clusters", "eligible_customization_templates", "eligible_folders", "eligible_hosts", "eligible_iso_images", "eligible_pxe_images", "eligible_pxe_servers", "eligible_resource_pools", "eligible_storages", "eligible_windows_images", "miq_provisions", "miq_request", "miq_request", "miq_request_tasks", "requester", "resource", "source", "tenant", "vm_template"]>
[----] I, [2017-05-26T13:00:53.729131 #28601:d73e08]  INFO -- : Q-task_id([service_template_provision_request_21000000000038]) <AutomationEngine> <AEMethod requested> CASE collect_dialog_totals() prov_option: number_of_vms
[----] I, [2017-05-26T13:00:53.729609 #28601:d73e08]  INFO -- : Q-task_id([service_template_provision_request_21000000000038]) <AutomationEngine> <AEMethod requested> CASE dialog_values() options: {:dialog_vm_name=>"nh", :dialog_number_of_vms=>12, :dialog_instance_type=>"m1.tiny"}
~~~


Version-Release number of selected component (if applicable):
CFME-5.7.2.1

How reproducible:
Always 


Actual results:


Expected results:


Additional info:

Comment 4 Sachin 2017-06-07 05:07:03 UTC
Value of 'number_of_vms' turns out to be 1 when below lines where inserted within code
~~~
--- requested.rb	2017-05-26 11:53:16.019079897 +0530
+++ requested_mod.rb	2017-05-26 14:39:52.521920486 +0530
@@ -5,6 +5,8 @@
     else
       next if @service_template.prov_type.starts_with?("generic")
       service_prov_option_value(prov_option, child_service_resource.resource, options_array)
+      $evm.log(:info, "CASE :number_of_vms #{child_service_resource.resource.get_option(:number_of_vms)}")
+      $evm.log(:info, "CASE :number_of_vms #{child_service_resource.resource.inspect}")
     end
   end
   options_array
~~~

Output:
~~~
[----] I, [2017-05-26T13:00:53.726396 #28601:ed5c10]  INFO -- : Q-task_id([service_template_provision_request_21000000000038]) <AutomationEngine> <AEMethod requested> CASE :number_of_vms 1
[----] I, [2017-05-26T13:00:53.728507 #28601:ed5c10]  INFO -- : Q-task_id([service_template_provision_request_21000000000038]) <AutomationEngine> <AEMethod requested> CASE :number_of_vms #<MiqAeServiceMiqProvisionRequestTemplate:0x5f963fc @object=#<MiqProvisionRequestTemplate id: 21000000000030, description: "Provision from [cirros] to [cfme###]", approval_state: "pending_approval", type: "MiqProvisionRequestTemplate", created_on: "2017-05-25 20:00:26", updated_on: "2017-05-25 20:30:11", fulfilled_on: nil, requester_id: 21000000000001, requester_name: "Administrator", request_type: "template", request_state: "pending", message: "VM Provisioning - Request Created", status: "Ok", options: {:initial_pass=>true, :service_template_request=>true, :miq_request_dialog_name=>"miq_provision_openstack_dialogs_template", :customize_enabled=>["enabled"], :requester_enabled=>["disabled"], :purpose_enabled=>["disabled"], :current_tab_key=>:customize, :owner_phone=>nil, :owner_country=>nil, :owner_phone_mobile=>nil, :owner_title=>nil, :owner_first_name=>nil, :owner_manager=>nil, :owner_address=>nil, :owner_company=>nil, :owner_last_name=>nil, :owner_manager_mail=>nil, :owner_city=>nil, :owner_department=>nil, :owner_load_ldap=>nil, :owner_manager_phone=>nil, :owner_state=>nil, :owner_office=>nil, :owner_zip=>nil, :owner_email=>nil, :request_notes=>nil, :vm_tags=>[], :placement_auto=>[false, 0], :placement_availability_zone=>[21000000000001, "nova"], :cloud_tenant=>[21000000000001, "admin"], :cloud_network=>[nil, nil], :security_groups=>[21000000000002, "default: Default security group"], :floating_ip_address=>[nil, nil], :number_of_vms=>[1, "1"], :vm_description=>nil, :vm_prefix=>nil, :src_vm_id=>[21000000000001, "cirros"], :vm_name=>"changeme", :schedule_type=>["immediately", "Immediately on Approval"], :schedule_time=>Fri, 26 May 2017 19:59:00 UTC +00:00, :retirement=>[2592000, "1 Month"], :retirement_warn=>[604800, "1 Week"], :instance_type=>[21000000000001, "m1.tiny"], :guest_access_key_pair=>[21000000000004, "psachin-key"], :name=>"Provision instances", :size=>nil, :delete_on_terminate=>false, :dns_servers=>nil, :dns_suffixes=>nil, :addr_mode=>["dhcp", "DHCP"], :linux_host_name=>nil, :gateway=>nil, :linux_domain_name=>nil, :subnet_mask=>nil, :customization_template_script=>"#cloud-config\nchpasswd:\n  list: |\n    root:<%= MiqPassword.decrypt(evm[:root_password]) %>\n  expire: False", :root_password=>"v2:{QqWAxpXPeS8HNk84LmMyzw==}", :hostname=>nil, :start_date=>"5/26/2017", :start_hour=>"19", :start_min=>"59", :description=>"Provision Instances", :long_description=>"", :provision_cost=>nil, :display=>true, :catalog_id=>21000000000001, :st_prov_type=>"openstack", :available_catalogs=>[["OpenStack", 21000000000001]], :retire_fqname=>"/Service/Retirement/StateMachines/ServiceRetirement/Default", :reconfigure_fqname=>"", :fqname=>"/Service/Provisioning/StateMachines/ServiceProvision_Template/CatalogItemInitialization", :available_dialogs=>{21000000000001=>"azure-single-vm-from-user-image", 21000000000002=>"OpenStack"}, :service_type=>"atomic", :dialog_id=>21000000000002, :src_vm_nics=>[], :src_vm_lans=>[], :src_ems_id=>[21000000000003, "RHOSP-11-quickvm"], :customization_template_id=>[21000000000007, "Basic root pass template"], :volumes=>[], :requester_group=>"EvmGroup-super_administrator", :delivered_on=>nil}, userid: "admin", source_id: 21000000000001, source_type: "VmOrTemplate", destination_id: nil, destination_type: nil, tenant_id: 21000000000001, service_order_id: nil, process: true>, @virtual_columns=["provision_type", "reason", "region_description", "region_number", "request_type_display", "resource_type", "stamped_on", "state", "v_approved_by", "v_approved_by_email"], @associations=["approvers", "destination", "eligible_clusters", "eligible_customization_templates", "eligible_folders", "eligible_hosts", "eligible_iso_images", "eligible_pxe_images", "eligible_pxe_servers", "eligible_resource_pools", "eligible_storages", "eligible_windows_images", "miq_provisions", "miq_request", "miq_request", "miq_request_tasks", "requester", "resource", "source", "tenant", "vm_template"]>
[----] I, [2017-05-26T13:00:53.729131 #28601:d73e08]  INFO -- : Q-task_id([service_template_provision_request_21000000000038]) <AutomationEngine> <AEMethod requested> CASE collect_dialog_totals() prov_option: number_of_vms
[----] I, [2017-05-26T13:00:53.729609 #28601:d73e08]  INFO -- : Q-task_id([service_template_provision_request_21000000000038]) <AutomationEngine> <AEMethod requested> CASE dialog_values() options: {:dialog_vm_name=>"nh", :dialog_number_of_vms=>12, :dialog_instance_type=>"m1.tiny"}
~~~

Comment 6 CFME Bot 2017-06-21 19:08:26 UTC
New commit detected on ManageIQ/manageiq-content/master:
https://github.com/ManageIQ/manageiq-content/commit/88a75a6f79efbe802f1c35a90baf9b87b60b1275

commit 88a75a6f79efbe802f1c35a90baf9b87b60b1275
Author:     william fitzgerald <wfitzger@redhat.com>
AuthorDate: Thu Jun 15 14:28:16 2017 -0400
Commit:     william fitzgerald <wfitzger@redhat.com>
CommitDate: Wed Jun 21 14:28:10 2017 -0400

    Fixed quota calculations for multiple vms in requested method.
    
    Values for cloud providers were incorrect when multiple vms were selected.
    Included Azure to cloud providers which was missing.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1455844

 .../QuotaMethods.class/__methods__/requested.rb    | 33 +++++++++++-----------
 .../method_validation/calculate_requested_spec.rb  | 18 ++++++++++++
 2 files changed, 35 insertions(+), 16 deletions(-)

Comment 9 Vatsal Parekh 2017-12-29 09:22:41 UTC
Able to verify this on Version 5.9.0.14.20171219165537_e3dea29


Note You need to log in before you can comment on or make changes to this bug.