Bugzilla (bugzilla.redhat.com) will be under maintenance for infrastructure upgrades and will not be available on July 31st between 12:30 AM - 05:30 AM UTC. We appreciate your understanding and patience. You can follow status.redhat.com for details.
Bug 1497169 - ems_refresh time increases when tries to update inventory for OpenStack10 provider with multiple tenants
Summary: ems_refresh time increases when tries to update inventory for OpenStack10 pro...
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat CloudForms Management Engine
Classification: Red Hat
Component: Providers
Version: 5.7.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: GA
: cfme-future
Assignee: Marek Aufart
QA Contact: Ola Pavlenko
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-09-29 11:05 UTC by Sachin
Modified: 2020-12-14 10:18 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-11-02 03:31:40 UTC
Category: Bug
Cloudforms Team: Openstack
Target Upstream Version:


Attachments (Terms of Use)

Description Sachin 2017-09-29 11:05:22 UTC
Description of problem:

Recently customer added new tenants in OpenStack10 provider from 5 to 140 and observed that the total ems_refresh time has increases ten times(from 300 to 3000 seconds).

Please find below data extracted from current logs from one appliance having 'ems_inventory' role in zone enabled. The collect_inventory_for_target and save_inventory is under limit but parse_legacy_inventory & parse_targeted_inventory is much higher.


| MessageID          | collect_inventory_for_target | parse_legacy_inventory | parse_targeted_inventory | save_inventory | ems_refresh |
|--------------------+------------------------------+------------------------+--------------------------+----------------+-------------|
| 16450095(full)     |                     0.000004 |            2471.296330 |              2471.296341 |       8.327366 | 2479.623937 |
| 16452364(full)     |                     0.000004 |            3032.477864 |              3032.477921 |       7.695983 | 3040.174192 |
| 16454435(full)     |                     0.000003 |            2988.104470 |              2988.104479 |       6.870958 | 2994.975635 |
| 16457320(targeted) |                     0.000003 |            2551.129330 |              2551.129341 |       8.136193 | 2559.265796 |
| 16449276(full)     |                     0.000004 |            5437.989589 |              5437.989606 |       7.208448 | 5445.198315 |
| 16450297(full)     |                     0.000005 |            5753.051083 |              5753.051096 |       3.397968 | 5756.449312 |


Customer has also provided list of methods spent most time,

- get_tenants
- get_flavors
- get_vailability_zones
- get_host_aggregates
- get_quotas
- get_key_pairs
- load_orchestration_stacks
- get_vnfs
- get_vnfds
- get_images
- get_servers
- get_cloud_services


Version-Release number of selected component (if applicable):
5.7.3.2

How reproducible:
Always

Steps to Reproduce:
1. 
2.
3.

Actual results:


Expected results:


Additional info:
- Newly added projects(135) do not have any VMs or other object. Basically they are empty.

Comment 4 Tzu-Mainn Chen 2017-10-02 12:37:26 UTC
Ah, I see! I think this may be related to this other BZ then:
https://bugzilla.redhat.com/show_bug.cgi?id=1460977. The fix for that BZ creates a config option that allows the refresh code to query all tenants at once for each query. This only works if the user is the 'admin' user. Would that fix your issue?

Comment 5 ITD27M01 2017-10-02 13:18:47 UTC
(In reply to Tzu-Mainn Chen from comment #4)
> Ah, I see! I think this may be related to this other BZ then:
> https://bugzilla.redhat.com/show_bug.cgi?id=1460977. The fix for that BZ
> creates a config option that allows the refresh code to query all tenants at
> once for each query. This only works if the user is the 'admin' user. Would
> that fix your issue?

Currently these ems settings is available only in CFME 5.8 and work if user is domain admin. I decided to upgrade 5.7 to 5.8 because I have no time to wait of backport for this bug.

For 'admin' user from default domain the collection of Cinder cloud volumes from another domains does not work. Do not forget that customers can use not only default domain but the domain with external authentication (like Active Directory). 

For this reason I create admin project in my custom domain and configure related advanced settings:

~~~~~~~~~~~~~~~~~~~~
:ems:
  :ems_openstack:
    :excon:
      :omit_default_port: true
      :read_timeout: 600
    :refresh:
      :is_admin: true  <----- Need to support fast refresh  by one api call
      :heat:
        :is_global_admin: true <----- Need to support fast refresh by one api call
~~~~~~~~~~~~~~~~~~~~

Currently the refresh timing in acceptable range - 1,5 to 3 minutes for CFME 5.8.1

Comment 8 Sachin 2017-11-02 03:31:40 UTC
Yes. Closing BZ as the fix is available in 5.8


Note You need to log in before you can comment on or make changes to this bug.