Bug 1460977 - performance issue in openstack collection
performance issue in openstack collection
Status: CLOSED CURRENTRELEASE
Product: Red Hat CloudForms Management Engine
Classification: Red Hat
Component: Providers (Show other bugs)
5.7.0
All All
unspecified Severity urgent
: GA
: 5.9.0
Assigned To: Sam Lucidi
Ido Ovadia
openstack
: TestOnly, ZStream
Depends On:
Blocks: 1469702 1469703
  Show dependency treegraph
 
Reported: 2017-06-13 05:54 EDT by Felix Dewaleyne
Modified: 2018-03-06 10:24 EST (History)
16 users (show)

See Also:
Fixed In Version: 5.9.0.1
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1469702 1469703 (view as bug list)
Environment:
Last Closed: 2018-03-06 10:24:49 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: Openstack


Attachments (Terms of Use)

  None (edit)
Description Felix Dewaleyne 2017-06-13 05:54:12 EDT
Description of problem:
Cloudforms refreshes of openstack environment take too long - more than 10 minutes for 74 projects (tenants), managing 247 virtual machines

Version-Release number of selected component (if applicable):
5.7.2

How reproducible:
all the time

Steps to Reproduce:
1. deploy openstack with around 300 vms and 74 tenants
2. deploy cloudforms with one provider connection to openstack
3.

Actual results:
it takes 10 minutes to finish a refresh 

Expected results:
it should take minutes, as few as possible, to meet two requirements 
- have in CFME full overview of OpenStack provider and its configuration items will all relations etc.
- reduce refresh time in case there is new VM discovered or event with its state change is collected 

Additional info:
This issue is blocking further deployment of Openstack and threatening the customer's buisness as is
tests were done with a hotfix (likely through another case) but the version hasn't been confirmed at the time of writing
Comment 11 ITD27M01 2017-10-03 08:49:47 EDT
Hi Guys

Configuration changes have some caveats:

1. I use custom domain in RHOSP 10 with ldap (Active Directory) authentication. By admin user of Default project I cannot collect information about cinder volumes from projects of my not-default domain on the CFME5.8 side (but can do it by openstack cli and api).

2. For heat templates collection the "stacks:global_index" rule should be set to "context_is_admin" in /etc/heat/policy.json configuration file on OpenStack controllers.

Please pay attention of the developers that customers can have not only default domain with default admin user (on openstack side) but custom domain with external auth like ldap or Active Directory.

As a workaround I currently have created admin project in my custom domain and have configured necessary rules in policy.json.
Comment 12 Sam Lucidi 2017-10-03 09:27:39 EDT
Hi,

With regard to point 1, I don't believe the fix associated with this BZ made any changes to cinder inventory collection. Can you elaborate on the behavior you are seeing?

Regarding point 2, Heat does need to have global admin enabled in order to enable fast inventory collection, which is why there's a separate setting to enable Heat speedups in CFME. Is it working for you with "stacks:global_index" set, or are you experiencing further problems?

Thanks,
Sam Lucidi
Comment 13 ITD27M01 2017-10-03 09:37:51 EDT
By setting "stacks:global_index" to "context_is_admin" collection of staks work well.(In reply to Sam Lucidi from comment #12)
> Hi,
> 
> With regard to point 1, I don't believe the fix associated with this BZ made
> any changes to cinder inventory collection. Can you elaborate on the
> behavior you are seeing?
> 
> Regarding point 2, Heat does need to have global admin enabled in order to
> enable fast inventory collection, which is why there's a separate setting to
> enable Heat speedups in CFME. Is it working for you with
> "stacks:global_index" set, or are you experiencing further problems?
> 
> Thanks,
> Sam Lucidi

If I use "Default" domain and "admin" user for provider connection I cannot see any cloud volumes from another domain in CFME. The collection work for tenants/instances/sec groups etc. but not for cloud volumes. At the moment I can not give the appropriate log. But if necessary I'll try to reproduce this issue. Do I need to do this ?

By setting "stacks:global_index" to "context_is_admin" collection of stacks work well.
Comment 14 Sam Lucidi 2017-10-04 13:10:12 EDT
It sounds like the problem you are experiencing with domains might be unrelated to this change, so you might consider opening a new ticket for it. Normally multiple domains are supported in CFME by configuring an Openstack provider for each domain, but it's possible that things might be different depending on how your keystone roles are set up. Please correct me if I'm misunderstanding and your problem is due to the fast refresh patch.
Comment 15 John Hardy 2017-11-15 09:13:03 EST
Whe will this ticket goto POST?
Comment 16 Ido Ovadia 2018-02-18 09:03:26 EST
Can only verified with ~150 tenants ~40 instances

Note You need to log in before you can comment on or make changes to this bug.