Bug 1460977

Summary:	performance issue in openstack collection
Product:	Red Hat CloudForms Management Engine	Reporter:	Felix Dewaleyne <fdewaley>
Component:	Providers	Assignee:	Sam Lucidi <slucidi>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Ido Ovadia <iovadia>
Severity:	urgent	Docs Contact:
Priority:	unspecified
Version:	5.7.0	CC:	aarapov, cpelland, fdewaley, gblomqui, igortiunov, iovadia, jfrey, jhardy, jocarter, mfeifer, obarenbo, opavlenk, sacpatil, simaishi, slucidi, vcojot
Target Milestone:	GA	Keywords:	TestOnly, ZStream
Target Release:	5.9.0
Hardware:	All
OS:	All
Whiteboard:	openstack
Fixed In Version:	5.9.0.1	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:
Clones:	1469702 1469703 (view as bug list)		Environment:
Last Closed:	2018-03-06 15:24:49 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	Openstack	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1469702, 1469703

Description Felix Dewaleyne 2017-06-13 09:54:12 UTC

Description of problem:
Cloudforms refreshes of openstack environment take too long - more than 10 minutes for 74 projects (tenants), managing 247 virtual machines

Version-Release number of selected component (if applicable):
5.7.2

How reproducible:
all the time

Steps to Reproduce:
1. deploy openstack with around 300 vms and 74 tenants
2. deploy cloudforms with one provider connection to openstack
3.

Actual results:
it takes 10 minutes to finish a refresh 

Expected results:
it should take minutes, as few as possible, to meet two requirements 
- have in CFME full overview of OpenStack provider and its configuration items will all relations etc.
- reduce refresh time in case there is new VM discovered or event with its state change is collected 

Additional info:
This issue is blocking further deployment of Openstack and threatening the customer's buisness as is
tests were done with a hotfix (likely through another case) but the version hasn't been confirmed at the time of writing

Comment 11 ITD27M01 2017-10-03 12:49:47 UTC

Hi Guys

Configuration changes have some caveats:

1. I use custom domain in RHOSP 10 with ldap (Active Directory) authentication. By admin user of Default project I cannot collect information about cinder volumes from projects of my not-default domain on the CFME5.8 side (but can do it by openstack cli and api).

2. For heat templates collection the "stacks:global_index" rule should be set to "context_is_admin" in /etc/heat/policy.json configuration file on OpenStack controllers.

Please pay attention of the developers that customers can have not only default domain with default admin user (on openstack side) but custom domain with external auth like ldap or Active Directory.

As a workaround I currently have created admin project in my custom domain and have configured necessary rules in policy.json.

Comment 12 Sam Lucidi 2017-10-03 13:27:39 UTC

Hi,

With regard to point 1, I don't believe the fix associated with this BZ made any changes to cinder inventory collection. Can you elaborate on the behavior you are seeing?

Regarding point 2, Heat does need to have global admin enabled in order to enable fast inventory collection, which is why there's a separate setting to enable Heat speedups in CFME. Is it working for you with "stacks:global_index" set, or are you experiencing further problems?

Thanks,
Sam Lucidi

Comment 13 ITD27M01 2017-10-03 13:37:51 UTC

By setting "stacks:global_index" to "context_is_admin" collection of staks work well.(In reply to Sam Lucidi from comment #12)
> Hi,
> 
> With regard to point 1, I don't believe the fix associated with this BZ made
> any changes to cinder inventory collection. Can you elaborate on the
> behavior you are seeing?
> 
> Regarding point 2, Heat does need to have global admin enabled in order to
> enable fast inventory collection, which is why there's a separate setting to
> enable Heat speedups in CFME. Is it working for you with
> "stacks:global_index" set, or are you experiencing further problems?
> 
> Thanks,
> Sam Lucidi

If I use "Default" domain and "admin" user for provider connection I cannot see any cloud volumes from another domain in CFME. The collection work for tenants/instances/sec groups etc. but not for cloud volumes. At the moment I can not give the appropriate log. But if necessary I'll try to reproduce this issue. Do I need to do this ?

By setting "stacks:global_index" to "context_is_admin" collection of stacks work well.

Comment 14 Sam Lucidi 2017-10-04 17:10:12 UTC

It sounds like the problem you are experiencing with domains might be unrelated to this change, so you might consider opening a new ticket for it. Normally multiple domains are supported in CFME by configuring an Openstack provider for each domain, but it's possible that things might be different depending on how your keystone roles are set up. Please correct me if I'm misunderstanding and your problem is due to the fast refresh patch.

Comment 15 John Hardy 2017-11-15 14:13:03 UTC

Whe will this ticket goto POST?

Comment 16 Ido Ovadia 2018-02-18 14:03:26 UTC

Can only verified with ~150 tenants ~40 instances