Red Hat Bugzilla – Bug 1460977
performance issue in openstack collection
Last modified: 2018-03-06 10:24:49 EST
Description of problem:
Cloudforms refreshes of openstack environment take too long - more than 10 minutes for 74 projects (tenants), managing 247 virtual machines
Version-Release number of selected component (if applicable):
all the time
Steps to Reproduce:
1. deploy openstack with around 300 vms and 74 tenants
2. deploy cloudforms with one provider connection to openstack
it takes 10 minutes to finish a refresh
it should take minutes, as few as possible, to meet two requirements
- have in CFME full overview of OpenStack provider and its configuration items will all relations etc.
- reduce refresh time in case there is new VM discovered or event with its state change is collected
This issue is blocking further deployment of Openstack and threatening the customer's buisness as is
tests were done with a hotfix (likely through another case) but the version hasn't been confirmed at the time of writing
Configuration changes have some caveats:
1. I use custom domain in RHOSP 10 with ldap (Active Directory) authentication. By admin user of Default project I cannot collect information about cinder volumes from projects of my not-default domain on the CFME5.8 side (but can do it by openstack cli and api).
2. For heat templates collection the "stacks:global_index" rule should be set to "context_is_admin" in /etc/heat/policy.json configuration file on OpenStack controllers.
Please pay attention of the developers that customers can have not only default domain with default admin user (on openstack side) but custom domain with external auth like ldap or Active Directory.
As a workaround I currently have created admin project in my custom domain and have configured necessary rules in policy.json.
With regard to point 1, I don't believe the fix associated with this BZ made any changes to cinder inventory collection. Can you elaborate on the behavior you are seeing?
Regarding point 2, Heat does need to have global admin enabled in order to enable fast inventory collection, which is why there's a separate setting to enable Heat speedups in CFME. Is it working for you with "stacks:global_index" set, or are you experiencing further problems?
By setting "stacks:global_index" to "context_is_admin" collection of staks work well.(In reply to Sam Lucidi from comment #12)
> With regard to point 1, I don't believe the fix associated with this BZ made
> any changes to cinder inventory collection. Can you elaborate on the
> behavior you are seeing?
> Regarding point 2, Heat does need to have global admin enabled in order to
> enable fast inventory collection, which is why there's a separate setting to
> enable Heat speedups in CFME. Is it working for you with
> "stacks:global_index" set, or are you experiencing further problems?
> Sam Lucidi
If I use "Default" domain and "admin" user for provider connection I cannot see any cloud volumes from another domain in CFME. The collection work for tenants/instances/sec groups etc. but not for cloud volumes. At the moment I can not give the appropriate log. But if necessary I'll try to reproduce this issue. Do I need to do this ?
By setting "stacks:global_index" to "context_is_admin" collection of stacks work well.
It sounds like the problem you are experiencing with domains might be unrelated to this change, so you might consider opening a new ticket for it. Normally multiple domains are supported in CFME by configuring an Openstack provider for each domain, but it's possible that things might be different depending on how your keystone roles are set up. Please correct me if I'm misunderstanding and your problem is due to the fast refresh patch.
Whe will this ticket goto POST?
Can only verified with ~150 tenants ~40 instances