Bug 1473379

Summary: Storage profiles causing refresh to exceed 30+ minutes
Product: Red Hat CloudForms Management Engine Reporter: Jared Deubel <jdeubel>
Component: ProvidersAssignee: Adam Grare <agrare>
Status: CLOSED ERRATA QA Contact: Alex Newman <anewman>
Severity: high Docs Contact:
Priority: high    
Version: 5.7.0CC: akarol, bsorota, gblomqui, jfrey, jhardy, obarenbo, saali, simaishi
Target Milestone: GAKeywords: Reopened, ZStream
Target Release: 5.9.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: 5.9.0.1 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1593914 (view as bug list) Environment:
Last Closed: 2018-06-21 15:36:13 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: CFME Core Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1593914    

Description Jared Deubel 2017-07-20 16:08:45 UTC
Description of problem:
When doing an EMS refresh we are seeing storage profiles taking 30+ minutes [1] to collect. Because other workers are relying on the refresh to finish they timeout exceeding the 3600 second timeout. 

[1]

PostgreSQLAdapter#log_after_checkout, connection_pool: size: 5, connections: 1, in use: 1, waiting_in_queue: 0
{:pbmProfilesByUid=>0.6636569499969482,
 :pbmQueryMatchingHub=>2148.5395801067352,
 :pbmQueryAssociatedEntity=>2.737989664077759,
 :collect_storage_profiles=>2151.941283226013}
=> nil

Version-Release number of selected component (if applicable):
5.7.1.311

Comment 5 CFME Bot 2017-07-27 16:55:11 UTC
New commit detected on ManageIQ/manageiq-providers-vmware/master:
https://github.com/ManageIQ/manageiq-providers-vmware/commit/5758db5b70dcd67a9c9e692ed3e3fa8186472372

commit 5758db5b70dcd67a9c9e692ed3e3fa8186472372
Author:     Adam Grare <agrare>
AuthorDate: Thu Jul 20 12:04:06 2017 -0400
Commit:     Adam Grare <agrare>
CommitDate: Thu Jul 20 12:24:14 2017 -0400

    Add benchmark and logging around storage profiles
    
    Collection of storage profiles were not reflected it their own benchmark
    timings and were not logged making it difficult to diagnose when their
    collection is slow
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1473379

 .../providers/vmware/infra_manager/refresher.rb    | 34 ++++++++++++++++++----
 1 file changed, 29 insertions(+), 5 deletions(-)

Comment 6 CFME Bot 2017-08-08 14:33:49 UTC
New commit detected on ManageIQ/manageiq-providers-vmware/master:
https://github.com/ManageIQ/manageiq-providers-vmware/commit/9c6163198970f467e03b14a6009d4824e7856360

commit 9c6163198970f467e03b14a6009d4824e7856360
Author:     Adam Grare <agrare>
AuthorDate: Thu Jul 20 12:31:14 2017 -0400
Commit:     Adam Grare <agrare>
CommitDate: Thu Jul 27 12:46:53 2017 -0400

    Collect storage profiles only on full refresh
    
    The collection of storage profiles can take a long time and cannot be
    filtered by target causing targeted refreshes to take significantly
    longer under certain circumstances.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1473379

 .../manageiq/providers/vmware/infra_manager/refresher.rb      | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

Comment 9 errata-xmlrpc 2018-03-01 13:15:21 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:0380

Comment 12 Adam Grare 2018-06-25 15:04:57 UTC
*** Bug 1593430 has been marked as a duplicate of this bug. ***