Bug 1393447

Summary: fetching pools can take 5-6 seconds, causing slow dashboard
Product: Red Hat Satellite Reporter: Chris Duryee <cduryee>
Component: DashboardAssignee: satellite6-bugs <satellite6-bugs>
Status: CLOSED ERRATA QA Contact: Jan Hutaƙ <jhutar>
Severity: medium Docs Contact:
Priority: medium    
Version: 6.2.4CC: bbuckingham, bkearney, cduryee, dlobatog, jcallaha, jhutar, tbrisker
Target Milestone: UnspecifiedKeywords: Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-02-21 16:54:37 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Chris Duryee 2016-11-09 15:04:59 UTC
Description of problem:

Given a large DB, a call to '/candlepin/owners/<owner>/pools?attribute=unmapped_guests_only:!true' can result in 5-6 second return times.

It looks like there is a 2.5-ish second query in there, but there may be other areas to improve as well.

To repro:

* load large database into Satellite 6.2.4
* view dashboard, note long-ish load time. There will be a 5+ second call in /var/log/candlepin/candlepin.log

Version-Release number of selected component (if applicable): 6.2.4, candlepin candlepin-0.9.54.14-1.el7.noarch


Additional info:

calls to /candlepin/owners/<owner>/info also take about 1 second, not sure if that is related or should be a different bz. These calls are made to populate the Satellite 6 dashboard, which is refreshed at regular intervals.

Comment 3 Barnaby Court 2016-11-14 13:29:33 UTC
Hi Chris,

This code has been extensively refactored going into the CP 2.0 branch if you wouldn't mind testing there I would greatly appreciate it. Generally, please open a separate BZ for each call.  

The time required for the owner/info call is separate but is mostly related to the number of different summation queries that are being performed in the call.

Comment 5 Barnaby Court 2016-11-17 16:25:08 UTC
Chris, Two questions:

1) What performance are you seeing using Candlepin 2.0.20? 
2) What is your target performance for these queries? We have not previously defined an SLA for Candlepin calls so if there needs to be ceiling on the amount of time calls can reasonably take we will have to work that out.

Comment 6 Chris Duryee 2016-11-29 17:28:07 UTC
I have not tested on Candlepin 2 yet. If there is a way to improve this with 0.9 that's not invasive, that would be great, but if it seems better to wait for CP2 we can do that.

Ideally we could get the entire dashboard to load in 4 seconds or less. There is a lot of stuff going on in that page, so I would say 1-2 seconds for the API call would be good.

Comment 7 Barnaby Court 2016-12-05 20:51:04 UTC
Do you have a list of all the Candlepin calls that are made during the dashboard load? 

Sadly, all the changes made in CP 2.0.x would be incredibly invasive to try to backport. That is why I ask about what your results are with the current 2.0.x build.

Comment 8 Barnaby Court 2016-12-16 19:44:54 UTC
A lot of work has been done on this in CP 2.0.x already. In the meantime, a lot of speed can be added to these calls through liberal use of the include & exclude parameters so you are only retrieving the data that you need from Candlepin.

Comment 9 Daniel Lobato Garcia 2016-12-19 11:01:41 UTC
This doesn't slow down the dashboard loading any more since http://projects.theforeman.org/issues/16723 

Tomer - for 6.2 would you feel comfortable sending a merge request for the code above?

Chris - Even on 0.9 Candlepin there's the include/exclude API parameters which can alleviate the problem and make calls to pools smaller. I'll check if I can use those

https://github.com/Katello/katello/commit/2011c57b286b2cad7115291a1b43e0ad468aee84
https://github.com/Katello/katello/pull/6501 evidence the problem gets better (would even say goes away) using those. I'll check how to use these directives in the dashboard.

Comment 12 Daniel Lobato Garcia 2017-06-21 08:55:05 UTC
So these fixes were merged for 6.2 as https://gitlab.sat.lab.tlv.redhat.com/satellite6/foreman/commit/5326e84a8c394ec171da0786be5b1a37f1a08005

Closing as POST - although I think QE has probably verified this, in this case feel free to mark as VERIFIED as soon as you are done.

Comment 15 Satellite Program 2018-02-21 16:54:37 UTC
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA.
> > 
> > For information on the advisory, and where to find the updated files, follow the link below.
> > 
> > If the solution does not work for you, open a new bug report.
> > 
> > https://access.redhat.com/errata/RHSA-2018:0336