Bug 976275 - Pages that have Pulp content take 4-5+ minutes to respond if there are a large number of repos
Pages that have Pulp content take 4-5+ minutes to respond if there are a larg...
Status: CLOSED ERRATA
Product: Red Hat Satellite 6
Classification: Red Hat
Component: Content Management (Show other bugs)
6.0.2
Unspecified Unspecified
unspecified Severity high (vote)
: Unspecified
: --
Assigned To: Justin Sherrill
Corey Welton
:
Depends On: 1019400
Blocks: sat6-pulp-future
  Show dependency treegraph
 
Reported: 2013-06-20 05:19 EDT by David Davis
Modified: 2017-07-26 15:43 EDT (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2015-08-12 01:07:16 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Snapshot from this morning (153.20 KB, image/png)
2013-06-20 05:38 EDT, David Davis
no flags Details

  None (edit)
Description David Davis 2013-06-20 05:19:27 EDT
Description of problem:

If you have hundreds of repos in Pulp (we tested with over 500), then going to a page such as /sync_management/index can take 4-5 minutes to load.


Version-Release number of selected component (if applicable):

1.4


How reproducible:

Pretty consistently.


Steps to Reproduce:
1. Grab a manifest with several subscriptions (RHEL, RHEV, Openshift, etc)
2. Upload the manifest to your Katello instance
3. Enable all the repositories via CLI
4. Go to /sync_management/index

Actual results:

Page takes several minutes to load.


Expected results:

Page should take 4-5 seconds to load like other pages in Katello.
Comment 1 RHEL Product and Program Management 2013-06-20 05:34:17 EDT
Since this issue was entered in Red Hat Bugzilla, the release flag has been
set to ? to ensure that it is properly evaluated for this release.
Comment 3 David Davis 2013-06-20 05:38:19 EDT
Created attachment 763383 [details]
Snapshot from this morning
Comment 4 David Davis 2013-06-20 08:01:05 EDT
Here are the package versions.

candlepin-0.8.9-1.el6_4.noarch
candlepin-scl-1-5.el6_4.noarch
candlepin-scl-quartz-2.1.5-5.el6_4.noarch
candlepin-scl-rhino-1.7R3-1.el6_4.noarch
candlepin-scl-runtime-1-5.el6_4.noarch
candlepin-selinux-0.8.9-1.el6_4.noarch
candlepin-tomcat6-0.8.9-1.el6_4.noarch
elasticsearch-0.19.9-8.el6sat.noarch
katello-1.4.2-14.el6sat.noarch
katello-all-1.4.2-14.el6sat.noarch
katello-candlepin-cert-key-pair-1.0-1.noarch
katello-certs-tools-1.4.2-2.el6sat.noarch
katello-cli-1.4.2-7.el6sat.noarch
katello-cli-common-1.4.2-7.el6sat.noarch
katello-common-1.4.2-14.el6sat.noarch
katello-configure-1.4.3-16.el6sat.noarch
katello-configure-foreman-1.4.3-16.el6sat.noarch
katello-foreman-all-1.4.2-14.el6sat.noarch
katello-glue-candlepin-1.4.2-14.el6sat.noarch
katello-glue-elasticsearch-1.4.2-14.el6sat.noarch
katello-glue-pulp-1.4.2-14.el6sat.noarch
katello-qpid-broker-key-pair-1.0-1.noarch
katello-qpid-client-key-pair-1.0-1.noarch
katello-selinux-1.4.3-3.el6sat.noarch
m2crypto-0.21.1.pulp-8.el6sat.x86_64
mod_wsgi-3.4-1.pulp.el6sat.x86_64
pulp-rpm-plugins-2.1.2-0.3.beta.el6sat.noarch
pulp-selinux-2.1.2-0.3.beta.el6sat.noarch
pulp-server-2.1.2-0.3.beta.el6sat.noarch
python-isodate-0.5.0-1.pulp.el6sat.noarch
python-oauth2-1.5.170-3.pulp.el6sat.noarch
python-pulp-common-2.1.2-0.3.beta.el6sat.noarch
python-pulp-rpm-common-2.1.2-0.3.beta.el6sat.noarch
python-qpid-0.18-5.el6_4.noarch
python-rhsm-1.8.0-1.pulp.el6sat.x86_64
qpid-cpp-client-0.14-22.el6_3.x86_64
qpid-cpp-client-ssl-0.14-22.el6_3.x86_64
qpid-cpp-server-0.14-22.el6_3.x86_64
qpid-cpp-server-ssl-0.14-22.el6_3.x86_64
ruby193-rubygem-foreman-katello-engine-0.0.8-6.el6sat.noarch
ruby193-rubygem-katello-foreman-engine-0.0.3-5.el6sat.noarch
ruby193-rubygem-katello_api-0.0.3-2.el6_4.noarch
ruby193-rubygem-ldap_fluff-0.2.2-1.el6sat.noarch
signo-katello-0.0.18-1.el6sat.noarch
Comment 5 Justin Sherrill 2013-06-20 09:13:44 EDT
Currently we are going out to pulp at least 2x for each repo.  Once to check if there is an active sync task for the repo, and then once to get its history.  Then for each repo we sort the history/tasks to use the most relevant.

The history API I believe is very much tied to a single repo, so there is not much we could do there if we still wanted to show if the previous sync for each repo failed or was successful. 

The Current task api we might be able to query for all repos, so we could potentially cut the # of responses to pulp by half which is not all that great.

We will probably want to request enhancements to the pulp api to help us with this.
Comment 7 Mike McCune 2013-09-30 16:03:47 EDT
*** Bug 1008636 has been marked as a duplicate of this bug. ***
Comment 8 Mike McCune 2013-09-30 16:08:25 EDT
See the duplicate bug I closed for other examples of pages that are really slow
Comment 10 Justin Sherrill 2013-10-10 17:04:52 EDT
https://github.com/Katello/katello/pull/3162

Two things were done to speed up this page
   *  Preload the list of orphaned products, as this is a very expensive call per product
   * Load each tab via ajax


This makes it so the page loads instantly and then the rpms tab loads in about 10 seconds.
Comment 11 Justin Sherrill 2013-10-10 17:37:35 EDT
Ignore the above comment, it was intended for bz 1008636
Comment 12 Justin Sherrill 2013-10-15 11:57:43 EDT
moving this back to mdp3, not much we can do without help from pulp and/or completely re-writing the page:

https://bugzilla.redhat.com/show_bug.cgi?id=1019400
Comment 13 Bryan Kearney 2014-09-23 10:12:07 EDT
What type of additions to the api do you need?
Comment 14 Justin Sherrill 2014-11-04 15:10:42 EST
This is now resolved in Satellite 6.0.  The page was not completely re-written but was migrated to use dynflow actions and not talk directly to pulp.

I am sure that this page will still be slow if you load in 1000s of repos, but at 400-500, it seems to perform and load quickly.
Comment 16 Corey Welton 2015-03-06 12:06:13 EST
Verified in Satellite-6.1.0-RHEL-7-20150303.0.  Yes, this page is still slow but it's better...
Comment 17 Bryan Kearney 2015-08-11 09:25:16 EDT
This bug is slated to be released with Satellite 6.1.
Comment 18 errata-xmlrpc 2015-08-12 01:07:16 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2015:1592

Note You need to log in before you can comment on or make changes to this bug.