Red Hat Bugzilla – Bug 976275
Pages that have Pulp content take 4-5+ minutes to respond if there are a large number of repos
Last modified: 2017-07-26 15:43:18 EDT
Description of problem:
If you have hundreds of repos in Pulp (we tested with over 500), then going to a page such as /sync_management/index can take 4-5 minutes to load.
Version-Release number of selected component (if applicable):
Steps to Reproduce:
1. Grab a manifest with several subscriptions (RHEL, RHEV, Openshift, etc)
2. Upload the manifest to your Katello instance
3. Enable all the repositories via CLI
4. Go to /sync_management/index
Page takes several minutes to load.
Page should take 4-5 seconds to load like other pages in Katello.
Since this issue was entered in Red Hat Bugzilla, the release flag has been
set to ? to ensure that it is properly evaluated for this release.
Created attachment 763383 [details]
Snapshot from this morning
Here are the package versions.
Currently we are going out to pulp at least 2x for each repo. Once to check if there is an active sync task for the repo, and then once to get its history. Then for each repo we sort the history/tasks to use the most relevant.
The history API I believe is very much tied to a single repo, so there is not much we could do there if we still wanted to show if the previous sync for each repo failed or was successful.
The Current task api we might be able to query for all repos, so we could potentially cut the # of responses to pulp by half which is not all that great.
We will probably want to request enhancements to the pulp api to help us with this.
*** Bug 1008636 has been marked as a duplicate of this bug. ***
See the duplicate bug I closed for other examples of pages that are really slow
Two things were done to speed up this page
* Preload the list of orphaned products, as this is a very expensive call per product
* Load each tab via ajax
This makes it so the page loads instantly and then the rpms tab loads in about 10 seconds.
Ignore the above comment, it was intended for bz 1008636
moving this back to mdp3, not much we can do without help from pulp and/or completely re-writing the page:
What type of additions to the api do you need?
This is now resolved in Satellite 6.0. The page was not completely re-written but was migrated to use dynflow actions and not talk directly to pulp.
I am sure that this page will still be slow if you load in 1000s of repos, but at 400-500, it seems to perform and load quickly.
Verified in Satellite-6.1.0-RHEL-7-20150303.0. Yes, this page is still slow but it's better...
This bug is slated to be released with Satellite 6.1.
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory, and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.