Description of problem: If you have hundreds of repos in Pulp (we tested with over 500), then going to a page such as /sync_management/index can take 4-5 minutes to load. Version-Release number of selected component (if applicable): 1.4 How reproducible: Pretty consistently. Steps to Reproduce: 1. Grab a manifest with several subscriptions (RHEL, RHEV, Openshift, etc) 2. Upload the manifest to your Katello instance 3. Enable all the repositories via CLI 4. Go to /sync_management/index Actual results: Page takes several minutes to load. Expected results: Page should take 4-5 seconds to load like other pages in Katello.
Since this issue was entered in Red Hat Bugzilla, the release flag has been set to ? to ensure that it is properly evaluated for this release.
Created attachment 763383 [details] Snapshot from this morning
Here are the package versions. candlepin-0.8.9-1.el6_4.noarch candlepin-scl-1-5.el6_4.noarch candlepin-scl-quartz-2.1.5-5.el6_4.noarch candlepin-scl-rhino-1.7R3-1.el6_4.noarch candlepin-scl-runtime-1-5.el6_4.noarch candlepin-selinux-0.8.9-1.el6_4.noarch candlepin-tomcat6-0.8.9-1.el6_4.noarch elasticsearch-0.19.9-8.el6sat.noarch katello-1.4.2-14.el6sat.noarch katello-all-1.4.2-14.el6sat.noarch katello-candlepin-cert-key-pair-1.0-1.noarch katello-certs-tools-1.4.2-2.el6sat.noarch katello-cli-1.4.2-7.el6sat.noarch katello-cli-common-1.4.2-7.el6sat.noarch katello-common-1.4.2-14.el6sat.noarch katello-configure-1.4.3-16.el6sat.noarch katello-configure-foreman-1.4.3-16.el6sat.noarch katello-foreman-all-1.4.2-14.el6sat.noarch katello-glue-candlepin-1.4.2-14.el6sat.noarch katello-glue-elasticsearch-1.4.2-14.el6sat.noarch katello-glue-pulp-1.4.2-14.el6sat.noarch katello-qpid-broker-key-pair-1.0-1.noarch katello-qpid-client-key-pair-1.0-1.noarch katello-selinux-1.4.3-3.el6sat.noarch m2crypto-0.21.1.pulp-8.el6sat.x86_64 mod_wsgi-3.4-1.pulp.el6sat.x86_64 pulp-rpm-plugins-2.1.2-0.3.beta.el6sat.noarch pulp-selinux-2.1.2-0.3.beta.el6sat.noarch pulp-server-2.1.2-0.3.beta.el6sat.noarch python-isodate-0.5.0-1.pulp.el6sat.noarch python-oauth2-1.5.170-3.pulp.el6sat.noarch python-pulp-common-2.1.2-0.3.beta.el6sat.noarch python-pulp-rpm-common-2.1.2-0.3.beta.el6sat.noarch python-qpid-0.18-5.el6_4.noarch python-rhsm-1.8.0-1.pulp.el6sat.x86_64 qpid-cpp-client-0.14-22.el6_3.x86_64 qpid-cpp-client-ssl-0.14-22.el6_3.x86_64 qpid-cpp-server-0.14-22.el6_3.x86_64 qpid-cpp-server-ssl-0.14-22.el6_3.x86_64 ruby193-rubygem-foreman-katello-engine-0.0.8-6.el6sat.noarch ruby193-rubygem-katello-foreman-engine-0.0.3-5.el6sat.noarch ruby193-rubygem-katello_api-0.0.3-2.el6_4.noarch ruby193-rubygem-ldap_fluff-0.2.2-1.el6sat.noarch signo-katello-0.0.18-1.el6sat.noarch
Currently we are going out to pulp at least 2x for each repo. Once to check if there is an active sync task for the repo, and then once to get its history. Then for each repo we sort the history/tasks to use the most relevant. The history API I believe is very much tied to a single repo, so there is not much we could do there if we still wanted to show if the previous sync for each repo failed or was successful. The Current task api we might be able to query for all repos, so we could potentially cut the # of responses to pulp by half which is not all that great. We will probably want to request enhancements to the pulp api to help us with this.
*** Bug 1008636 has been marked as a duplicate of this bug. ***
See the duplicate bug I closed for other examples of pages that are really slow
https://github.com/Katello/katello/pull/3162 Two things were done to speed up this page * Preload the list of orphaned products, as this is a very expensive call per product * Load each tab via ajax This makes it so the page loads instantly and then the rpms tab loads in about 10 seconds.
Ignore the above comment, it was intended for bz 1008636
moving this back to mdp3, not much we can do without help from pulp and/or completely re-writing the page: https://bugzilla.redhat.com/show_bug.cgi?id=1019400
What type of additions to the api do you need?
This is now resolved in Satellite 6.0. The page was not completely re-written but was migrated to use dynflow actions and not talk directly to pulp. I am sure that this page will still be slow if you load in 1000s of repos, but at 400-500, it seems to perform and load quickly.
Verified in Satellite-6.1.0-RHEL-7-20150303.0. Yes, this page is still slow but it's better...
This bug is slated to be released with Satellite 6.1.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2015:1592