Bug 1014390 - Content View publish takes 2-10x as long with MDP2
Content View publish takes 2-10x as long with MDP2
Status: CLOSED CURRENTRELEASE
Product: Red Hat Satellite 6
Classification: Red Hat
Component: Content Management (Show other bugs)
6.0.2
Unspecified Unspecified
unspecified Severity high (vote)
: Unspecified
: --
Assigned To: Partha Aji
Garik Khachikyan
: Triaged
: 1017359 (view as bug list)
Depends On:
Blocks: 950743 1014663 1017359
  Show dependency treegraph
 
Reported: 2013-10-01 17:10 EDT by Mike McCune
Modified: 2015-01-04 17:00 EST (History)
8 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1014663 (view as bug list)
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Mike McCune 2013-10-01 17:10:57 EDT
Content View publishing of a moderate sized repo (3k packages, RHEL 6.4-x86_64) takes 20+ minutes to publish the first content view in MDP2

2013-10-01T16:07:48-0400: [Worker(delayed_job host:dhcp-8-30-94.lab.eng.rdu2.redhat.com pid:18493)] ContentViewDefinition#generate_repos completed after 1328.9986

With MDP1 it takes around 3 minutes:

2013-09-08T12:11:41-0400: [Worker(delayed_job host:satellite6.usersys.redhat.com pid:2493)] ContentViewDefinition#generate_repos completed after 372.8071



Steps to reproduce:

1) Enable RHEL 6.4 x86_64 from Red Hat repositories
2) Sync
3) create a content view with only this one repo in it, no filters
4) publish

wait 20 minutes ...

We are touting MDP2 to be faster with CV publishing and it doesn't appear to be so.
Comment 2 David Davis 2013-10-02 09:14:08 EDT
Publishing a definition with only two small zoo repos is taking about 100 seconds. 93 seconds are being spent in ContentViewDefinition#generate_repos. Here's a breakdown of the times.

32 seconds
https://github.com/Katello/katello/blob/cc81e688b0d01deb19b6e85a7d5d094ad17e186e/app/models/content_view_definition.rb#L292-L294

34 seconds
https://github.com/Katello/katello/blob/cc81e688b0d01deb19b6e85a7d5d094ad17e186e/app/models/content_view_definition.rb#L311-L312

23 seconds
https://github.com/Katello/katello/blob/cc81e688b0d01deb19b6e85a7d5d094ad17e186e/app/models/content_view_definition.rb#L89

95% of the time here is spent waiting on Pulp.
Comment 3 David Davis 2013-10-02 09:19:11 EDT
Here is the pulp log:

https://gist.github.com/daviddavis/a8d6bcc8ce82afe489da
Comment 4 David Davis 2013-10-02 09:22:47 EDT
I'm using pulp 2.3.0-0.12.alpha but I believe MDP2 may be on 2.3.0-0.14.alpha.
Comment 5 RHEL Product and Program Management 2013-10-02 09:52:48 EDT
Since this issue was entered in Red Hat Bugzilla, the release flag has been
set to ? to ensure that it is properly evaluated for this release.
Comment 7 David Davis 2013-10-04 08:32:52 EDT
https://github.com/Katello/katello/pull/3091
Comment 8 David Davis 2013-10-07 08:28:38 EDT
katello SHA e44ee1fd765d895be65819a35f8a7669d9495d8b

1014390: Speed up pulp actions by using a variable wait time

Speed up content view publishing by 3-4 fold.
Comment 11 Mike McCune 2013-10-08 19:42:42 EDT
more fixes inbound for this, moving back to ASSIGNED
Comment 12 Mike McCune 2013-10-08 19:44:11 EDT
from Partha:

https://github.com/Katello/katello/pull/3136
Comment 15 Mike McCune 2013-10-17 16:29:42 EDT
QE:

at a minimum we should verify that publishes take between 5-10 minutes and not 20-30 minutes.

Ideally we could do direct comparison between MDP1 and MDP2 and publishing a view and the time it takes.
Comment 16 Mike McCune 2013-10-17 16:44:58 EDT
*** Bug 1017359 has been marked as a duplicate of this bug. ***
Comment 17 Garik Khachikyan 2013-10-21 08:04:11 EDT
thanks Mike for the scenario.

I'll take QA contact trying to reproduce the publish of the whole RHEL 6.4 x86_64 and look at logs for duration taken. thanks.
Comment 18 Garik Khachikyan 2013-10-21 10:10:34 EDT
# VERIFIED

it took just an impressive-4min only :)

checked on:
===
* apr-util-ldap-1.3.9-3.el6_0.1.x86_64
* candlepin-0.8.25-1.el6sam.noarch
* candlepin-scl-1-5.el6_4.noarch
* candlepin-scl-quartz-2.1.5-5.el6_4.noarch
* candlepin-scl-rhino-1.7R3-1.el6_4.noarch
* candlepin-scl-runtime-1-5.el6_4.noarch
* candlepin-selinux-0.8.25-1.el6sam.noarch
* candlepin-tomcat6-0.8.25-1.el6sam.noarch
* elasticsearch-0.19.9-8.el6sat.noarch
* foreman-1.3.0-18.el6sat.noarch
* foreman-compute-1.3.0-18.el6sat.noarch
* foreman-libvirt-1.3.0-18.el6sat.noarch
* foreman-postgresql-1.3.0-18.el6sat.noarch
* foreman-proxy-1.3.0-3.el6sat.noarch
* katello-1.4.6-39.el6sat.noarch
* katello-all-1.4.6-39.el6sat.noarch
* katello-candlepin-cert-key-pair-1.0-1.noarch
* katello-certs-tools-1.4.4-1.el6sat.noarch
* katello-cli-1.4.3-24.el6sat.noarch
* katello-cli-common-1.4.3-24.el6sat.noarch
* katello-common-1.4.6-39.el6sat.noarch
* katello-configure-1.4.7-5.el6sat.noarch
* katello-configure-foreman-1.4.7-5.el6sat.noarch
* katello-configure-foreman-proxy-1.4.7-5.el6sat.noarch
* katello-foreman-all-1.4.6-39.el6sat.noarch
* katello-glue-candlepin-1.4.6-39.el6sat.noarch
* katello-glue-elasticsearch-1.4.6-39.el6sat.noarch
* katello-glue-pulp-1.4.6-39.el6sat.noarch
* katello-qpid-broker-key-pair-1.0-1.noarch
* katello-qpid-client-key-pair-1.0-1.noarch
* katello-selinux-1.4.4-4.el6sat.noarch
* openldap-2.4.23-31.el6.x86_64
* pulp-katello-plugins-0.2-1.el6sat.noarch
* pulp-nodes-common-2.3.0-0.22.beta.el6sat.noarch
* pulp-nodes-parent-2.3.0-0.22.beta.el6sat.noarch
* pulp-puppet-plugins-2.3.0-0.22.beta.el6sat.noarch
* pulp-rpm-plugins-2.3.0-0.22.beta.el6sat.noarch
* pulp-selinux-2.3.0-0.22.beta.el6sat.noarch
* pulp-server-2.3.0-0.22.beta.el6sat.noarch
* python-ldap-2.3.10-1.el6.x86_64
* ruby193-rubygem-ldap_fluff-0.2.2-2.el6sat.noarch
* ruby193-rubygem-net-ldap-0.3.1-3.el6sat.noarch
* ruby193-rubygem-runcible-1.0.7-1.el6sat.noarch
* signo-0.0.22-2.el6sat.noarch
* signo-katello-0.0.22-2.el6sat.noarch
Comment 21 Bryan Kearney 2014-04-24 13:10:30 EDT
This was verified and delivered with MDP2. Closing it out.
Comment 22 Bryan Kearney 2014-04-24 13:11:43 EDT
This was delivered and verified with MDP2. Closing the bug.

Note You need to log in before you can comment on or make changes to this bug.