Bug 837483 - Promotions take too long; repos are promoted every time
Promotions take too long; repos are promoted every time
Product: Red Hat Satellite 6
Classification: Red Hat
Component: Content Management (Show other bugs)
x86_64 Linux
high Severity high (vote)
: Unspecified
: --
Assigned To: Katello Bug Bin
Katello QA List
: Triaged
Depends On:
  Show dependency treegraph
Reported: 2012-07-04 01:08 EDT by Forrest Taylor
Modified: 2014-09-18 11:31 EDT (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2014-05-27 17:48:45 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
updated system_template.rb to no longer promote repos if they are part of template (19.23 KB, application/octet-stream)
2012-07-13 14:14 EDT, Mike McCune
no flags Details

  None (edit)
Description Forrest Taylor 2012-07-04 01:08:43 EDT
Promotions take too long to complete.  If I promote a system template, but make a "one-liner" mistake (forgot a package/package group, distribution, etc.), the second promotion takes as long as the first (about 15-20 minutes).  It seems that the repos are promoted every single time I do a promotion.

Version-Release number of selected component (if applicable):
Using CF 1.0.1

How reproducible:

Steps to Reproduce:
1. Promote a system template
2. Promote the system template to the same environment
Actual results:
It takes a long time.

Expected results:
The second time should be much shorter.  The repos should not be promoted every single time I promote a system template.

Additional info:
This has significant impact on our training course.  We are having to work around this issue because it takes so long to do any promotion.  I have had several students spend an hour doing promotions because they forgot one item in a system template and they must repeatedly promote the system template.
Comment 1 Forrest Taylor 2012-07-04 01:12:23 EDT
I just noticed that the Dashboard shows that the repos were all sync'ed every time I promoted a system template.  So, not only are they promoted every time, but they are sync'ed as well.
Comment 2 Mike McCune 2012-07-12 13:42:31 EDT
Tere are 2 main issues that we are running into.

== Issue 1 ==

The first major issue is that we are still calling /sync/ on the child 
repos contained in the system template when that template gets promoted.

If you setup a template with a repo and then promote *just* the template 
we still end up calling Pulp's /sync/ on the repos contained in the template. 
I'm not yet sure why this is but calling sync on a child clone still 
takes time.

scenario is this:

1) setup custom product, CustomProduct1, with a few small repos (10-200 
packages) and one larger repo (3k packages)

2) setup Dev env

3) sync the product and all its repos

4) create a system template, say Template1

5) promote CustomProduct1 and Template1 to Dev the 1st time.  This takes 
approx 30 seconds:

Changeset#promote_content completed after 29.2365

6) create a 2nd changeset, include the only the Template1 and NOTHING else in the changeset, promote ...

This ends up calling /sync/ on the repo in Dev:

HTTP/1.1" 202 362

the whole promotion takes around 2 minutes because of this sync call

  Changeset#promote_content completed after 120.6568

I don't yet know why we are forcibly re-promoting the entire repo if you 
promote just a system template but this seems like a really bad idea 
because you are in effect, pulling *all* the updated content into the 
next env even if all you think you are promoting is just a template.

This would cause all errata, packages and content in that repo to get 
pushed into the next env in a changeset that only lists a template.

The other issue is that of performance.  If you have included say, the 
base RHEL6 or RHEL5 repo in your template this can have ~10K packages 
and on a slower system that sync call could easily go up into the 10-15 
minute range.  If your template had 5-6 repos with RHEL, Optional, 
Supplimentary, etc ... that is a lot of needless sync calls.

== Issue 2 ==

The second, but less severe issue is the time we are taking to 
re-promote a product or repo.  Whenever you re-promote a product and its 
repos you end up calling the Pulp /sync/ command on each repo in the 
target environment.  At first this sounds harmless but it does take a 
good amount of time even if the repos are entirely the same between 
parent and child.  In most cases you are spending a long time on a 
glorified NOOP if there is no content change between Library -> Dev.

If there was some way we could compare the parent to the child and not 
do an entire sync if there were no differences we could probably reduce 
the speed way down on re-promotions.
Comment 3 Mike McCune 2012-07-12 13:43:10 EDT
proposing for 1.0.2 z stream release
Comment 4 Mike McCune 2012-07-13 13:53:45 EDT
Have a patch for this, attaching ASAP
Comment 5 Mike McCune 2012-07-13 14:14:44 EDT
Created attachment 598131 [details]
updated system_template.rb to no longer promote repos if they are part of template

make a backup of /usr/share/katello/app/models/system_template.rb and copy to that location.  restart katello and katello-jobs and if you promote *just* a template it should not force a promote of the repos associated with that template.
Comment 6 Bryan Kearney 2014-01-08 16:46:22 EST
We saw this in the training. Bumping this up in priority. We need to look at both the initial publish and promotion. If we are doing a full repo, it should be a copy
Comment 7 Mike McCune 2014-05-27 17:48:45 EDT
We made significant performance improvements that will be included in the Satellite 6 Beta and GA releases that help with promotion timings.

Small repos can publish and promote in under a minute and large 20k package repos can often finish a publish in 10-12 minutes.

We also include much more detailed progress reporting during the promotion process so the user is aware of what is going on.

Going to CLOSE:WONTFIX this bug since this entire process of managing changesets and promoting content has been redesigned and improved.

If the release of Satellite 6 continues to not address this issue feel free to re-open.

Note You need to log in before you can comment on or make changes to this bug.