Red Hat Bugzilla – Bug 837483
Promotions take too long; repos are promoted every time
Last modified: 2014-09-18 11:31:34 EDT
Promotions take too long to complete. If I promote a system template, but make a "one-liner" mistake (forgot a package/package group, distribution, etc.), the second promotion takes as long as the first (about 15-20 minutes). It seems that the repos are promoted every single time I do a promotion.
Version-Release number of selected component (if applicable):
Using CF 1.0.1
Steps to Reproduce:
1. Promote a system template
2. Promote the system template to the same environment
It takes a long time.
The second time should be much shorter. The repos should not be promoted every single time I promote a system template.
This has significant impact on our training course. We are having to work around this issue because it takes so long to do any promotion. I have had several students spend an hour doing promotions because they forgot one item in a system template and they must repeatedly promote the system template.
I just noticed that the Dashboard shows that the repos were all sync'ed every time I promoted a system template. So, not only are they promoted every time, but they are sync'ed as well.
Tere are 2 main issues that we are running into.
== Issue 1 ==
The first major issue is that we are still calling /sync/ on the child
repos contained in the system template when that template gets promoted.
If you setup a template with a repo and then promote *just* the template
we still end up calling Pulp's /sync/ on the repos contained in the template.
I'm not yet sure why this is but calling sync on a child clone still
scenario is this:
1) setup custom product, CustomProduct1, with a few small repos (10-200
packages) and one larger repo (3k packages)
2) setup Dev env
3) sync the product and all its repos
4) create a system template, say Template1
5) promote CustomProduct1 and Template1 to Dev the 1st time. This takes
approx 30 seconds:
Changeset#promote_content completed after 29.2365
6) create a 2nd changeset, include the only the Template1 and NOTHING else in the changeset, promote ...
This ends up calling /sync/ on the repo in Dev:
HTTP/1.1" 202 362
the whole promotion takes around 2 minutes because of this sync call
Changeset#promote_content completed after 120.6568
I don't yet know why we are forcibly re-promoting the entire repo if you
promote just a system template but this seems like a really bad idea
because you are in effect, pulling *all* the updated content into the
next env even if all you think you are promoting is just a template.
This would cause all errata, packages and content in that repo to get
pushed into the next env in a changeset that only lists a template.
The other issue is that of performance. If you have included say, the
base RHEL6 or RHEL5 repo in your template this can have ~10K packages
and on a slower system that sync call could easily go up into the 10-15
minute range. If your template had 5-6 repos with RHEL, Optional,
Supplimentary, etc ... that is a lot of needless sync calls.
== Issue 2 ==
The second, but less severe issue is the time we are taking to
re-promote a product or repo. Whenever you re-promote a product and its
repos you end up calling the Pulp /sync/ command on each repo in the
target environment. At first this sounds harmless but it does take a
good amount of time even if the repos are entirely the same between
parent and child. In most cases you are spending a long time on a
glorified NOOP if there is no content change between Library -> Dev.
If there was some way we could compare the parent to the child and not
do an entire sync if there were no differences we could probably reduce
the speed way down on re-promotions.
proposing for 1.0.2 z stream release
Have a patch for this, attaching ASAP
Created attachment 598131 [details]
updated system_template.rb to no longer promote repos if they are part of template
make a backup of /usr/share/katello/app/models/system_template.rb and copy to that location. restart katello and katello-jobs and if you promote *just* a template it should not force a promote of the repos associated with that template.
We saw this in the training. Bumping this up in priority. We need to look at both the initial publish and promotion. If we are doing a full repo, it should be a copy
We made significant performance improvements that will be included in the Satellite 6 Beta and GA releases that help with promotion timings.
Small repos can publish and promote in under a minute and large 20k package repos can often finish a publish in 10-12 minutes.
We also include much more detailed progress reporting during the promotion process so the user is aware of what is going on.
Going to CLOSE:WONTFIX this bug since this entire process of managing changesets and promoting content has been redesigned and improved.
If the release of Satellite 6 continues to not address this issue feel free to re-open.