Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 837483 - Promotions take too long; repos are promoted every time
Summary: Promotions take too long; repos are promoted every time
Keywords:
Status: CLOSED UPSTREAM
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Content Management
Version: 6.0.0
Hardware: x86_64
OS: Linux
high
high
Target Milestone: Unspecified
Assignee: Katello Bug Bin
QA Contact: Katello QA List
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2012-07-04 05:08 UTC by Forrest Taylor
Modified: 2019-09-26 13:34 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2014-05-27 21:48:45 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
updated system_template.rb to no longer promote repos if they are part of template (19.23 KB, application/octet-stream)
2012-07-13 18:14 UTC, Mike McCune
no flags Details

Description Forrest Taylor 2012-07-04 05:08:43 UTC
Promotions take too long to complete.  If I promote a system template, but make a "one-liner" mistake (forgot a package/package group, distribution, etc.), the second promotion takes as long as the first (about 15-20 minutes).  It seems that the repos are promoted every single time I do a promotion.


Version-Release number of selected component (if applicable):
Using CF 1.0.1


How reproducible:
Always

Steps to Reproduce:
1. Promote a system template
2. Promote the system template to the same environment
  
Actual results:
It takes a long time.

Expected results:
The second time should be much shorter.  The repos should not be promoted every single time I promote a system template.

Additional info:
This has significant impact on our training course.  We are having to work around this issue because it takes so long to do any promotion.  I have had several students spend an hour doing promotions because they forgot one item in a system template and they must repeatedly promote the system template.

Comment 1 Forrest Taylor 2012-07-04 05:12:23 UTC
I just noticed that the Dashboard shows that the repos were all sync'ed every time I promoted a system template.  So, not only are they promoted every time, but they are sync'ed as well.

Comment 2 Mike McCune 2012-07-12 17:42:31 UTC
Tere are 2 main issues that we are running into.

== Issue 1 ==

The first major issue is that we are still calling /sync/ on the child 
repos contained in the system template when that template gets promoted.

If you setup a template with a repo and then promote *just* the template 
we still end up calling Pulp's /sync/ on the repos contained in the template. 
I'm not yet sure why this is but calling sync on a child clone still 
takes time.

scenario is this:

1) setup custom product, CustomProduct1, with a few small repos (10-200 
packages) and one larger repo (3k packages)

2) setup Dev env

3) sync the product and all its repos

4) create a system template, say Template1

5) promote CustomProduct1 and Template1 to Dev the 1st time.  This takes 
approx 30 seconds:

Changeset#promote_content completed after 29.2365

6) create a 2nd changeset, include the only the Template1 and NOTHING else in the changeset, promote ...

This ends up calling /sync/ on the repo in Dev:

"POST 
/pulp/api//repositories/ACME_Corporation-DEV-EL6-local-el6-x86_64/sync/ 
HTTP/1.1" 202 362

the whole promotion takes around 2 minutes because of this sync call

  Changeset#promote_content completed after 120.6568

I don't yet know why we are forcibly re-promoting the entire repo if you 
promote just a system template but this seems like a really bad idea 
because you are in effect, pulling *all* the updated content into the 
next env even if all you think you are promoting is just a template.

This would cause all errata, packages and content in that repo to get 
pushed into the next env in a changeset that only lists a template.

The other issue is that of performance.  If you have included say, the 
base RHEL6 or RHEL5 repo in your template this can have ~10K packages 
and on a slower system that sync call could easily go up into the 10-15 
minute range.  If your template had 5-6 repos with RHEL, Optional, 
Supplimentary, etc ... that is a lot of needless sync calls.


== Issue 2 ==

The second, but less severe issue is the time we are taking to 
re-promote a product or repo.  Whenever you re-promote a product and its 
repos you end up calling the Pulp /sync/ command on each repo in the 
target environment.  At first this sounds harmless but it does take a 
good amount of time even if the repos are entirely the same between 
parent and child.  In most cases you are spending a long time on a 
glorified NOOP if there is no content change between Library -> Dev.

If there was some way we could compare the parent to the child and not 
do an entire sync if there were no differences we could probably reduce 
the speed way down on re-promotions.

Comment 3 Mike McCune 2012-07-12 17:43:10 UTC
proposing for 1.0.2 z stream release

Comment 4 Mike McCune 2012-07-13 17:53:45 UTC
Have a patch for this, attaching ASAP

Comment 5 Mike McCune 2012-07-13 18:14:44 UTC
Created attachment 598131 [details]
updated system_template.rb to no longer promote repos if they are part of template

make a backup of /usr/share/katello/app/models/system_template.rb and copy to that location.  restart katello and katello-jobs and if you promote *just* a template it should not force a promote of the repos associated with that template.

Comment 6 Bryan Kearney 2014-01-08 21:46:22 UTC
We saw this in the training. Bumping this up in priority. We need to look at both the initial publish and promotion. If we are doing a full repo, it should be a copy

Comment 7 Mike McCune 2014-05-27 21:48:45 UTC
We made significant performance improvements that will be included in the Satellite 6 Beta and GA releases that help with promotion timings.

Small repos can publish and promote in under a minute and large 20k package repos can often finish a publish in 10-12 minutes.

We also include much more detailed progress reporting during the promotion process so the user is aware of what is going on.

Going to CLOSE:WONTFIX this bug since this entire process of managing changesets and promoting content has been redesigned and improved.

If the release of Satellite 6 continues to not address this issue feel free to re-open.


Note You need to log in before you can comment on or make changes to this bug.