Bug 652815

Summary: Satellite 5.4: supposedly an empty sync takes just as long as a new (full) sync (download-packages part)
Product: Red Hat Satellite 5 Reporter: Milan Zázrivec <mzazrivec>
Component: Satellite SynchronizationAssignee: Michael Mráka <mmraka>
Status: CLOSED ERRATA QA Contact: Jiri Kastner <jkastner>
Severity: medium Docs Contact:
Priority: high    
Version: 540CC: cperry, gbock, jfenal, jkastner, mmraka
Target Milestone: ---Keywords: Regression
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: spacewalk-backend-1.2.13-12 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2010-12-13 14:31:39 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 646488    
Attachments:
Description Flags
log from satellite-sync on Satellite 5.3 (w/ all relevant erratas applied)
none
log from satellite-sync on Satellite 5.4 (incomplete)
none
satellite-sync log none

Description Milan Zázrivec 2010-11-12 21:05:53 UTC 
Description of problem:
* Stock Satellite v5.4
* Several channels synced

Every satellite-sync following the initial (big) sync, should (supposedly)
be quite fast (depending on the amount of changes since the last sync).

On stock Satellite 5.4 though, every sync following the initial sync
takes just as long as the initial sync.

The biggest slowdown comparing to Satellite 5.3 seems to be in
step 'download-packages', where the sync seems to be parsing
metadata of all channel packages as opposed to metadata of packages
which were born since the last sync (i.e. missing packages).

Version-Release number of selected component (if applicable):
spacewalk-backend-1.2.13-11

How reproducible:
Always

Steps to Reproduce:
1. Satellite 5.4, sync a base channel, few child channels, measure the
   length of the synchronization.
2. After the initial sync if finished, measure the length of another
   satellite-sync, run without any parameters (i.e. like you just want
   to update channels you already have in your Satellite).
3. Compare the both values.
  
Actual results:
Subsequent sync takes just as long as the initial sync.

Expected results:
Subsequent sync should be much shorter (behavior as in Satellite 5.3)

Additional info:
Problem seems to stem in commit
http://git.fedorahosted.org/git/?p=spacewalk.git;a=commitdiff;h=c191fcfb95e730a13666921c52ba6860669d89ac

and following change in _missing_not_cached_packages() routine (satsync.py):

-        for channel, pids in self._missing_channel_packages.items():
+        for channel, pids in self._channel_packages.items():

This results into satellite-sync thinking all packages in the particular
channel need to be processed, regardless of what we already have synced
previously.
Comment 1 Milan Zázrivec 2010-11-12 21:07:21 UTC 
Created attachment 460151 [details]
log from satellite-sync on Satellite 5.3 (w/ all relevant erratas applied)
Comment 2 Milan Zázrivec 2010-11-12 21:08:51 UTC 
Created attachment 460152 [details]
log from satellite-sync on Satellite 5.4 (incomplete)
Comment 3 Michael Mráka 2010-11-14 09:53:38 UTC 
_missing_not_cached_packages() has been fixed in spacewalk git by
commit 1f9730525a66e71dc9410506160d216dc4047bdf
    download and parse only missing packages

Also 'Retrieving short package metadata' step has been speeded up by (in spacewalk git)
commit 6f766c4d87b02ab4bc71068720a46cd37f0e38c1
    skip packages we already processed
Comment 4 Michael Mráka 2010-11-15 12:52:37 UTC 
Two more commits fixing Diffing and Downloading phases; spacewalk git:

commit 9274a7dbfd4ed0164e070377d6f1e38035773dc6
    652815 - made list of ids in non-full dicts disjunct
commit a573e9b497b7aa848d6410afdbb755e086616182
    652815 - created _channel_*_full dicts to hold full list of ids fo channel
Comment 5 Michael Mráka 2010-11-15 15:30:18 UTC 
Backported to SATELLITE-5.4:
commit b2e7f2000d1a45abf89f7c26dd978b94147d7b9c
    652815 - made list of ids in non-full dicts disjunct
commit 3620c2ee519ee42c20d2de3a38c97b9192e1345a
    652815 - created _channel_*_full dicts to hold full list of ids fo channel  
    original dicts should contain every id only once so we don't download
    and parse them repeatedly
commit 3a1429fd5e40924a9d6539b1b97b8fc9a84e5280
    download and parse only missing packages
Comment 7 Milan Zázrivec 2010-11-18 11:50:13 UTC 
Created attachment 461279 [details]
satellite-sync log

satellite-sync with the fixes from this bug applied, running on satdump01
(phx2), all RHEL-3,4,5,6 channels (678 channels).
Comment 8 Jiri Kastner 2010-11-24 15:50:06 UTC 
before upgrade:
    Import complete:
        Begin time: Tue Nov 23 05:03:47 2010
        End time:   Tue Nov 23 23:14:12 2010
        Elapsed:    18 hours, 10 minutes, 24 seconds


after update and removal of /var/satellite/{redhat,rhn}/*:
    Import complete:
        Begin time: Wed Nov 24 02:56:11 2010
        End time:   Wed Nov 24 06:06:50 2010
        Elapsed:    3 hours, 10 minutes, 39 seconds

empty sync
    Import complete:
        Begin time: Wed Nov 24 06:34:06 2010
        End time:   Wed Nov 24 06:35:33 2010
        Elapsed:    0 hours, 1 minutes, 26 seconds

satellite-sycn rhel5i386 +rhel6-x86_64 (~10000 packages)
10:34:57 Imported kickstartable trees (7)
    Import complete:
        Begin time: Wed Nov 24 06:47:18 2010
        End time:   Wed Nov 24 10:34:57 2010
        Elapsed:    3 hours, 47 minutes, 38 seconds

empty sync:
    Import complete:
        Begin time: Wed Nov 24 10:38:55 2010
        End time:   Wed Nov 24 10:41:43 2010
        Elapsed:    0 hours, 2 minutes, 48 seconds
Comment 9 errata-xmlrpc 2010-12-13 14:31:39 UTC 
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2010-0974.html