Description of problem: * Stock Satellite v5.4 * Several channels synced Every satellite-sync following the initial (big) sync, should (supposedly) be quite fast (depending on the amount of changes since the last sync). On stock Satellite 5.4 though, every sync following the initial sync takes just as long as the initial sync. The biggest slowdown comparing to Satellite 5.3 seems to be in step 'download-packages', where the sync seems to be parsing metadata of all channel packages as opposed to metadata of packages which were born since the last sync (i.e. missing packages). Version-Release number of selected component (if applicable): spacewalk-backend-1.2.13-11 How reproducible: Always Steps to Reproduce: 1. Satellite 5.4, sync a base channel, few child channels, measure the length of the synchronization. 2. After the initial sync if finished, measure the length of another satellite-sync, run without any parameters (i.e. like you just want to update channels you already have in your Satellite). 3. Compare the both values. Actual results: Subsequent sync takes just as long as the initial sync. Expected results: Subsequent sync should be much shorter (behavior as in Satellite 5.3) Additional info: Problem seems to stem in commit http://git.fedorahosted.org/git/?p=spacewalk.git;a=commitdiff;h=c191fcfb95e730a13666921c52ba6860669d89ac and following change in _missing_not_cached_packages() routine (satsync.py): - for channel, pids in self._missing_channel_packages.items(): + for channel, pids in self._channel_packages.items(): This results into satellite-sync thinking all packages in the particular channel need to be processed, regardless of what we already have synced previously.
Created attachment 460151 [details] log from satellite-sync on Satellite 5.3 (w/ all relevant erratas applied)
Created attachment 460152 [details] log from satellite-sync on Satellite 5.4 (incomplete)
_missing_not_cached_packages() has been fixed in spacewalk git by commit 1f9730525a66e71dc9410506160d216dc4047bdf download and parse only missing packages Also 'Retrieving short package metadata' step has been speeded up by (in spacewalk git) commit 6f766c4d87b02ab4bc71068720a46cd37f0e38c1 skip packages we already processed
Two more commits fixing Diffing and Downloading phases; spacewalk git: commit 9274a7dbfd4ed0164e070377d6f1e38035773dc6 652815 - made list of ids in non-full dicts disjunct commit a573e9b497b7aa848d6410afdbb755e086616182 652815 - created _channel_*_full dicts to hold full list of ids fo channel
Backported to SATELLITE-5.4: commit b2e7f2000d1a45abf89f7c26dd978b94147d7b9c 652815 - made list of ids in non-full dicts disjunct commit 3620c2ee519ee42c20d2de3a38c97b9192e1345a 652815 - created _channel_*_full dicts to hold full list of ids fo channel original dicts should contain every id only once so we don't download and parse them repeatedly commit 3a1429fd5e40924a9d6539b1b97b8fc9a84e5280 download and parse only missing packages
Created attachment 461279 [details] satellite-sync log satellite-sync with the fixes from this bug applied, running on satdump01 (phx2), all RHEL-3,4,5,6 channels (678 channels).
before upgrade: Import complete: Begin time: Tue Nov 23 05:03:47 2010 End time: Tue Nov 23 23:14:12 2010 Elapsed: 18 hours, 10 minutes, 24 seconds after update and removal of /var/satellite/{redhat,rhn}/*: Import complete: Begin time: Wed Nov 24 02:56:11 2010 End time: Wed Nov 24 06:06:50 2010 Elapsed: 3 hours, 10 minutes, 39 seconds empty sync Import complete: Begin time: Wed Nov 24 06:34:06 2010 End time: Wed Nov 24 06:35:33 2010 Elapsed: 0 hours, 1 minutes, 26 seconds satellite-sycn rhel5i386 +rhel6-x86_64 (~10000 packages) 10:34:57 Imported kickstartable trees (7) Import complete: Begin time: Wed Nov 24 06:47:18 2010 End time: Wed Nov 24 10:34:57 2010 Elapsed: 3 hours, 47 minutes, 38 seconds empty sync: Import complete: Begin time: Wed Nov 24 10:38:55 2010 End time: Wed Nov 24 10:41:43 2010 Elapsed: 0 hours, 2 minutes, 48 seconds
An advisory has been issued which should help the problem described in this bug report. This report is therefore being closed with a resolution of ERRATA. For more information on therefore solution and/or where to find the updated files, please follow the link below. You may reopen this bug report if the solution does not work for you. http://rhn.redhat.com/errata/RHBA-2010-0974.html