Bug 1419463 - cdn-sync "importing packages" phase is 30% slower than for satellite-sync
Summary: cdn-sync "importing packages" phase is 30% slower than for satellite-sync
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Satellite 5
Classification: Red Hat
Component: Satellite Synchronization
Version: 580
Hardware: Unspecified
OS: Unspecified
unspecified
low
Target Milestone: ---
Assignee: Jan Dobes
QA Contact: Jan Hutař
URL:
Whiteboard:
Depends On:
Blocks: sat580-low
TreeView+ depends on / blocked
 
Reported: 2017-02-06 08:59 UTC by Jan Hutař
Modified: 2017-06-21 12:10 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-06-21 12:10:14 UTC
Target Upstream Version:


Attachments (Terms of Use)

Description Jan Hutař 2017-02-06 08:59:57 UTC
Description of problem:
cdn-sync "importing packages" phase is 30% slower than for satellite-sync on same HW. Syncing rhel-x86_64-server-6, it looks like:

 * 5.6.0: Importing package metadata: 9880
 * 5.7.0: Importing package metadata: 10287
 * 5.8Beta: Importing package: 12997


Version-Release number of selected component (if applicable):
Satellite-5.8-RHEL-6-20170118.0-Satellite-x86_64


How reproducible:
1 of 1


Steps to Reproduce:
1. Compare import times on 5.8Beta and older satellites


Actual results:
It is 30% slower


Expected results:
It should be equally fast, but maybe these values can not be compared, maybe they are doing something else?

Comment 1 Tomas Lestach 2017-02-06 09:25:46 UTC
>  * 5.6.0: Importing package metadata: 9880
>  * 5.7.0: Importing package metadata: 10287
>  * 5.8Beta: Importing package: 12997

So, what are these numbers? Seconds?


In case this is only package metadata import data, what percentage does it make from the whole sync?
What are the overall sync times?

Comment 2 Jan Hutař 2017-02-06 09:34:32 UTC
Yes, these numbers are seconds.

Overall, cdn-sync is much faster (about 5 hours compared to much more - still running).

Comment 3 Jan Dobes 2017-02-06 10:07:37 UTC
It's hard to compare because satellite-sync is consuming package metadata from XML prepared on RHN and cdn-sync is extracting them from RPMs. However, it's worth investigating if this overhead really makes it 30% slower or there is something else to optimize.

Comment 4 Jan Dobes 2017-05-18 15:48:22 UTC
I think the speed of package import should be better now and also possibly improvable after --batch-size option was added. Basically, this bugzilla was opened when batch size was hardcoded 1. Default value is now 10.

Btw. satellite-sync is using default value 50 but as far as we found out it's not much faster + memory consumption will rise. Also satellite-sync has other bottlenecks. But we can experiment with this value in cdn-sync and change default value if needed.


Note You need to log in before you can comment on or make changes to this bug.