Bug 1419463

Summary:	cdn-sync "importing packages" phase is 30% slower than for satellite-sync
Product:	Red Hat Satellite 5	Reporter:	Jan Hutař <jhutar>
Component:	Satellite Synchronization	Assignee:	Jan Dobes <jdobes>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Jan Hutař <jhutar>
Severity:	low	Docs Contact:
Priority:	unspecified
Version:	580	CC:	tlestach
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2017-06-21 12:10:14 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1358815

Description Jan Hutař 2017-02-06 08:59:57 UTC

Description of problem:
cdn-sync "importing packages" phase is 30% slower than for satellite-sync on same HW. Syncing rhel-x86_64-server-6, it looks like:

 * 5.6.0: Importing package metadata: 9880
 * 5.7.0: Importing package metadata: 10287
 * 5.8Beta: Importing package: 12997


Version-Release number of selected component (if applicable):
Satellite-5.8-RHEL-6-20170118.0-Satellite-x86_64


How reproducible:
1 of 1


Steps to Reproduce:
1. Compare import times on 5.8Beta and older satellites


Actual results:
It is 30% slower


Expected results:
It should be equally fast, but maybe these values can not be compared, maybe they are doing something else?

Comment 1 Tomas Lestach 2017-02-06 09:25:46 UTC

>  * 5.6.0: Importing package metadata: 9880
>  * 5.7.0: Importing package metadata: 10287
>  * 5.8Beta: Importing package: 12997

So, what are these numbers? Seconds?


In case this is only package metadata import data, what percentage does it make from the whole sync?
What are the overall sync times?

Comment 2 Jan Hutař 2017-02-06 09:34:32 UTC

Yes, these numbers are seconds.

Overall, cdn-sync is much faster (about 5 hours compared to much more - still running).

Comment 3 Jan Dobes 2017-02-06 10:07:37 UTC

It's hard to compare because satellite-sync is consuming package metadata from XML prepared on RHN and cdn-sync is extracting them from RPMs. However, it's worth investigating if this overhead really makes it 30% slower or there is something else to optimize.

Comment 4 Jan Dobes 2017-05-18 15:48:22 UTC

I think the speed of package import should be better now and also possibly improvable after --batch-size option was added. Basically, this bugzilla was opened when batch size was hardcoded 1. Default value is now 10.

Btw. satellite-sync is using default value 50 but as far as we found out it's not much faster + memory consumption will rise. Also satellite-sync has other bottlenecks. But we can experiment with this value in cdn-sync and change default value if needed.