Description of problem:
cdn-sync "importing packages" phase is 30% slower than for satellite-sync on same HW. Syncing rhel-x86_64-server-6, it looks like:
* 5.6.0: Importing package metadata: 9880
* 5.7.0: Importing package metadata: 10287
* 5.8Beta: Importing package: 12997
Version-Release number of selected component (if applicable):
1 of 1
Steps to Reproduce:
1. Compare import times on 5.8Beta and older satellites
It is 30% slower
It should be equally fast, but maybe these values can not be compared, maybe they are doing something else?
> * 5.6.0: Importing package metadata: 9880
> * 5.7.0: Importing package metadata: 10287
> * 5.8Beta: Importing package: 12997
So, what are these numbers? Seconds?
In case this is only package metadata import data, what percentage does it make from the whole sync?
What are the overall sync times?
Yes, these numbers are seconds.
Overall, cdn-sync is much faster (about 5 hours compared to much more - still running).
It's hard to compare because satellite-sync is consuming package metadata from XML prepared on RHN and cdn-sync is extracting them from RPMs. However, it's worth investigating if this overhead really makes it 30% slower or there is something else to optimize.
I think the speed of package import should be better now and also possibly improvable after --batch-size option was added. Basically, this bugzilla was opened when batch size was hardcoded 1. Default value is now 10.
Btw. satellite-sync is using default value 50 but as far as we found out it's not much faster + memory consumption will rise. Also satellite-sync has other bottlenecks. But we can experiment with this value in cdn-sync and change default value if needed.