Bug 848828

Summary:	re-promotion(sync) on a cloned repo takes longer due to additional filesystem checks
Product:	[Retired] Pulp	Reporter:	Pradeep Kilambi <pkilambi>
Component:	user-experience	Assignee:	Pradeep Kilambi <pkilambi>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Preethi Thomas <pthomas>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	1.2.0	CC:	bkearney, jortel, tsanders
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2013-01-07 14:11:57 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Pradeep Kilambi 2012-08-16 14:09:33 UTC

Description of problem:


The second, but less severe issue is the time we are taking to re-promote a product or repo.  Whenever you re-promote a product and its repos you end up calling the Pulp /sync/ command on each repo in the target environment.  At first this sounds harmless but it does take a good amount of time even if the repos are entirely the same between parent and child.  In most cases you are spending a long time on a glorified NOOP if there is no content change between Library -> Dev.

If there was some way we could compare the parent to the child and not do an entire sync if there were no differences we could probably reduce the speed way down on re-promotions.

Comment 1 Pradeep Kilambi 2012-08-17 13:46:21 UTC

fixed! commit d3ebf40b5c953c265c406c2f58b769dbaad54591

Test set up:

RHEL 6 repo with 3592 items to sync/clone

* Sync the repo:

$ pulp-admin repo sync --id=testel6 -F
Sync for repository testel6 started
Sync: Finished
3592/3592 new items downloaded
0/3592 existing items processed

Item Details:
Tree Files: 7/7
RPMs: 3585/3585

* clone the repo with parent feed(defaults):

$ time pulp-admin repo clone --id=testel6 --clone_id=testel6_cln -F
Repository [testel6] is being cloned as [testel6_cln]
Clone: Finished
Item Details:
Tree Files: 7/7
RPMs: 3585/3585


real    0m35.173s
user    0m2.075s
sys     0m0.142s

* Re-Promotion without fix:

$ time pulp-admin repo sync --id=testel6_cln -F
Sync for repository testel6_cln started
Sync: Finished
2/3592 new items downloaded
3590/3592 existing items processed

Item Details: 
Tree Files: 7/7
RPMs: 3585/3585


real    5m43.505s
user    0m20.055s
sys     0m0.668s

* Re-promotion with fix:

$ time pulp-admin repo sync --id=testel6_cln -F
Sync for repository testel6_cln started
Sync: Finished
0/3592 new items downloaded
3592/3592 existing items processed

Item Details:
Tree Files: 7/7
RPMs: 3585/3585


real    0m16.144s
user    0m1.065s
sys     0m0.105s

With fix the process took 16seconds vs 5minutes 43seconds, we are basically leveraging the improvements we made to the clone process thus improving the performance.

Comment 2 Jeff Ortel 2012-08-17 22:27:02 UTC

build: 1.1.12

Comment 3 Preethi Thomas 2012-08-20 15:41:17 UTC

verified

[root@pulp-v1-server ~]# rpm -q pulp
pulp-1.1.12-1.el6.noarch
[root@pulp-v1-server ~]# 


[root@pulp-v1-server ~]# pulp-admin -u admin -p admin repo sync --id rhel6_2 -FSync for repository rhel6_2 started
Sync: Finished
7287/7287 new items downloaded
0/7287 existing items processed

Item Details: 
Tree Files: 6/6
RPMs: 7281/7281

[root@pulp-v1-server ~]# time pulp-admin -u admin -p admin repo clone --id=rhel6_2 --clone_id=rhel6_2_cln -F
Repository [rhel6_2] is being cloned as [rhel6_2_cln]
Clone: Finished
Item Details: 
Tree Files: 6/6
RPMs: 7281/7281


real	1m14.411s
user	0m3.294s
sys	0m0.225s
[root@pulp-v1-server ~]# time pulp-admin -u admin -p admin repo sync --id=rhel6_2_cln -F
Sync for repository rhel6_2_cln started
Sync: Finished
0/7287 new items downloaded
7287/7287 existing items processed

Item Details: 
Tree Files: 6/6
RPMs: 7281/7281


real	0m38.605s
user	0m1.744s
sys	0m0.167s
[root@pulp-v1-server ~]#

Comment 4 Preethi Thomas 2013-01-07 14:11:57 UTC

Pulp 2.0 released.