Bug 848828 - re-promotion(sync) on a cloned repo takes longer due to additional filesystem checks
re-promotion(sync) on a cloned repo takes longer due to additional filesystem...
Status: CLOSED CURRENTRELEASE
Product: Pulp
Classification: Community
Component: user-experience (Show other bugs)
1.2.0
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Pradeep Kilambi
Preethi Thomas
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-08-16 10:09 EDT by Pradeep Kilambi
Modified: 2013-09-09 12:35 EDT (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-01-07 09:11:57 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Pradeep Kilambi 2012-08-16 10:09:33 EDT
Description of problem:


The second, but less severe issue is the time we are taking to re-promote a product or repo.  Whenever you re-promote a product and its repos you end up calling the Pulp /sync/ command on each repo in the target environment.  At first this sounds harmless but it does take a good amount of time even if the repos are entirely the same between parent and child.  In most cases you are spending a long time on a glorified NOOP if there is no content change between Library -> Dev.

If there was some way we could compare the parent to the child and not do an entire sync if there were no differences we could probably reduce the speed way down on re-promotions.
Comment 1 Pradeep Kilambi 2012-08-17 09:46:21 EDT
fixed! commit d3ebf40b5c953c265c406c2f58b769dbaad54591

Test set up:

RHEL 6 repo with 3592 items to sync/clone

* Sync the repo:

$ pulp-admin repo sync --id=testel6 -F
Sync for repository testel6 started
Sync: Finished
3592/3592 new items downloaded
0/3592 existing items processed

Item Details:
Tree Files: 7/7
RPMs: 3585/3585

* clone the repo with parent feed(defaults):

$ time pulp-admin repo clone --id=testel6 --clone_id=testel6_cln -F
Repository [testel6] is being cloned as [testel6_cln]
Clone: Finished
Item Details:
Tree Files: 7/7
RPMs: 3585/3585


real    0m35.173s
user    0m2.075s
sys     0m0.142s

* Re-Promotion without fix:

$ time pulp-admin repo sync --id=testel6_cln -F
Sync for repository testel6_cln started
Sync: Finished
2/3592 new items downloaded
3590/3592 existing items processed

Item Details: 
Tree Files: 7/7
RPMs: 3585/3585


real    5m43.505s
user    0m20.055s
sys     0m0.668s

* Re-promotion with fix:

$ time pulp-admin repo sync --id=testel6_cln -F
Sync for repository testel6_cln started
Sync: Finished
0/3592 new items downloaded
3592/3592 existing items processed

Item Details:
Tree Files: 7/7
RPMs: 3585/3585


real    0m16.144s
user    0m1.065s
sys     0m0.105s

With fix the process took 16seconds vs 5minutes 43seconds, we are basically leveraging the improvements we made to the clone process thus improving the performance.
Comment 2 Jeff Ortel 2012-08-17 18:27:02 EDT
build: 1.1.12
Comment 3 Preethi Thomas 2012-08-20 11:41:17 EDT
verified

[root@pulp-v1-server ~]# rpm -q pulp
pulp-1.1.12-1.el6.noarch
[root@pulp-v1-server ~]# 


[root@pulp-v1-server ~]# pulp-admin -u admin -p admin repo sync --id rhel6_2 -FSync for repository rhel6_2 started
Sync: Finished
7287/7287 new items downloaded
0/7287 existing items processed

Item Details: 
Tree Files: 6/6
RPMs: 7281/7281

[root@pulp-v1-server ~]# time pulp-admin -u admin -p admin repo clone --id=rhel6_2 --clone_id=rhel6_2_cln -F
Repository [rhel6_2] is being cloned as [rhel6_2_cln]
Clone: Finished
Item Details: 
Tree Files: 6/6
RPMs: 7281/7281


real	1m14.411s
user	0m3.294s
sys	0m0.225s
[root@pulp-v1-server ~]# time pulp-admin -u admin -p admin repo sync --id=rhel6_2_cln -F
Sync for repository rhel6_2_cln started
Sync: Finished
0/7287 new items downloaded
7287/7287 existing items processed

Item Details: 
Tree Files: 6/6
RPMs: 7281/7281


real	0m38.605s
user	0m1.744s
sys	0m0.167s
[root@pulp-v1-server ~]#
Comment 4 Preethi Thomas 2013-01-07 09:11:57 EST
Pulp 2.0 released.

Note You need to log in before you can comment on or make changes to this bug.