Bug 655143 - Speed up metadata updates for group/category data
Summary: Speed up metadata updates for group/category data
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Pulp
Classification: Retired
Component: z_other
Version: unspecified
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: Sprint 21
Assignee: John Matthews
QA Contact: Preethi Thomas
URL:
Whiteboard:
Depends On:
Blocks: verified-to-close
TreeView+ depends on / blocked
 
Reported: 2010-11-19 17:56 UTC by John Matthews
Modified: 2011-08-16 13:58 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2011-08-16 13:58:26 UTC
Embargoed:


Attachments (Terms of Use)

Description John Matthews 2010-11-19 17:56:13 UTC
Description of problem:

We should look to see if we can isolate comps.xml update separate from regenerating all metadata.  We are currently using "createrepo -update" and scanning all packages for changes, then making change for metadata.  

Usage of package groups/categories will be poor if we don't fix this performance issue.


Creating a single package group took almost 4 minutes on a pulp server that had 1 repo of 19k packages (fedora-12).  

$ time sudo pulp-admin -u admin -p admin packagegroup create -r f12 -n demo_zsh --id demo_zsh
Package group [demo_zsh] created in repository [f12]


real	3m50.234s
user	0m0.318s
sys	0m0.079s

Comment 1 John Matthews 2011-02-15 22:04:37 UTC
QE:  Sync a repo with a lot of packages, example below I'm using f12 updates with 9k rpms.  

PackageGroup operations should complete within a few seconds.

Below is an example.


$ sudo pulp-admin repo create --id f12_updates --feed yum:http://download.fedora.devel.redhat.com/pub/fedora/linux/updates/12/x86_64/
Successfully created repository [ f12_updates ]

$ sudo pulp-admin repo sync --id f12_updates -F
Sync for repository f12_updates started
Sync: Finished
12031/12031 new items downloaded
0/12031 existing items processed

Item Details: 
Rpms: 9645/9645
Delta_Rpms: 2386/2386


$ time sudo pulp-admin packagegroup create --name test_group1 --id test_group1 --repoid f12_updates
Package group [test_group1] created in repository [f12_updates]


real	0m3.562s
user	0m0.357s
sys	0m0.077s

Comment 2 Jeff Ortel 2011-02-16 20:09:34 UTC
Build: 0.139

Comment 3 Preethi Thomas 2011-05-31 18:47:53 UTC
verified
[root@preethi ~]# rpm -q pulp
pulp-0.0.181-1.fc14.noarch

[root@preethi ~]# pulp-admin repo sync --id=f13_updates -FSync for repository f13_updates started
Sync: Finished
15659/15659 new items downloaded
0/15659 existing items processed

Item Details: 
Rpms: 9616/9616
Delta_Rpms: 6043/6043


[root@preethi ~]# time sudo pulp-admin packagegroup create --name test_group1 --id test_group1 --repoid f13_updates
Package group [test_group1] created in repository [f13_updates]


real	0m2.606s
user	0m0.259s
sys	0m0.073s
[root@preethi ~]#

Comment 4 Preethi Thomas 2011-08-16 13:58:26 UTC
Closing with Community Release 15

pulp-0.0.223-4.


Note You need to log in before you can comment on or make changes to this bug.