Bug 655143

Summary:	Speed up metadata updates for group/category data
Product:	[Retired] Pulp	Reporter:	John Matthews <jmatthew>
Component:	z_other	Assignee:	John Matthews <jmatthew>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Preethi Thomas <pthomas>
Severity:	medium	Docs Contact:
Priority:	medium
Version:	unspecified	CC:	jason.dobies, jortel
Target Milestone:	---	Keywords:	Triaged
Target Release:	Sprint 21
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2011-08-16 13:58:26 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	647488

Description John Matthews 2010-11-19 17:56:13 UTC

Description of problem:

We should look to see if we can isolate comps.xml update separate from regenerating all metadata.  We are currently using "createrepo -update" and scanning all packages for changes, then making change for metadata.  

Usage of package groups/categories will be poor if we don't fix this performance issue.


Creating a single package group took almost 4 minutes on a pulp server that had 1 repo of 19k packages (fedora-12).  

$ time sudo pulp-admin -u admin -p admin packagegroup create -r f12 -n demo_zsh --id demo_zsh
Package group [demo_zsh] created in repository [f12]


real	3m50.234s
user	0m0.318s
sys	0m0.079s

Comment 1 John Matthews 2011-02-15 22:04:37 UTC

QE:  Sync a repo with a lot of packages, example below I'm using f12 updates with 9k rpms.  

PackageGroup operations should complete within a few seconds.

Below is an example.


$ sudo pulp-admin repo create --id f12_updates --feed yum:http://download.fedora.devel.redhat.com/pub/fedora/linux/updates/12/x86_64/
Successfully created repository [ f12_updates ]

$ sudo pulp-admin repo sync --id f12_updates -F
Sync for repository f12_updates started
Sync: Finished
12031/12031 new items downloaded
0/12031 existing items processed

Item Details: 
Rpms: 9645/9645
Delta_Rpms: 2386/2386


$ time sudo pulp-admin packagegroup create --name test_group1 --id test_group1 --repoid f12_updates
Package group [test_group1] created in repository [f12_updates]


real	0m3.562s
user	0m0.357s
sys	0m0.077s

Comment 2 Jeff Ortel 2011-02-16 20:09:34 UTC

Build: 0.139

Comment 3 Preethi Thomas 2011-05-31 18:47:53 UTC

verified
[root@preethi ~]# rpm -q pulp
pulp-0.0.181-1.fc14.noarch

[root@preethi ~]# pulp-admin repo sync --id=f13_updates -FSync for repository f13_updates started
Sync: Finished
15659/15659 new items downloaded
0/15659 existing items processed

Item Details: 
Rpms: 9616/9616
Delta_Rpms: 6043/6043


[root@preethi ~]# time sudo pulp-admin packagegroup create --name test_group1 --id test_group1 --repoid f13_updates
Package group [test_group1] created in repository [f13_updates]


real	0m2.606s
user	0m0.259s
sys	0m0.073s
[root@preethi ~]#

Comment 4 Preethi Thomas 2011-08-16 13:58:26 UTC

Closing with Community Release 15

pulp-0.0.223-4.