Bugzilla (bugzilla.redhat.com) will be under maintenance for infrastructure upgrades and will not be available on July 31st between 12:30 AM - 05:30 AM UTC. We appreciate your understanding and patience. You can follow status.redhat.com for details.
Bug 662244 - Lack of performance listing content for a repo (pulp-admin repo content --id=rhel5-i386)
Summary: Lack of performance listing content for a repo (pulp-admin repo content --id=...
Alias: None
Product: Pulp
Classification: Retired
Component: z_other
Version: unspecified
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
: ---
Assignee: Pradeep Kilambi
QA Contact: Preethi Thomas
Depends On:
Blocks: pulp-verified
TreeView+ depends on / blocked
Reported: 2010-12-11 00:00 UTC by Todd Sanders
Modified: 2011-08-16 14:02 UTC (History)
0 users

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Last Closed: 2011-08-16 14:02:59 UTC

Attachments (Terms of Use)

Description Todd Sanders 2010-12-11 00:00:41 UTC
Description of problem:

*Fedora 14
*Pulp v0.109

I created and sync'd the RHEL5, i386 repo from the CDN.

[ec2-user@ip-10-203-21-224 /]$ sudo pulp-admin repo list
       List of Available Repositories

Label              	rhel5-i386               
Name               	rhel5-i386               
FeedURL            	https://cdn.redhat.com/content/dist/rhel/rhui/server/5Server/i386/os
FeedType           	yum                      
Arch               	noarch                   
Sync Schedule      	None                     
Packages           	7827                     
Files              	0                        
Distributions      	None                     
Publish            	True                     
Clones             	[]  

Then attempted to list out the package contents of the repo:

[ec2-user@ip-10-203-21-224 /]$ sudo pulp-admin repo content --id=rhel5-i386

Time Statistics:
real	0m53.257s
user	0m0.660s
sys	0m0.230s

So, 53 seconds to list the contents of the repo (7827 packages).

Listing the same files directly from the filesystem:

ls /data/pulp/repos/content/dist/rhel/rhui/server/5Server/i386/os/

Time Statistics:
real	0m0.192s
user	0m0.048s
sys	0m0.136s

Comment 1 Pradeep Kilambi 2010-12-22 18:49:46 UTC
commit 56d10a409787a8c1427a42cb9ee6b9db616ff267
Author: Pradeep Kilambi <pkilambi@redhat.com>
Date:   Fri Dec 17 11:28:52 2010 -0500

    Adding a call to pull in both errata and package info in one call

commit cc847e159ac95afa9a079442a882a8e97efbcb0f
Author: Pradeep Kilambi <pkilambi@redhat.com>
Date:   Thu Dec 16 14:37:15 2010 -0500

    improving the package filename fetch query to do a batch lookup to be more performant

$ time sudo pulp-admin repo content --id=f13-updates > /tmp/foo.txt

real	0m21.988s
user	0m0.943s
sys	0m0.101s

# of packages:

[pkilambi@prad ~]$ cat /tmp/foo.txt |grep rpm|wc -l

# of rest including errata:

$ cat /tmp/foo.txt |grep -v rpm|wc -l

Comment 2 Jay Dobies 2010-12-22 20:56:33 UTC
Fixed in 0.119.

Comment 3 Preethi Thomas 2011-01-03 21:00:49 UTC
[root@preethi ~]# rpm -q pulp

[root@preethi ~]#  time sudo pulp-admin repo content --id=f13-updates > /tmp/foo.txt

real	0m28.647s
user	0m0.305s
sys	0m0.090s
[root@preethi ~]# cat /tmp/foo.txt |grep rpm|wc -l
[root@preethi ~]# cat /tmp/foo.txt |grep -v rpm|wc -l

Comment 4 Preethi Thomas 2011-08-16 14:02:59 UTC
Closing with Community Release 15


Note You need to log in before you can comment on or make changes to this bug.