Bug 662244 - Lack of performance listing content for a repo (pulp-admin repo content --id=rhel5-i386)
Summary: Lack of performance listing content for a repo (pulp-admin repo content --id=...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Pulp
Classification: Retired
Component: z_other
Version: unspecified
Hardware: Unspecified
OS: Unspecified
low
medium
Target Milestone: ---
: ---
Assignee: Pradeep Kilambi
QA Contact: Preethi Thomas
URL:
Whiteboard:
Depends On:
Blocks: pulp-verified
TreeView+ depends on / blocked
 
Reported: 2010-12-11 00:00 UTC by Todd Sanders
Modified: 2011-08-16 14:02 UTC (History)
0 users

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2011-08-16 14:02:59 UTC
Embargoed:


Attachments (Terms of Use)

Description Todd Sanders 2010-12-11 00:00:41 UTC
Description of problem:

*Fedora 14
*Pulp v0.109

I created and sync'd the RHEL5, i386 repo from the CDN.

[ec2-user@ip-10-203-21-224 /]$ sudo pulp-admin repo list
+------------------------------------------+
       List of Available Repositories
+------------------------------------------+

Label              	rhel5-i386               
Name               	rhel5-i386               
FeedURL            	https://cdn.redhat.com/content/dist/rhel/rhui/server/5Server/i386/os
FeedType           	yum                      
Arch               	noarch                   
Sync Schedule      	None                     
Packages           	7827                     
Files              	0                        
Distributions      	None                     
Publish            	True                     
Clones             	[]  

Then attempted to list out the package contents of the repo:

[ec2-user@ip-10-203-21-224 /]$ sudo pulp-admin repo content --id=rhel5-i386

Time Statistics:
real	0m53.257s
user	0m0.660s
sys	0m0.230s

So, 53 seconds to list the contents of the repo (7827 packages).

Listing the same files directly from the filesystem:

ls /data/pulp/repos/content/dist/rhel/rhui/server/5Server/i386/os/

Time Statistics:
real	0m0.192s
user	0m0.048s
sys	0m0.136s

Comment 1 Pradeep Kilambi 2010-12-22 18:49:46 UTC
commit 56d10a409787a8c1427a42cb9ee6b9db616ff267
Author: Pradeep Kilambi <pkilambi>
Date:   Fri Dec 17 11:28:52 2010 -0500

    Adding a call to pull in both errata and package info in one call


commit cc847e159ac95afa9a079442a882a8e97efbcb0f
Author: Pradeep Kilambi <pkilambi>
Date:   Thu Dec 16 14:37:15 2010 -0500

    improving the package filename fetch query to do a batch lookup to be more performant



$ time sudo pulp-admin repo content --id=f13-updates > /tmp/foo.txt

real	0m21.988s
user	0m0.943s
sys	0m0.101s

# of packages:

[pkilambi@prad ~]$ cat /tmp/foo.txt |grep rpm|wc -l
8174

# of rest including errata:

$ cat /tmp/foo.txt |grep -v rpm|wc -l
2497

Comment 2 Jay Dobies 2010-12-22 20:56:33 UTC
Fixed in 0.119.

Comment 3 Preethi Thomas 2011-01-03 21:00:49 UTC
verified
[root@preethi ~]# rpm -q pulp
pulp-0.0.119-1.fc14.noarch


[root@preethi ~]#  time sudo pulp-admin repo content --id=f13-updates > /tmp/foo.txt

real	0m28.647s
user	0m0.305s
sys	0m0.090s
[root@preethi ~]# cat /tmp/foo.txt |grep rpm|wc -l
8502
[root@preethi ~]# cat /tmp/foo.txt |grep -v rpm|wc -l
2613

Comment 4 Preethi Thomas 2011-08-16 14:02:59 UTC
Closing with Community Release 15

pulp-0.0.223-4.


Note You need to log in before you can comment on or make changes to this bug.