Bug 662244

Summary: Lack of performance listing content for a repo (pulp-admin repo content --id=rhel5-i386)
Product: [Retired] Pulp Reporter: Todd Sanders <tsanders>
Component: z_otherAssignee: Pradeep Kilambi <pkilambi>
Status: CLOSED CURRENTRELEASE QA Contact: Preethi Thomas <pthomas>
Severity: medium Docs Contact:
Priority: low    
Version: unspecified   
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-08-16 14:02:59 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 641987    

Description Todd Sanders 2010-12-11 00:00:41 UTC
Description of problem:

*Fedora 14
*Pulp v0.109

I created and sync'd the RHEL5, i386 repo from the CDN.

[ec2-user@ip-10-203-21-224 /]$ sudo pulp-admin repo list
+------------------------------------------+
       List of Available Repositories
+------------------------------------------+

Label              	rhel5-i386               
Name               	rhel5-i386               
FeedURL            	https://cdn.redhat.com/content/dist/rhel/rhui/server/5Server/i386/os
FeedType           	yum                      
Arch               	noarch                   
Sync Schedule      	None                     
Packages           	7827                     
Files              	0                        
Distributions      	None                     
Publish            	True                     
Clones             	[]  

Then attempted to list out the package contents of the repo:

[ec2-user@ip-10-203-21-224 /]$ sudo pulp-admin repo content --id=rhel5-i386

Time Statistics:
real	0m53.257s
user	0m0.660s
sys	0m0.230s

So, 53 seconds to list the contents of the repo (7827 packages).

Listing the same files directly from the filesystem:

ls /data/pulp/repos/content/dist/rhel/rhui/server/5Server/i386/os/

Time Statistics:
real	0m0.192s
user	0m0.048s
sys	0m0.136s

Comment 1 Pradeep Kilambi 2010-12-22 18:49:46 UTC
commit 56d10a409787a8c1427a42cb9ee6b9db616ff267
Author: Pradeep Kilambi <pkilambi>
Date:   Fri Dec 17 11:28:52 2010 -0500

    Adding a call to pull in both errata and package info in one call


commit cc847e159ac95afa9a079442a882a8e97efbcb0f
Author: Pradeep Kilambi <pkilambi>
Date:   Thu Dec 16 14:37:15 2010 -0500

    improving the package filename fetch query to do a batch lookup to be more performant



$ time sudo pulp-admin repo content --id=f13-updates > /tmp/foo.txt

real	0m21.988s
user	0m0.943s
sys	0m0.101s

# of packages:

[pkilambi@prad ~]$ cat /tmp/foo.txt |grep rpm|wc -l
8174

# of rest including errata:

$ cat /tmp/foo.txt |grep -v rpm|wc -l
2497

Comment 2 Jay Dobies 2010-12-22 20:56:33 UTC
Fixed in 0.119.

Comment 3 Preethi Thomas 2011-01-03 21:00:49 UTC
verified
[root@preethi ~]# rpm -q pulp
pulp-0.0.119-1.fc14.noarch


[root@preethi ~]#  time sudo pulp-admin repo content --id=f13-updates > /tmp/foo.txt

real	0m28.647s
user	0m0.305s
sys	0m0.090s
[root@preethi ~]# cat /tmp/foo.txt |grep rpm|wc -l
8502
[root@preethi ~]# cat /tmp/foo.txt |grep -v rpm|wc -l
2613

Comment 4 Preethi Thomas 2011-08-16 14:02:59 UTC
Closing with Community Release 15

pulp-0.0.223-4.