Bug 805193

Summary: migration 35 is very slow when lots of errata are present
Product: [Retired] Pulp Reporter: James Slagle <jslagle>
Component: z_otherAssignee: James Slagle <jslagle>
Status: CLOSED CURRENTRELEASE QA Contact: Preethi Thomas <pthomas>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 1.1.0CC: kbidarka, sghai, tsanders
Target Milestone: ---Keywords: Triaged
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 805192
: 805195 (view as bug list) Environment:
Last Closed: 2012-05-25 14:14:32 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On: 805192    
Bug Blocks: 805195    

Description James Slagle 2012-03-20 16:25:17 UTC 
+++ This bug was initially created as a clone of Bug #805192 +++

migration 35 is very slow when lots of errata are present

The EC2 rhui's have 3000+ errata and migration 35 is taking 2+ hours to run.

--- Additional comment from jslagle on 2012-03-20 12:24:27 EDT ---

Because this is RHUI and no one has yet migrated past version 35, we can modify migration version 35 directly.  This will make it so that we can add the needed indexes and rewrite the slow part of the migration to be much more efficient.

We can do these same changes in pulp, but we need an additional migration (version 39) to add the indexes.

We can then also pull migration 39 back into rhui so that the 2 branches stay on the same db version.  version 39 will essentially be a no-op in the rhui branch since the indexes were already added as part of 35.
Comment 1 James Slagle 2012-03-20 16:56:30 UTC 
cherry picked to pulp_v1 d8de721b9c0141d232f642cff0b3aa3e91f976e4
Comment 2 Jeff Ortel 2012-04-12 22:59:09 UTC 
build: 0.282
Comment 3 Preethi Thomas 2012-05-02 19:31:53 UTC 
verified
[root@pulp-v1-server ~]# rpm -q pulp
pulp-1.1.6-1.el6.noarch
[root@pulp-v1-server ~]# 

[[root@pulp-v1-server ~]# pulp-migrate
removing persisted tasks
database migration to version 40 complete


migration to data model version 37 started
migration to data model version 37 complete
migration to data model version 38 started
migration to data model version 38 complete
migration to data model version 39 started
Dropping index description_-1 from errata collection.
Dropping index description_-1 from package collection.
Dropping index packagegroups_-1 from package collection.
Dropping index packagegroupcategories_-1 from package collection.
Adding index on errata to Repo collection.
migration to data model version 39 complete
migration to data model version 40 started
migration to data model version 40 complete

[root@pulp-v1-server ~]# pulp-admin -u admin -p admin repo list
+------------------------------------------+
       List of Available Repositories
+------------------------------------------+

Id                 	rhel6_2                  
Name               	rhel6_2                  
Repo URL           	https://pulp-v1-server.usersys.redhat.com/pulp/repos/content/dist/rhel/rhui/server/6/6.2/x86_64/os/
Feed URL           	https://cdn.redhat.com/content/dist/rhel/rhui/server/6/6.2/x86_64/os/
Feed Type          	remote                   
Content Type       	yum                      
Feed Certs         	CA:Yes   Cert:Yes
Consumer Certs     	CA:No   Cert:No
Architecture       	noarch                   
Sync Schedule      	None                     
Packages           	7079                     
Files              	0                        
Distributions      	ks-Red Hat Enterprise Linux-Server-6.2-x86_64
Publish            	True                     
Clones             	[]                       
Groups             	None                     
Filters            	[]                       
Notes              	{}                       
Preserve Metadata  	False                    
Checksum Type      	sha                      


Id                 	zoo                      
Name               	zoo                      
Repo URL           	https://pulp-v1-server.usersys.redhat.com/pulp/repos/zoo/
Feed URL           	http://repos.fedorapeople.org/repos/katello/katello/6Server/x86_64/
Feed Type          	remote                   
Content Type       	yum                      
Feed Certs         	CA:No   Cert:No
Consumer Certs     	CA:No   Cert:No
Architecture       	noarch                   
Sync Schedule      	None                     
Packages           	159                      
Files              	0                        
Distributions      	None                     
Publish            	True                     
Clones             	[]                       
Groups             	None                     
Filters            	[]                       
Notes              	{u' ': u' '}             
Preserve Metadata  	False                    
Checksum Type      	sha256
Comment 4 Preethi Thomas 2012-05-25 14:14:32 UTC 
Pulp v1.1 Release