Bug 801929

Summary: Repo model not migrated correctly when notes field is None
Product: [Retired] Pulp Reporter: James Slagle <jslagle>
Component: z_otherAssignee: James Slagle <jslagle>
Status: CLOSED CURRENTRELEASE QA Contact: Preethi Thomas <pthomas>
Severity: high Docs Contact:
Priority: unspecified    
Version: 1.1.0CC: jslagle, kbidarka, sghai, tsanders, whayutin
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 801161 Environment:
Last Closed: 2012-05-25 14:16:34 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 801161    
Bug Blocks:    

Description James Slagle 2012-03-09 21:04:43 UTC
+++ This bug was initially created as a clone of Bug #801161 +++

Description of problem:

recreate:
1. setup rhui 2.0.1
2. create redhat repos, protected custom repos, unprotected custom repos
3. validate rhui 2.0.1 works e2e
4. upgrade to 2.0.2 using https://engineering.redhat.com/trac/mgmt-integrated/wiki/cloude/rhui-202-release-notes
5. during the pulp-migrate step.. you'll see a failure like the one below.


[root@dhcp231-185 yum.repos.d]# pulp-migrate
1 errors on validation, see /var/log/pulp/db.log for details
[root@dhcp231-185 yum.repos.d]# less /var/log/pulp/db.log
[root@dhcp231-185 yum.repos.d]# cat /var/log/pulp/db.log
migration to data model version 1 started
migration to data model version 1 complete
migration to data model version 2 started
migration to data model version 2 complete
migration to data model version 3 started
migration to data model version 3 complete
migration to data model version 4 started
migration to data model version 4 complete
migration to data model version 5 started
migration to data model version 5 complete
migration to data model version 7 started
migration to data model version 7 complete
migration to data model version 8 started
migration to data model version 8 complete
migration to data model version 9 started
migration to data model version 9 complete
migration to data model version 10 started
migration to data model version 10 complete
migration to data model version 11 started
migration to data model version 11 complete
migration to data model version 12 started
migration to data model version 12 complete
migration to data model version 13 started
migration to data model version 13 complete
migration to data model version 14 started
migration to data model version 14 complete
migration to data model version 15 started
migration to data model version 15 complete
migration to data model version 16 started
migration to data model version 16 complete
migration to data model version 17 started
migration to data model version 17 complete
migration to data model version 18 started
migration to data model version 18 complete
migration to data model version 19 started
migration to data model version 19 complete
migration to data model version 20 started
migration to data model version 20 complete
migration to data model version 21 started
migration to data model version 21 complete
migration to data model version 22 started
migration to data model version 22 complete
migration to data model version 23 started
migration to data model version 23 complete
migration to data model version 24 started
validation on previous data model version started
validation complete; data model at version 25
migration to data model version 26 started
migration to data model version 26 complete
migration to data model version 27 started
migration to data model version 27 complete
migration to data model version 28 started
migration to data model version 28 complete
migration to data model version 29 started
migration to data model version 29 complete
migration to data model version 30 started
migration to data model version 30 complete
migration to data model version 31 started
migration to data model version 31 complete
migration to data model version 32 started
migration to data model version 32 complete
Moving /etc/pki/content/rhel-x86_64-5-rhui-1-rpms-5Server-x86_64/feed-rhel-x86_64-5-rhui-1-rpms-5Server-x86_64.cert to /etc/pki/pulp/content/rhel-x86_64-5-rhui-1-rpms-5Server-x86_64/feed-rhel-x86_64-5-rhui-1-rpms-5Server-x86_64.cert
Moving /etc/pki/content/rhel-x86_64-5-rhui-1-rpms-5Server-x86_64/feed-rhel-x86_64-5-rhui-1-rpms-5Server-x86_64.ca to /etc/pki/pulp/content/rhel-x86_64-5-rhui-1-rpms-5Server-x86_64/feed-rhel-x86_64-5-rhui-1-rpms-5Server-x86_64.ca
Moving /etc/pki/content/rhel-x86_64-5-rhui-1-rpms-5Server-x86_64/consumer-rhel-x86_64-5-rhui-1-rpms-5Server-x86_64.ca to /etc/pki/pulp/content/rhel-x86_64-5-rhui-1-rpms-5Server-x86_64/consumer-rhel-x86_64-5-rhui-1-rpms-5Server-x86_64.ca
Moving /etc/pki/content/rhel-x86_64-5-rhui-1-rpms-5Server-x86_64/consumer-rhel-x86_64-5-rhui-1-rpms-5Server-x86_64.cert to /etc/pki/pulp/content/rhel-x86_64-5-rhui-1-rpms-5Server-x86_64/consumer-rhel-x86_64-5-rhui-1-rpms-5Server-x86_64.cert
Moving /etc/pki/content/wes-custom2/consumer-wes-custom2.ca to /etc/pki/pulp/content/wes-custom2/consumer-wes-custom2.ca
Moving /etc/pki/content/wes-custom2/consumer-wes-custom2.cert to /etc/pki/pulp/content/wes-custom2/consumer-wes-custom2.cert
Moving /etc/pki/content/rhel-x86_64-6-rhui-2-rpms-6Server-x86_64/feed-rhel-x86_64-6-rhui-2-rpms-6Server-x86_64.cert to /etc/pki/pulp/content/rhel-x86_64-6-rhui-2-rpms-6Server-x86_64/feed-rhel-x86_64-6-rhui-2-rpms-6Server-x86_64.cert
Moving /etc/pki/content/rhel-x86_64-6-rhui-2-rpms-6Server-x86_64/feed-rhel-x86_64-6-rhui-2-rpms-6Server-x86_64.ca to /etc/pki/pulp/content/rhel-x86_64-6-rhui-2-rpms-6Server-x86_64/feed-rhel-x86_64-6-rhui-2-rpms-6Server-x86_64.ca
Moving /etc/pki/content/rhel-x86_64-6-rhui-2-rpms-6Server-x86_64/consumer-rhel-x86_64-6-rhui-2-rpms-6Server-x86_64.ca to /etc/pki/pulp/content/rhel-x86_64-6-rhui-2-rpms-6Server-x86_64/consumer-rhel-x86_64-6-rhui-2-rpms-6Server-x86_64.ca
Moving /etc/pki/content/rhel-x86_64-6-rhui-2-rpms-6Server-x86_64/consumer-rhel-x86_64-6-rhui-2-rpms-6Server-x86_64.cert to /etc/pki/pulp/content/rhel-x86_64-6-rhui-2-rpms-6Server-x86_64/consumer-rhel-x86_64-6-rhui-2-rpms-6Server-x86_64.cert
migration to data model version 34 started
migration to data model version 34 complete
migrating packages to include repoids field
migrating 48 packages
migrating errata to include repoids field
migrating 0 errata
migration to data model version 36 started
migration to data model version 36 complete
migration to data model version 37 started
migration to data model version 37 complete
model validation failure in Repo for model wes-custom: field notes is <type 'NoneType'> not <type 'dict'>

--- Additional comment from whayutin on 2012-03-07 15:25:31 EST ---

> db.repos.remove({'id' : 'wes-custom'})
> quit
function () {
    return nativeHelper.apply(quit_, arguments);
}
> exit
bye
[root@dhcp231-185 admin]# pulp-migrate 
data model in use matches the current version
database migration to version 37 complete
[root@dhcp231-185 admin]#

--- Additional comment from jslagle on 2012-03-07 16:07:57 EST ---

committed to pulp master: b3813c26e45eea286facca70cb19509bde1ed436
committed to pulp rhui: 95315f675291170de132fdeb3de32fd85fe2a820

built into pulp-0.263-16

--- Additional comment from jslagle on 2012-03-07 17:13:17 EST ---

Newer ISO build:
http://download.lab.bos.redhat.com/devel/candidates/RHEL-6.2-RHUI-2.0.2-20120307.
+2/2.0.2/Server/x86_64/iso/RHEL-6.2-RHUI-2.0.2-20120307.2-Server-x86_64-DVD1.iso

yum repo on cdn.rcm-qa also updated.

--- Additional comment from kbidarka on 2012-03-08 08:03:25 EST ---

migration to data model version 23 complete
migration to data model version 24 started
validation on previous data model version started
validation complete; data model at version 25
migration to data model version 26 started
migration to data model version 26 complete
migration to data model version 27 started
migration to data model version 27 complete
migration to data model version 28 started
migration to data model version 28 complete
migration to data model version 29 started
migration to data model version 29 complete
migration to data model version 30 started
migration to data model version 30 complete
migration to data model version 31 started
migration to data model version 31 complete
migration to data model version 32 started
migration to data model version 32 complete
Moving /etc/pki/content/rhel-x86_64-6-rhui-2-rpms-6Server-x86_64/feed-rhel-x86_64-6-rhui-2-rpms-6Server-x86_64.cert to /etc/pki/pulp/content/rhel-x86_64-6-rhui-2-rpms-6Server-x86_64/feed-rhel-x86_64-6-rhui-2-rpms-6Server-x86_64.cert
Moving /etc/pki/content/rhel-x86_64-6-rhui-2-rpms-6Server-x86_64/feed-rhel-x86_64-6-rhui-2-rpms-6Server-x86_64.ca to /etc/pki/pulp/content/rhel-x86_64-6-rhui-2-rpms-6Server-x86_64/feed-rhel-x86_64-6-rhui-2-rpms-6Server-x86_64.ca
Moving /etc/pki/content/rhel-x86_64-6-rhui-2-rpms-6Server-x86_64/consumer-rhel-x86_64-6-rhui-2-rpms-6Server-x86_64.ca to /etc/pki/pulp/content/rhel-x86_64-6-rhui-2-rpms-6Server-x86_64/consumer-rhel-x86_64-6-rhui-2-rpms-6Server-x86_64.ca
Moving /etc/pki/content/rhel-x86_64-6-rhui-2-rpms-6Server-x86_64/consumer-rhel-x86_64-6-rhui-2-rpms-6Server-x86_64.cert to /etc/pki/pulp/content/rhel-x86_64-6-rhui-2-rpms-6Server-x86_64/consumer-rhel-x86_64-6-rhui-2-rpms-6Server-x86_64.cert
Moving /etc/pki/content/custom_001/consumer-custom_001.ca to /etc/pki/pulp/content/custom_001/consumer-custom_001.ca
Moving /etc/pki/content/custom_001/consumer-custom_001.cert to /etc/pki/pulp/content/custom_001/consumer-custom_001.cert
migration to data model version 34 started
migration to data model version 34 complete
migrating packages to include repoids field
migrating 35 packages
migrating errata to include repoids field
migrating 0 errata
migration to data model version 36 started
migration to data model version 36 complete
migration to data model version 37 started
migration to data model version 37 complete
migration to data model version 38 started
migration to data model version 38 complete

Comment 1 James Slagle 2012-03-09 21:10:50 UTC
the notes files on the repo model can not be None, it should be {}.

I added a new migration version 38 to fix this.

Comment 2 James Slagle 2012-03-09 21:34:33 UTC
cherry picked to pulp_v1 95315f675291170de132fdeb3de32fd85fe2a820

Comment 3 Jeff Ortel 2012-03-10 01:32:43 UTC
build: 0.273

Comment 4 Preethi Thomas 2012-05-02 19:28:05 UTC
[[root@pulp-v1-server ~]# pulp-migrate
removing persisted tasks
database migration to version 40 complete


migration to data model version 37 started
migration to data model version 37 complete
migration to data model version 38 started
migration to data model version 38 complete
migration to data model version 39 started
Dropping index description_-1 from errata collection.
Dropping index description_-1 from package collection.
Dropping index packagegroups_-1 from package collection.
Dropping index packagegroupcategories_-1 from package collection.
Adding index on errata to Repo collection.
migration to data model version 39 complete
migration to data model version 40 started
migration to data model version 40 complete

[root@pulp-v1-server ~]# pulp-admin -u admin -p admin repo list
+------------------------------------------+
       List of Available Repositories
+------------------------------------------+

Id                 	rhel6_2                  
Name               	rhel6_2                  
Repo URL           	https://pulp-v1-server.usersys.redhat.com/pulp/repos/content/dist/rhel/rhui/server/6/6.2/x86_64/os/
Feed URL           	https://cdn.redhat.com/content/dist/rhel/rhui/server/6/6.2/x86_64/os/
Feed Type          	remote                   
Content Type       	yum                      
Feed Certs         	CA:Yes   Cert:Yes
Consumer Certs     	CA:No   Cert:No
Architecture       	noarch                   
Sync Schedule      	None                     
Packages           	7079                     
Files              	0                        
Distributions      	ks-Red Hat Enterprise Linux-Server-6.2-x86_64
Publish            	True                     
Clones             	[]                       
Groups             	None                     
Filters            	[]                       
Notes              	{}                       
Preserve Metadata  	False                    
Checksum Type      	sha                      


Id                 	zoo                      
Name               	zoo                      
Repo URL           	https://pulp-v1-server.usersys.redhat.com/pulp/repos/zoo/
Feed URL           	http://repos.fedorapeople.org/repos/katello/katello/6Server/x86_64/
Feed Type          	remote                   
Content Type       	yum                      
Feed Certs         	CA:No   Cert:No
Consumer Certs     	CA:No   Cert:No
Architecture       	noarch                   
Sync Schedule      	None                     
Packages           	159                      
Files              	0                        
Distributions      	None                     
Publish            	True                     
Clones             	[]                       
Groups             	None                     
Filters            	[]                       
Notes              	{u' ': u' '}             
Preserve Metadata  	False                    
Checksum Type      	sha256                   


[root@pulp-v1-server ~]#

Comment 5 Preethi Thomas 2012-05-25 14:16:34 UTC
Pulp v1.1 Release