Bug 1053351

Summary: script to migrate data from bz db to hwcert db for public veiw
Product: [Retired] Red Hat Hardware Certification Program Reporter: Tony Fu <tfu>
Component: Hardware CatalogAssignee: Tony Fu <tfu>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 5.3CC: hwcert-catalog, hzhang, junwang, rlandry
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-03-14 05:03:43 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1059044, 1059926, 1059928, 1059929, 1061026, 1061029, 1062107, 1062157    
Bug Blocks: 1053334    
Attachments:
Description Flags
public view db schema
none
sql queries to make db shema changes for public view
none
script to generate data for two new tables - products and versions
none
sql queries to make db shema changes for public view none

Description Tony Fu 2014-01-15 05:00:44 UTC
Description of problem:


We need a script to migrate data from bz db to hwcert db for existing public certs.

Comment 1 Huan Zhang 2014-01-15 08:30:11 UTC
We can try to write this script in Python, so that it will be more easy for us to maintain.

Comment 2 Rob Landry 2014-01-15 19:30:04 UTC
(In reply to Huan Zhang from comment #1)
> We can try to write this script in Python, so that it will be more easy for
> us to maintain.

This sentence doesn't include any claims to back it's assertion.


(In reply to Tony Fu from comment #0)
> Description of problem:
> 
> 
> We need a script to migrate data from bz db to hwcert db for existing public
> certs.

This needs to be further clarified, what each script will do, how it will be run how it will be verified, etc. etc.


More likely this should be a (set of) management tool(s) that is(are) part of the catalog administration.  Also recall that the data will need to be cleaned, there is still rubbish in there like weird all caps version mixed with the normal capitalization and makes that need to be merged.  Doing these as a one off script means they cannot be reused to clean up future issue.

Comment 3 Huan Zhang 2014-01-16 01:30:32 UTC
(In reply to Rob Landry from comment #2)
> (In reply to Huan Zhang from comment #1)
> > We can try to write this script in Python, so that it will be more easy for
> > us to maintain.
> 
> This sentence doesn't include any claims to back it's assertion.
> 
Please ignore comment #1 :)

> 
> (In reply to Tony Fu from comment #0)
> > Description of problem:
> > 
> > 
> > We need a script to migrate data from bz db to hwcert db for existing public
> > certs.
> 
> This needs to be further clarified, what each script will do, how it will be
> run how it will be verified, etc. etc.
> 
> 
> More likely this should be a (set of) management tool(s) that is(are) part
> of the catalog administration.  Also recall that the data will need to be
> cleaned, there is still rubbish in there like weird all caps version mixed
> with the normal capitalization and makes that need to be merged.  Doing
> these as a one off script means they cannot be reused to clean up future
> issue.

But a simple script is more easy to reach our purpose.

Comment 5 Tony Fu 2014-01-29 04:02:04 UTC
The fields we need for public view are

1. vendor/make/model/category
2. spec url
3. product url
4. support url
5. product
6. version
7.platforms
8. status (it will always display one constant value for public cert, we don't need to worry about it just now)
9. kbase data which is handled by other bugs.

So we need 7 data migration script for first 7 fields

Comment 6 Tony Fu 2014-01-29 04:05:18 UTC
(In reply to Tony Fu from comment #5)
> The fields we need for public view are
> 
> 1. vendor/make/model/category
> 2. spec url
> 3. product url
> 4. support url
> 5. product
> 6. version
> 7.platforms
> 8. status (it will always display one constant value for public cert, we
> don't need to worry about it just now)
> 9. kbase data which is handled by other bugs.
> 
> So we need 7 data migration script for first 7 fields

field 3 and 4 have been in hwcert db, we won't need script to migrate them.

Comment 7 Tony Fu 2014-02-13 10:35:45 UTC
Created attachment 862684 [details]
public view db schema

We probably should save all new fields as ids instead of strings, but since they will need data cleaning up, we just save a part of fields as ids and leave the rest as what they are in bz db.

Comment 8 Tony Fu 2014-02-14 00:12:17 UTC
Created attachment 863033 [details]
sql queries to make db shema changes for public view

Just combined all db schema changes sql files, so eng ops can make all required db schema changes in one go.

Please note, it doesn't include kbase related changes which is included in other bugs.

Comment 9 Tony Fu 2014-02-14 01:05:05 UTC
Created attachment 863035 [details]
script to generate data for two new tables - products and versions

Comment 10 Tony Fu 2014-02-14 06:43:44 UTC
Created attachment 863124 [details]
sql queries to make db shema changes for public view

Comment 11 Tony Fu 2014-03-14 05:03:43 UTC
verified on live.