Bug 640966 - migrate cf_target_release to new field Release
Summary: migrate cf_target_release to new field Release
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Bugzilla
Classification: Community
Component: Bugzilla General
Version: 3.6
Hardware: All
OS: Linux
low
medium
Target Milestone: ---
Assignee: Noura El hawary
QA Contact:
URL:
Whiteboard:
Depends On: 584956
Blocks:
TreeView+ depends on / blocked
 
Reported: 2010-10-07 11:59 UTC by Noura El hawary
Modified: 2013-06-24 03:40 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2012-05-10 00:54:32 UTC
Embargoed:


Attachments (Terms of Use)
v2 of script to migrate target_release values (1.44 KB, text/plain)
2010-11-04 10:29 UTC, Noura El hawary
nelhawar: review-
Details
v3 of script to migrate cf_targetrelease.. added change to bugs activity table (2.01 KB, patch)
2010-11-18 10:40 UTC, Noura El hawary
dkl: review+
Details | Diff

Description Noura El hawary 2010-10-07 11:59:08 UTC
write a separate script to migrate the current values from
the cf_target_release field to the new Release field. You could some as simple
as a regex that for example looks for 5\..* values and adds them to the RHEL5
product, 6\..* values are added to the RHEL6 product and so on. Some of the
others may need manual work to move.

Comment 1 Noura El hawary 2010-10-13 09:29:00 UTC
Hi Dave,

I started thinking about how to implement this, basically we need to migrate 2 things:

1- The actual values of the cf_target_release field to the right Release field in the right product, this can be done manually as it is easier than writing a script to do it, currently in the cf_target_release we have the following values:
 - 0.1.1,   0.1.2,   0.1.3,  0.2.0 => which product shall we move those values to?
 - 4.*  => those will be moved to RHEL4?
 - 5.*  => those will be moved to RHEL5?
 - 6.*  => those will be moved to RHEL6?

2- The values of the cf_target_release in the bugs, and those can be moved by writing a script that uses SQL queries to read the value of the cf_target_release from the bugs table and copies it to the target_release field in the bugs table for the same bug, and then deleting the value from the cf_target_release.

after that we can remove the cf_target_release.


how does that sound?

Thanks,
Noura

Comment 2 David Lawrence 2010-10-28 05:17:05 UTC
Thanks Noura. Let's go ahead and do the mentioned steps on bz-web2-test.devel.redhat.com and see what it looks like.

As for the 0.1.1, ... values, leave those in cf_target_release for
now and rename the old custom field description to "Internal Target Release".

We will talk to Andrius and others to see if they know who the odd values belong to and if the can eventually be removed.

Also go ahead and migrate the values over using SQL if possible from the old field to the new one for ones that we know about such as 4.*, 5.*, and 6.*. Do 
not migrate the values we are not yet sure of though.

Also look into using the bug_check_can_change_field hook in extensions/RedHat/Extension.pm for converting over the ACLs for the old custom field to the new field.

Thanks
Dave

Comment 3 David Lawrence 2010-11-02 20:47:51 UTC
Andrius, Chris, if you look back through the comments, we are hoping to migrate
the current values in the custom field Target Release to a new core Bugzilla field of the same name. Noura is working on a script to move them to the right products, such as 5.x to RHEL5, 6.x to RHEL6, etc. Also she is implementing similar access control code around the new field to disallow certain people from setting the values for certain products. Any objections to doing this so far? it will upset your bookmarks and/or save searches when it rolls out so we would need to discuss how to update those.

Also do you know what the other values are for and who owns those? such as 0.1.1, 0.1.2, 0.1.3, 0.2.0, etc. We could as a last resort leave the current custom field with those remaining values in it if we cannot track down who owns them. We would then rename the custom field to "Internal Target Release" or similar.

Thanks
Dave

Comment 4 Noura El hawary 2010-11-03 12:01:36 UTC
Created attachment 457415 [details]
v1 of script to migrate cf_targetreleases to the new target_release field in the RHEL product

Hi Dave,

attached is a script to do the migration on the bugs from cf_targetrelease to target_release field.. I didn't migrate the cf_targetrelease values that we don't know which product they belong to, I have run the script against the bz-db2 .. and all values are migrated, the only values that haven't been migrated are:

mysql> select bug_id, cf_targetrelease from bugs where cf_targetrelease > 0;
+--------+------------------+
| bug_id | cf_targetrelease |
+--------+------------------+
|    966 | 0.1.1            | 
| 548076 | 0.1.1            | 
| 552304 | 0.2.0            | 
| 629314 | 0.1.3            | 
+--------+------------------+
4 rows in set (1.49 sec)


also I have removed the values from cf_targetrelease that I migrated to the target_release field.

Comment 5 Noura El hawary 2010-11-03 12:05:26 UTC
Comment on attachment 457415 [details]
v1 of script to migrate cf_targetreleases to the new target_release field in the RHEL product

>#!/usr/bin/env perl 
>use strict;
>use warnings;
>use DBI;
>
># script:   migrate_targetrelease.pl
># descr:    one time script to migrate values from cf_targetrelease to target_release 
>#           field in the bugs table
># rt:       https://bugzilla.redhat.com/show_bug.cgi?id=640966
>
>###############################################################################
># connect to the database
>###############################################################################
>my $user = 'bugs';
>my $pass = '';
>my $host = 'bz-db2-test.devel.redhat.com';
>
>my $dsn = "dbi:mysql:dbname=bugs36_devel;host=$host";
>my $dbh = DBI->connect(
>    $dsn, $user, $pass,
>    {   PrintError => 0,
>        RaiseError => 1,
>        AutoCommit => 1,
>    }
>) or die "Couldn't connect database: " . DBI->errstr;
>
>##############################################################################
>
>my $MIGRATION = 1;
>
># read the bug ids and the cf_targetrelease from the bugs table and move
># the cf_targetrelease to the column target_release, then delete the column
># cf_targetrelease
>my $sth = $dbh->prepare("SELECT bug_id, cf_targetrelease 
>                           FROM bugs  
>                          WHERE cf_targetrelease > 1
>                       ORDER BY bug_id");
>$sth->execute;
>while ( my $row = $sth->fetchrow_arrayref ) {
>    my ( $id, $cf_targetrelease ) = @$row;
>
>    my $q = "UPDATE bugs SET target_release='$cf_targetrelease' WHERE bug_id=$id";
>    print "$q\n";
>    $dbh->do($q) if $MIGRATION;
>
>    $q = "UPDATE bugs SET cf_targetrelease='' WHERE bug_id=$id AND cf_targetrelease='$cf_targetrelease'";
>    print "$q\n";
>    $dbh->do($q) if $MIGRATION;
>
>}
>
>

Comment 6 Noura El hawary 2010-11-03 12:32:23 UTC
Hi Dave,

with regards to the extension bug_check_can_change_field hook in extensions/RedHat/Extension.pm and to use it for adding the old ACLs of the cf_targetrelease to the new target_release, I was working on implementing that but then found that currently for the cf_targetrelease we have the following permissions:

- if the user is a partner user or in the emp or the devel group then they can read and write.

- the user is only a redhat then they can see only, but the field is disabled for them.

- if the user is non of the above then the field is hidden from them.

so this means that there is still a visibility code that we need to add there, and code to enable and disable the field. if we use the only the extension you mentioned then the field will always be visible to anyone but only if an authorized user tries to change it then they will get error message. I guess we still need to have the code in the extension template to hide the field for an authorized access but we will add the product name to the authorization.

Regards,
Noura

Comment 8 David Lawrence 2010-11-03 15:29:41 UTC
(In reply to comment #6)
> Hi Dave,
> 
> with regards to the extension bug_check_can_change_field hook in
> extensions/RedHat/Extension.pm and to use it for adding the old ACLs of the
> cf_targetrelease to the new target_release, I was working on implementing that
> but then found that currently for the cf_targetrelease we have the following
> permissions:
> 
> - if the user is a partner user or in the emp or the devel group then they can
> read and write.
> 
> - the user is only a redhat then they can see only, but the field is disabled
> for them.

use bug_check_can_change_field for these two.

> 
> - if the user is non of the above then the field is hidden from them.

Then in addition use bug_filter_fields for the above requirement.

So it will be a combination effort to make it work as before.

Dave

Comment 9 Andrius Benokraitis 2010-11-03 17:24:17 UTC
(In reply to comment #3)
> Andrius, Chris, if you look back through the comments, we are hoping to migrate
> the current values in the custom field Target Release to a new core Bugzilla
> field of the same name. Noura is working on a script to move them to the right
> products, such as 5.x to RHEL5, 6.x to RHEL6, etc. Also she is implementing
> similar access control code around the new field to disallow certain people
> from setting the values for certain products. Any objections to doing this so
> far? it will upset your bookmarks and/or save searches when it rolls out so we
> would need to discuss how to update those.

No objections, but yeah it will make my canned searches unhappy I assume.

> 
> Also do you know what the other values are for and who owns those? such as
> 0.1.1, 0.1.2, 0.1.3, 0.2.0, etc. We could as a last resort leave the current
> custom field with those remaining values in it if we cannot track down who owns
> them. We would then rename the custom field to "Internal Target Release" or
> similar.

I'm not aware of those values or what they were for.

> 
> Thanks
> Dave

Comment 10 Noura El hawary 2010-11-04 10:29:03 UTC
Created attachment 457759 [details]
v2 of script to migrate target_release values

Thanks for the review Dave , another version of the script with your suggestions, I have actually already run the script against bz-db2 yesterday, and the reason I had the select statement having cf_target_release > 1 was to ignore migrating the values that we are not sure where to migrate it to.

Regards,
Noura

Comment 11 Noura El hawary 2010-11-04 10:30:46 UTC
(In reply to comment #9)
> (In reply to comment #3)
> > Andrius, Chris, if you look back through the comments, we are hoping to migrate
> > the current values in the custom field Target Release to a new core Bugzilla
> > field of the same name. Noura is working on a script to move them to the right
> > products, such as 5.x to RHEL5, 6.x to RHEL6, etc. Also she is implementing
> > similar access control code around the new field to disallow certain people
> > from setting the values for certain products. Any objections to doing this so
> > far? it will upset your bookmarks and/or save searches when it rolls out so we
> > would need to discuss how to update those.
> 
> No objections, but yeah it will make my canned searches unhappy I assume.
> 
> > 
> > Also do you know what the other values are for and who owns those? such as
> > 0.1.1, 0.1.2, 0.1.3, 0.2.0, etc. We could as a last resort leave the current
> > custom field with those remaining values in it if we cannot track down who owns
> > them. We would then rename the custom field to "Internal Target Release" or
> > similar.
> 
> I'm not aware of those values or what they were for.

Hi Andrius,

here is a list of the bugs with the releases that we are not sure about to help you and Chris decide which products shall we be moving them to:

+--------+------------------+
| bug_id | cf_targetrelease |
+--------+------------------+
|    966 | 0.1.1            | 
| 548076 | 0.1.1            | 
| 552304 | 0.2.0            | 
| 629314 | 0.1.3            | 
+--------+------------------+

Thanks,
Noura

> 
> > 
> > Thanks
> > Dave

Comment 12 Andrius Benokraitis 2010-11-04 13:21:37 UTC
> Hi Andrius,
> 
> here is a list of the bugs with the releases that we are not sure about to help
> you and Chris decide which products shall we be moving them to:
> 
> +--------+------------------+
> | bug_id | cf_targetrelease |
> +--------+------------------+
> |    966 | 0.1.1            | 
> | 548076 | 0.1.1            | 
> | 552304 | 0.2.0            | 
> | 629314 | 0.1.3            | 
> +--------+------------------+
> 
> Thanks,
> Noura
> 

Yeah, these aren't RHEL versions. I'd kill em! :-)

Comment 13 David Lawrence 2010-11-05 20:55:55 UTC
Comment on attachment 457759 [details]
v2 of script to migrate target_release values

Thanks Noura. Additionally, we will need to update bugs_activity as well. Should be simple as changing the field id from the cf_targetrelease value to the value for target_release instead. Please add that to your script and test please.

Dave

Comment 14 Noura El hawary 2010-11-08 10:40:13 UTC
Comment on attachment 457759 [details]
v2 of script to migrate target_release values

wrong bugs

Comment 15 Chris Ward 2010-11-08 10:57:47 UTC
I have no objections here. I'm curious though, why the sudden migration from cf_ to core field?

Comment 16 David Lawrence 2010-11-08 16:00:51 UTC
(In reply to comment #15)
> I have no objections here. I'm curious though, why the sudden migration from
> cf_ to core field?

So each product can have it's own unique set of values instead of mixing all together. This was a requirement from jira migration.  

Dave

Comment 17 Noura El hawary 2010-11-18 10:40:32 UTC
Created attachment 461263 [details]
v3 of script to migrate cf_targetrelease.. added change to bugs activity table

added code to change bugs_activity table to the new target_release field

Comment 18 David Lawrence 2010-11-23 22:11:49 UTC
(In reply to comment #17)
> Created attachment 461263 [details]
> v3 of script to migrate cf_targetrelease.. added change to bugs activity table
> 
> added code to change bugs_activity table to the new target_release field

Looks good Noura. We will need to test it very well though to make sure data is not lost. We can try it again on the next data refresh on bz-web2-test.

Dave

Comment 19 Noura El hawary 2011-01-06 13:11:19 UTC
Hi Andrius & Chris,

Just wanted to confirm with you.. Are you guys happy with us deleting the target_releases below (as they don not belong to rhel products) as Andrius mentioned.. we are going to push those changes live to production next week. and we just need your confirmation.

Thanks,
Noura

(In reply to comment #12)
> > Hi Andrius,
> > 
> > here is a list of the bugs with the releases that we are not sure about to help
> > you and Chris decide which products shall we be moving them to:
> > 
> > +--------+------------------+
> > | bug_id | cf_targetrelease |
> > +--------+------------------+
> > |    966 | 0.1.1            | 
> > | 548076 | 0.1.1            | 
> > | 552304 | 0.2.0            | 
> > | 629314 | 0.1.3            | 
> > +--------+------------------+
> > 
> > Thanks,
> > Noura
> > 
> 
> Yeah, these aren't RHEL versions. I'd kill em! :-)

Comment 20 Andrius Benokraitis 2011-01-06 14:41:15 UTC
(In reply to comment #19)
> Hi Andrius & Chris,
> 
> Just wanted to confirm with you.. Are you guys happy with us deleting the
> target_releases below (as they don not belong to rhel products) as Andrius
> mentioned.. we are going to push those changes live to production next week.
> and we just need your confirmation.
> 
> Thanks,
> Noura
> 

ACK!

Comment 21 Chris Ward 2011-01-10 13:34:17 UTC
No problem here.

Comment 22 Noura El hawary 2011-01-14 15:04:55 UTC
Hi All,

Please note that this change has been pushed to our staging bugzilla server:

https://partner-bugzilla.redhat.com/

In preparation for it to go live in production bugzilla servers next week. it will be great if you can take the time to test the functinality in partner-bugzilla and report any issues if found.

Thanks,
Noura

Comment 25 Simon Green 2012-05-10 00:54:32 UTC
Change was put live some time before July last year.

  -- simon


Note You need to log in before you can comment on or make changes to this bug.