Bug 406201 - 3.10: Commandline utilities and cron jobs
Summary: 3.10: Commandline utilities and cron jobs
Keywords:
Status: CLOSED NEXTRELEASE
Alias: None
Product: Bugzilla
Classification: Community
Component: Bugzilla General
Version: 3.2
Hardware: All
OS: Linux
high
medium
Target Milestone: ---
Assignee: Noura El hawary
QA Contact:
URL:
Whiteboard:
Depends On: 406261 406301
Blocks: RHBZ30UpgradeTracker 427059
TreeView+ depends on / blocked
 
Reported: 2007-11-30 16:58 UTC by David Lawrence
Modified: 2013-06-24 04:16 UTC (History)
0 users

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2008-05-22 06:36:28 UTC
Embargoed:


Attachments (Terms of Use)
command line utility mail_log.pl to mail log4perl logging (2.92 KB, patch)
2008-04-02 04:32 UTC, Noura El hawary
dkl: review-
Details | Diff
v2 for cron script mail_log.pl (2.70 KB, patch)
2008-04-14 04:10 UTC, Noura El hawary
dkl: review+
Details | Diff
cron script sanitycheck.pl (7.33 KB, patch)
2008-04-14 04:11 UTC, Noura El hawary
no flags Details | Diff
cron script for sanitycheck.pl (451 bytes, patch)
2008-04-14 14:34 UTC, Noura El hawary
dkl: review+
Details | Diff

Description David Lawrence 2007-11-30 16:58:12 UTC
Description:
Red Hat has implemented several command line Perl scripts for maintenance and debugging purposes. sanitycheck.pl emails a log nightly of possible data issues int he Bugzilla database. mail_log.pl emails nightly the bz.log file that contains entries helpful for performance debugging and benchmarking. bz-expire-cookies.pl cleans out the logincookies table in the Bugzilla database of stale entries.

Function Requirements:
mail_log.pl, sanitycheck.pl, and bz-expire-cookies.pl

Comment 1 Noura El hawary 2007-12-20 07:17:17 UTC
mail_log.pl ,, basically with this script not much change is required as it
doesn't interact with bugzilla modules much ,, mostly with external perl-modules
in its interaction with bugzilla ,, we will need to change the code that calls
Bugzilla::Config to use the Params() function as in bugzilla 3 it will use :
Bugzilla->params->{'maintainer'}; instead so this is bout 3 LOC ,, also it needs
to interact with Bugzilla.pm to get the log4perl config file details ,, and
basically this can not happen until we have done the log4perl porting to 3.2 so
will make this bug depending on the (406301 bugs for log4perl)


sanitycheck.pl : also this script will not require much change ,, only the
following:
1- replace Bugzilla::Config to use the Params() with
Bugzilla->params->{'maintainer'} as this is how it is done in  bz3.2 = 3 LOC
2- add check option for extra check found in 3.2 for flag consistency in
products and components = 5 LOC
3- this script needs to interact with Bugzilla/SanityCheck.pm so it is dependent
on the bug that port this module from 2.18 to 3.2 to replace sanitycheck.cgi
so will make also this bug dependent on bug 406261


bz-expire-cookies.pl ,, this script will need to be changed as it calls
Bugzilla::Auth::Cookie->clean_expired_tokens(); in bz 2_18 ,, 
However the way that cookies are handled in 3.2 is different and the module
structure is completely different. I don't think we currently have a task in the
upgrade for the cookies ,, maybe we need to create one,, but for that script the
code would only need to call a subroutine to clean the expired cookies so
expecting about 10 LOC


So basically TOTAL LOC = 25 LOC

Comment 2 Kevin Baker 2007-12-29 05:58:07 UTC
Noura,

please create a task for resolving the cookie code. 

Comment 3 Kevin Baker 2007-12-29 18:28:36 UTC
sorry, I sopke to soon. The cookie code is handled under bug 406231. I added 
it as a dependency

Comment 4 Noura El hawary 2008-04-02 04:32:45 UTC
Created attachment 300010 [details]
command line utility mail_log.pl to mail log4perl logging

First section of this bug is the commanline utility mail_log.pl to compress and
mail the generated log file from log4perl. as we are done with log4perl so I
thought i would get this done as well. Basically is it exactly similar to
rh_bugzilla_2_18 only one difference was replacing
Param("maintainer") with Bugzilla->params->{maintainer} 

I tested it on bugdev and it works good.

Noura

Comment 5 Noura El hawary 2008-04-02 04:34:15 UTC
worked 30 minutes. 

also one more thing where would be the best place to keep those cron utilities
will we still keep them under the main bugzilla directory ?

Noura

Comment 6 David Lawrence 2008-04-02 14:16:12 UTC
Comment on attachment 300010 [details]
command line utility mail_log.pl to mail log4perl logging

mail_log.pl is no longer necessary as I understand it since
eng-sysadmins now copy the log files nightly to a separate storage area
that we have access to. So I think we can omit that script now.

Dave

Comment 7 David Lawrence 2008-04-02 14:22:12 UTC
Comment on attachment 300010 [details]
command line utility mail_log.pl to mail log4perl logging

>Index: mail_log.pl
>===================================================================
>+
>+use MIME::QuotedPrint;
>+use MIME::Base64;
>+use Mail::Sendmail 0.75;    # doesn't work with v. 0.74!

use Bugzilla::Mailer::MessageToMTA here instead for 3.2

>+use File::Basename;
>+use Bugzilla::Config; # for Param method

This is no longer needed since you can just use Bugzilla->params->{param}.

>+use Log::Log4perl;
>+use Bugzilla;
>+
>+main(@ARGV) if not caller();
>+
>+sub main {
>+    my ( $sec, $min, $hour, $mday, $mon, $year, $wday, $yday, $isdst )
>+        = localtime(time);
>+    my $current_date = sprintf(
>+        "%.4d/%.2d/%.2d %.2d:%.2d:%.2d",
>+        $year + 1900,
>+        $mon + 1, $mday, $hour, $min, $sec
>+    );
>+
>+    my %mail = (
>+        from    => 'bugzilla',
>+        to      => Bugzilla->params->{maintainer},
>+        subject => "[$current_date] Bugzilla Log File",
>+        smtp    => 'localhost',
>+    );
>+
>+    my $boundary = "====" . time() . "====";
>+    $mail{'content-type'} = "multipart/mixed; boundary=\"$boundary\"";
>+
>+    my $message = encode_qp("Bugzilla Log File");
>+
>+    # Get the log file name from Log4perl config file
>+    my $log_file = Bugzilla->log_filename;
>+
>+    # Exit if file does not exist
>+    return if !-e $log_file;
>+
>+    # Gzip the current log file and rename to logfile.gz
>+    my $tmp_file = $log_file . ".gz";
>+    system("/bin/gzip -f $log_file") == 0
>+        or die "Error executing gzip -f $log_file";
>+
>+    open( F, $tmp_file ) or die "Cannot read $tmp_file: $!\n";
>+    binmode F;
>+    undef $/;
>+    $mail{body} = encode_base64(<F>);
>+    close F;
>+
>+    # Need just filename for email filename
>+    my $email_file = basename( $tmp_file, [".gz"] );
>+
>+    $boundary = '--' . $boundary;
>+    $mail{body} = <<END_OF_BODY;
>+$boundary
>+Content-Type: text/plain; charset="iso-8859-1"
>+Content-Transfer-Encoding: quoted-printable
>+
>+$message
>+$boundary
>+Content-Type: application/octet-stream; name="$email_file"
>+Content-Transfer-Encoding: base64
>+Content-Disposition: attachment; filename="$email_file"
>+
>+$mail{body}
>+$boundary--
>+END_OF_BODY
>+
>+    # Send the email
>+    sendmail(%mail) || die "Error: $Mail::Sendmail::error\n";
>+

my $maintainer = Bugzilla->params->{maintainer};
my $message = <<END;
From: bugzilla
To: $maintainer
Subject: [$current_date] Bugzilla Log File

$mailbody
END

MessageToMTA($message);

>+    # Removed the compressed file
>+    unlink($tmp_file) || die "Error: could not remove $tmp_file: $!\n";
>+}
>+
>+1;
>+__END__
>+

Comment 8 Kevin Baker 2008-04-03 08:39:40 UTC
(In reply to comment #6)

> that we have access to. So I think we can omit that script now.

Confirmed. We don't need this. The logs are copied to 
engineering.redhat.com:/storage/bzlogs/bz-web[12]

Comment 9 Noura El hawary 2008-04-14 04:10:24 UTC
Created attachment 302304 [details]
v2 for cron script mail_log.pl

Thanks for the review Dave. I have fixed mail_log.pl as you suggested.

Comment 10 Noura El hawary 2008-04-14 04:11:31 UTC
Created attachment 302305 [details]
cron script sanitycheck.pl

This is second cron job sanitycheck.pl that depends on Bug #406261

Comment 11 David Lawrence 2008-04-14 04:24:27 UTC
Comment on attachment 302304 [details]
v2 for cron script mail_log.pl

Looks good. Check in.

Dave

Comment 12 David Lawrence 2008-04-14 04:27:04 UTC
Comment on attachment 302305 [details]
cron script sanitycheck.pl

This looks like the actual sanitycheck.pl script attached that is already being
reviewed on Bug #406261. Did you mean to attach this or the sanitycheck.cron
scrip instead?

Dave

Comment 13 Noura El hawary 2008-04-14 14:34:14 UTC
Created attachment 302344 [details]
cron script for sanitycheck.pl

Hi Dave, Yes you are right I meant the cron script ,, here we go attahed, are
we going to keep in in the same place as how it was in RHBZ2.18 under config/
dir ?

Thanks,
Noura

Comment 14 David Lawrence 2008-04-14 16:34:14 UTC
Comment on attachment 302344 [details]
cron script for sanitycheck.pl

Looks good Noura. Please check in.

Dave

Comment 15 Kevin Baker 2008-04-14 20:24:35 UTC
shouldn't the mail_log.pl script attached be obsoleted?


Note You need to log in before you can comment on or make changes to this bug.