Bug 406201

Summary: 3.10: Commandline utilities and cron jobs
Product: [Community] Bugzilla Reporter: David Lawrence <dkl>
Component: Bugzilla GeneralAssignee: Noura El hawary <nelhawar>
Status: CLOSED NEXTRELEASE QA Contact:
Severity: medium Docs Contact:
Priority: high    
Version: 3.2   
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2008-05-22 06:36:28 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 406261, 406301    
Bug Blocks: 406071, 427059    
Attachments:
Description Flags
command line utility mail_log.pl to mail log4perl logging
dkl: review-
v2 for cron script mail_log.pl
dkl: review+
cron script sanitycheck.pl
none
cron script for sanitycheck.pl dkl: review+

Description David Lawrence 2007-11-30 16:58:12 UTC
Description:
Red Hat has implemented several command line Perl scripts for maintenance and debugging purposes. sanitycheck.pl emails a log nightly of possible data issues int he Bugzilla database. mail_log.pl emails nightly the bz.log file that contains entries helpful for performance debugging and benchmarking. bz-expire-cookies.pl cleans out the logincookies table in the Bugzilla database of stale entries.

Function Requirements:
mail_log.pl, sanitycheck.pl, and bz-expire-cookies.pl

Comment 1 Noura El hawary 2007-12-20 07:17:17 UTC
mail_log.pl ,, basically with this script not much change is required as it
doesn't interact with bugzilla modules much ,, mostly with external perl-modules
in its interaction with bugzilla ,, we will need to change the code that calls
Bugzilla::Config to use the Params() function as in bugzilla 3 it will use :
Bugzilla->params->{'maintainer'}; instead so this is bout 3 LOC ,, also it needs
to interact with Bugzilla.pm to get the log4perl config file details ,, and
basically this can not happen until we have done the log4perl porting to 3.2 so
will make this bug depending on the (406301 bugs for log4perl)


sanitycheck.pl : also this script will not require much change ,, only the
following:
1- replace Bugzilla::Config to use the Params() with
Bugzilla->params->{'maintainer'} as this is how it is done in  bz3.2 = 3 LOC
2- add check option for extra check found in 3.2 for flag consistency in
products and components = 5 LOC
3- this script needs to interact with Bugzilla/SanityCheck.pm so it is dependent
on the bug that port this module from 2.18 to 3.2 to replace sanitycheck.cgi
so will make also this bug dependent on bug 406261


bz-expire-cookies.pl ,, this script will need to be changed as it calls
Bugzilla::Auth::Cookie->clean_expired_tokens(); in bz 2_18 ,, 
However the way that cookies are handled in 3.2 is different and the module
structure is completely different. I don't think we currently have a task in the
upgrade for the cookies ,, maybe we need to create one,, but for that script the
code would only need to call a subroutine to clean the expired cookies so
expecting about 10 LOC


So basically TOTAL LOC = 25 LOC

Comment 2 Kevin Baker 2007-12-29 05:58:07 UTC
Noura,

please create a task for resolving the cookie code. 

Comment 3 Kevin Baker 2007-12-29 18:28:36 UTC
sorry, I sopke to soon. The cookie code is handled under bug 406231. I added 
it as a dependency

Comment 4 Noura El hawary 2008-04-02 04:32:45 UTC
Created attachment 300010 [details]
command line utility mail_log.pl to mail log4perl logging

First section of this bug is the commanline utility mail_log.pl to compress and
mail the generated log file from log4perl. as we are done with log4perl so I
thought i would get this done as well. Basically is it exactly similar to
rh_bugzilla_2_18 only one difference was replacing
Param("maintainer") with Bugzilla->params->{maintainer} 

I tested it on bugdev and it works good.

Noura

Comment 5 Noura El hawary 2008-04-02 04:34:15 UTC
worked 30 minutes. 

also one more thing where would be the best place to keep those cron utilities
will we still keep them under the main bugzilla directory ?

Noura

Comment 6 David Lawrence 2008-04-02 14:16:12 UTC
Comment on attachment 300010 [details]
command line utility mail_log.pl to mail log4perl logging

mail_log.pl is no longer necessary as I understand it since
eng-sysadmins now copy the log files nightly to a separate storage area
that we have access to. So I think we can omit that script now.

Dave

Comment 7 David Lawrence 2008-04-02 14:22:12 UTC
Comment on attachment 300010 [details]
command line utility mail_log.pl to mail log4perl logging

>Index: mail_log.pl
>===================================================================
>+
>+use MIME::QuotedPrint;
>+use MIME::Base64;
>+use Mail::Sendmail 0.75;    # doesn't work with v. 0.74!

use Bugzilla::Mailer::MessageToMTA here instead for 3.2

>+use File::Basename;
>+use Bugzilla::Config; # for Param method

This is no longer needed since you can just use Bugzilla->params->{param}.

>+use Log::Log4perl;
>+use Bugzilla;
>+
>+main(@ARGV) if not caller();
>+
>+sub main {
>+    my ( $sec, $min, $hour, $mday, $mon, $year, $wday, $yday, $isdst )
>+        = localtime(time);
>+    my $current_date = sprintf(
>+        "%.4d/%.2d/%.2d %.2d:%.2d:%.2d",
>+        $year + 1900,
>+        $mon + 1, $mday, $hour, $min, $sec
>+    );
>+
>+    my %mail = (
>+        from    => 'bugzilla',
>+        to      => Bugzilla->params->{maintainer},
>+        subject => "[$current_date] Bugzilla Log File",
>+        smtp    => 'localhost',
>+    );
>+
>+    my $boundary = "====" . time() . "====";
>+    $mail{'content-type'} = "multipart/mixed; boundary=\"$boundary\"";
>+
>+    my $message = encode_qp("Bugzilla Log File");
>+
>+    # Get the log file name from Log4perl config file
>+    my $log_file = Bugzilla->log_filename;
>+
>+    # Exit if file does not exist
>+    return if !-e $log_file;
>+
>+    # Gzip the current log file and rename to logfile.gz
>+    my $tmp_file = $log_file . ".gz";
>+    system("/bin/gzip -f $log_file") == 0
>+        or die "Error executing gzip -f $log_file";
>+
>+    open( F, $tmp_file ) or die "Cannot read $tmp_file: $!\n";
>+    binmode F;
>+    undef $/;
>+    $mail{body} = encode_base64(<F>);
>+    close F;
>+
>+    # Need just filename for email filename
>+    my $email_file = basename( $tmp_file, [".gz"] );
>+
>+    $boundary = '--' . $boundary;
>+    $mail{body} = <<END_OF_BODY;
>+$boundary
>+Content-Type: text/plain; charset="iso-8859-1"
>+Content-Transfer-Encoding: quoted-printable
>+
>+$message
>+$boundary
>+Content-Type: application/octet-stream; name="$email_file"
>+Content-Transfer-Encoding: base64
>+Content-Disposition: attachment; filename="$email_file"
>+
>+$mail{body}
>+$boundary--
>+END_OF_BODY
>+
>+    # Send the email
>+    sendmail(%mail) || die "Error: $Mail::Sendmail::error\n";
>+

my $maintainer = Bugzilla->params->{maintainer};
my $message = <<END;
From: bugzilla
To: $maintainer
Subject: [$current_date] Bugzilla Log File

$mailbody
END

MessageToMTA($message);

>+    # Removed the compressed file
>+    unlink($tmp_file) || die "Error: could not remove $tmp_file: $!\n";
>+}
>+
>+1;
>+__END__
>+

Comment 8 Kevin Baker 2008-04-03 08:39:40 UTC
(In reply to comment #6)

> that we have access to. So I think we can omit that script now.

Confirmed. We don't need this. The logs are copied to 
engineering.redhat.com:/storage/bzlogs/bz-web[12]

Comment 9 Noura El hawary 2008-04-14 04:10:24 UTC
Created attachment 302304 [details]
v2 for cron script mail_log.pl

Thanks for the review Dave. I have fixed mail_log.pl as you suggested.

Comment 10 Noura El hawary 2008-04-14 04:11:31 UTC
Created attachment 302305 [details]
cron script sanitycheck.pl

This is second cron job sanitycheck.pl that depends on Bug #406261

Comment 11 David Lawrence 2008-04-14 04:24:27 UTC
Comment on attachment 302304 [details]
v2 for cron script mail_log.pl

Looks good. Check in.

Dave

Comment 12 David Lawrence 2008-04-14 04:27:04 UTC
Comment on attachment 302305 [details]
cron script sanitycheck.pl

This looks like the actual sanitycheck.pl script attached that is already being
reviewed on Bug #406261. Did you mean to attach this or the sanitycheck.cron
scrip instead?

Dave

Comment 13 Noura El hawary 2008-04-14 14:34:14 UTC
Created attachment 302344 [details]
cron script for sanitycheck.pl

Hi Dave, Yes you are right I meant the cron script ,, here we go attahed, are
we going to keep in in the same place as how it was in RHBZ2.18 under config/
dir ?

Thanks,
Noura

Comment 14 David Lawrence 2008-04-14 16:34:14 UTC
Comment on attachment 302344 [details]
cron script for sanitycheck.pl

Looks good Noura. Please check in.

Dave

Comment 15 Kevin Baker 2008-04-14 20:24:35 UTC
shouldn't the mail_log.pl script attached be obsoleted?