Bug 788760

Summary: [RFE] Logconv improvements
Product: Red Hat Enterprise Linux 6 Reporter: Rich Megginson <rmeggins>
Component: 389-ds-baseAssignee: Rich Megginson <rmeggins>
Status: CLOSED ERRATA QA Contact: IDM QE LIST <seceng-idm-qe-list>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 6.3CC: amsharma, jgalipea, mreynolds
Target Milestone: rcKeywords: FutureFeature
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: 389-ds-base-1.2.10.0-1.el6 Doc Type: Enhancement
Doc Text:
Cause: need for detailed interval reporting Consequence: unable to extract logging info in a useful report Change: added new options for minute/second reporting. Result: the option is now available for interval reporting
 Story Points: ---
Clone Of: Environment:
Last Closed: 2012-06-20 07:14:36 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
access log to use with logconv.pl none

Description Rich Megginson 2012-02-08 23:20:59 UTC 
This bug is created as a clone of upstream ticket:
https://fedorahosted.org/389/ticket/175

https://bugzilla.redhat.com/show_bug.cgi?id=761614

{{{
Description of problem:

logconv.pl currently only produces a summary of operations for a file or for a
requested period.

It would help to spot peaks if some sort of running per-period values could be
generated.


Version-Release number of selected component (if applicable):

redhat-ds-base-8.2.6-1.el5dsrv


Attached is a patch against logconv.pl to add optional generation of per-second
and per-minute statistics in CSV format, allowing for further post-processing.

Adds the following command line options

  -m <per second stats file>
  -M <per minute stats file>

One small side-effect should be an improvement in speed since the
time-conversion calls have been optimised to accommodate the stats changes.
}}}
Comment 2 mreynolds 2012-05-11 20:16:32 UTC 
Created attachment 583918 [details]
access log to use with logconv.pl
Comment 3 mreynolds 2012-05-11 20:17:52 UTC 
Here is the output from running:

logconv.pl -m /tmp/out /tmp/access

/tmp/out:

Time,time_t,Results,Search,Add,Mod,Modrdn,Delete,Abandon,Connections,SSL Conns,B
ind,Anon Bind,Unbind,Unindexed
07/May/2012:18:32:46 -0400,1336350766,2,1,0,0,0,0,0,0,0,0,0,1,1,0,1
07/May/2012:18:32:54 -0400,1336350774,2,1,0,0,0,0,0,0,0,0,0,1,1,0,1
07/May/2012:18:32:46 -0400,1336350766,2,1,0,0,0,0,0,0,0,0,0,1,0,0,1
07/May/2012:18:32:54 -0400,1336350774,2,1,0,0,0,0,0,0,0,0,0,1,0,0,1
07/May/2012:18:35:54 -0400,1336350954,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
Comment 4 Amita Sharma 2012-05-14 09:10:01 UTC 
[root@dhcp201-194 /]#  /usr/bin/logconv.pl -M log-minute-stats-csv.out /var/log/dirsrv/slapd-dhcp201-194/access

Access Log Analyzer 6.1

Command : logconv.pl -M log-minute-stats-csv.out /var/log/dirsrv/slapd-dhcp201-194/access

Processing 1 Access Log(s)...

Filename			   Total Lines
--------------------------------------------------
[01] /var/log/dirsrv/slapd-dhcp201-194/access      31
         31 Lines Processed



Total Log Lines Analysed:  31


----------- Access Log Output ------------

Start of Log:    14/May/2012:12:32:53
End of Log:      14/May/2012:12:35:15

Processed Log Time:   Hours, 2 Minutes, 22 Seconds

Restarts:                     1
Total Connections:            0
SSL Connections:              0
Peak Concurrent Connections:  1
Total Operations:             8
Total Results:                8
Overall Performance:          100.0%

Searches:                     4          (0.03/sec)  (1.69/min)
Modifications:                0          (0.00/sec)  (0.00/min)
Adds:                         0          (0.00/sec)  (0.00/min)
Deletes:                      0          (0.00/sec)  (0.00/min)
Mod RDNs:                     0          (0.00/sec)  (0.00/min)
Mod DNs:                      0          (0.00/sec)  (0.00/min)
Compares:                     0          (0.00/sec)  (0.00/min)
Binds:                        4          (0.03/sec)  (1.69/min)

Proxied Auth Operations:      0
Persistent Searches:          0
Internal Operations:          0
Entry Operations:             0
Extended Operations:          0
Abandoned Requests:           0
Smart Referrals Received:     0

VLV Operations:               0
VLV Unindexed Searches:       0
SORT Operations:              0

Entire Search Base Queries:   0
Unindexed Searches:           0

FDs Taken:                    4
FDs Returned:                 4
Highest FD Taken:             64

Broken Pipes:                 0
Connections Reset By Peer:    0
Resource Unavailable:         0

Binds:                        4
Unbinds:                      4

 LDAP v2 Binds:               0
 LDAP v3 Binds:               4
 SSL Client Binds:            0
 Failed SSL Client Binds:     0
 SASL Binds:                  0

 Directory Manager Binds:     0
 Anonymous Binds:             4
 Other Binds:                 0

[root@dhcp201-194 /]# cat log-minute-stats-csv.out
Time,time_t,Results,Search,Add,Mod,Modrdn,Delete,Abandon,Connections,SSL Conns,Bind,Anon Bind,Unbind,Unindexed
14/May/2012:12:32:00 +051800,1336890720,4,2,0,0,0,0,0,0,0,0,0,2,2,0,0
14/May/2012:12:34:00 +051800,1336890840,2,1,0,0,0,0,0,0,0,0,0,1,1,0,0
14/May/2012:12:35:00 +051800,1336890900,2,1,0,0,0,0,0,0,0,0,0,1,1,0,0
Comment 5 mreynolds 2012-05-25 15:15:06 UTC 
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
Cause: need for detailed interval reporting
Consequence: unable to extract logging info in a useful report
Change: added new options for minute/second reporting.
Result: the option is now available for interval reporting
Comment 6 errata-xmlrpc 2012-06-20 07:14:36 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2012-0813.html