Bug 125726 - System freezes during append mode audit with 2.1GB audit log
Summary: System freezes during append mode audit with 2.1GB audit log
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise Linux 3
Classification: Red Hat
Component: laus   
(Show other bugs)
Version: 3.0
Hardware: i686
OS: Linux
medium
high
Target Milestone: ---
Assignee: Jason Vas Dias
QA Contact: Jay Turner
URL:
Whiteboard:
Keywords:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2004-06-10 17:05 UTC by Peggy Proffitt
Modified: 2015-01-08 00:08 UTC (History)
2 users (show)

Fixed In Version: laus-0.1-70RHEL3
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2005-06-02 01:02:08 UTC
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Tar of audit configuration files (10.95 KB, application/x-tar)
2004-06-10 17:07 UTC, Peggy Proffitt
no flags Details

Description Peggy Proffitt 2004-06-10 17:05:50 UTC
Description of problem:

The system freezes when the audit file /var/log/audit reaches the 
size of ~2.1GB. We are running audit in append mode because we want 
to process and archive our audit data on a daily basis. The size of 
the active audit file had reached 2,147,483,647 bytes when the 
problem occurred.

We booted into rescue mode to disable audit and move the active audit 
file to a new file name. We then rebooted and started audit 
successfully. Next, we stopped audit, moved the large file back 
to /var/log/audit, and restarted audit. Attempts to start new 
sessions on the server(e.g. su, ssh, telnet, etc) all hung. We then 
stopped audit. The original hanging session attempts remained hung. 
Additional attempts to start sessions also hung. We had to reboot the 
system to clear the problem.

We are currently working with audit on a test system, but are 
scheduled to place it on production systems within the next couple of 
weeks. Is the size of the audit file a true limitation? If so, how 
can we work around it without losing audit data?



Version-Release number of selected component (if applicable):
laus-0.1-58RHEL3, laus-libs-0.1-59RHEL3

How reproducible:
Always

Steps to Reproduce:
1.Configure system to audit in append mode at boot time.
2.Reboot system.
3.Run jobs that generate enough audit data that the /var/log/audit 
file reaches a size of over 2.1GB.
  
Actual results:
When the /var/log/audit file reached a size of 2,147,483,647 bytes 
the system hangs.

Expected results:
The system should continue to process jobs.

Additional info:
The audit daemon should be coded to move the existing file to a 
unique name and create a new file if the active audit file approaches 
a size limitation. It appears that the bin mode audit does allow 
this, but bin mode makes it difficult to process a single day's worth 
of audit at a time.

Comment 1 Peggy Proffitt 2004-06-10 17:07:08 UTC
Created attachment 101041 [details]
Tar of audit configuration files

Comment 2 Jason Vas Dias 2005-06-02 01:02:08 UTC
Very sorry for the delay in getting back to you about this bug -
I've recently taken over the LAuS package and this bug had been
overlooked .
The size limitation could have been a filesystem size limitation,
which is no longer the case with later RHEL-3 kernels .
I'd suggest not using append mode, but using a normal set of binary
audit logs, such as:
output {
        mode            = bin;
        num-files       = 4;
        file-size       = 20M;
        file-name       = "/var/log/audit.d/bin";
        notify          = "/etc/audit/process_log.sh";
        ...
        }
You could create your /etc/audit/process_log.sh as a script which
performs the analysis / archival functions on the single binary
audit log given as its first argument "$1", and then invokes 
'/usr/sbin/audbin -C -S /var/log/audit.d/save.%u $1' to clear it out.
There is also new audbin -T threshold and -N notify parameters 
to handle the case when there is not enough space to hold the
saved audit log - see man audbin(1) .
Please try the latest laus-0.1-70RHEL3 in RHEL-3-U5 and available
for download from: http://people.redhat.com/~jvdias/laus/
I don't think this problem will reoccur with that version and the
latest RHEL-3 kernel.


Note You need to log in before you can comment on or make changes to this bug.