Bug 125726

Summary: System freezes during append mode audit with 2.1GB audit log
Product: Red Hat Enterprise Linux 3 Reporter: Peggy Proffitt <peggy.proffitt>
Component: lausAssignee: Jason Vas Dias <jvdias>
Status: CLOSED CURRENTRELEASE QA Contact: Jay Turner <jturner>
Severity: high Docs Contact:
Priority: medium    
Version: 3.0CC: srevivo, tao
Target Milestone: ---   
Target Release: ---   
Hardware: i686   
OS: Linux   
Whiteboard:
Fixed In Version: laus-0.1-70RHEL3 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2005-06-02 01:02:08 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Tar of audit configuration files none

Description Peggy Proffitt 2004-06-10 17:05:50 UTC 
Description of problem:

The system freezes when the audit file /var/log/audit reaches the 
size of ~2.1GB. We are running audit in append mode because we want 
to process and archive our audit data on a daily basis. The size of 
the active audit file had reached 2,147,483,647 bytes when the 
problem occurred.

We booted into rescue mode to disable audit and move the active audit 
file to a new file name. We then rebooted and started audit 
successfully. Next, we stopped audit, moved the large file back 
to /var/log/audit, and restarted audit. Attempts to start new 
sessions on the server(e.g. su, ssh, telnet, etc) all hung. We then 
stopped audit. The original hanging session attempts remained hung. 
Additional attempts to start sessions also hung. We had to reboot the 
system to clear the problem.

We are currently working with audit on a test system, but are 
scheduled to place it on production systems within the next couple of 
weeks. Is the size of the audit file a true limitation? If so, how 
can we work around it without losing audit data?



Version-Release number of selected component (if applicable):
laus-0.1-58RHEL3, laus-libs-0.1-59RHEL3

How reproducible:
Always

Steps to Reproduce:
1.Configure system to audit in append mode at boot time.
2.Reboot system.
3.Run jobs that generate enough audit data that the /var/log/audit 
file reaches a size of over 2.1GB.
  
Actual results:
When the /var/log/audit file reached a size of 2,147,483,647 bytes 
the system hangs.

Expected results:
The system should continue to process jobs.

Additional info:
The audit daemon should be coded to move the existing file to a 
unique name and create a new file if the active audit file approaches 
a size limitation. It appears that the bin mode audit does allow 
this, but bin mode makes it difficult to process a single day's worth 
of audit at a time.
Comment 1 Peggy Proffitt 2004-06-10 17:07:08 UTC 
Created attachment 101041 [details]
Tar of audit configuration files
Comment 2 Jason Vas Dias 2005-06-02 01:02:08 UTC 
Very sorry for the delay in getting back to you about this bug -
I've recently taken over the LAuS package and this bug had been
overlooked .
The size limitation could have been a filesystem size limitation,
which is no longer the case with later RHEL-3 kernels .
I'd suggest not using append mode, but using a normal set of binary
audit logs, such as:
output {
        mode            = bin;
        num-files       = 4;
        file-size       = 20M;
        file-name       = "/var/log/audit.d/bin";
        notify          = "/etc/audit/process_log.sh";
        ...
        }
You could create your /etc/audit/process_log.sh as a script which
performs the analysis / archival functions on the single binary
audit log given as its first argument "$1", and then invokes 
'/usr/sbin/audbin -C -S /var/log/audit.d/save.%u $1' to clear it out.
There is also new audbin -T threshold and -N notify parameters 
to handle the case when there is not enough space to hold the
saved audit log - see man audbin(1) .
Please try the latest laus-0.1-70RHEL3 in RHEL-3-U5 and available
for download from: http://people.redhat.com/~jvdias/laus/
I don't think this problem will reoccur with that version and the
latest RHEL-3 kernel.