Bug 595782 - RFE: rate limit the auditors
Summary: RFE: rate limit the auditors
Alias: None
Product: Fedora
Classification: Fedora
Component: audit
Version: 13
Hardware: All
OS: Linux
Target Milestone: ---
Assignee: Steve Grubb
QA Contact: Fedora Extras Quality Assurance
Depends On:
TreeView+ depends on / blocked
Reported: 2010-05-25 15:30 UTC by LC Bruzenak
Modified: 2011-06-27 16:38 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Last Closed: 2011-06-27 16:38:33 UTC
Type: ---

Attachments (Terms of Use)

Description LC Bruzenak 2010-05-25 15:30:05 UTC
Description of problem:
A runaway AVC auditor can overwhelm the audisp-remote queue.

Version-Release number of selected component (if applicable):

It would be desirable to have a rate-limit configurable parameter to stop a process from doing a audit event flood. I have found that in the field, when a process gets into a state not seen in testing, it can produce prodigious amounts of audit data in a short time (600MB in 3 minutes for example).
If we could rate-limit the event input to some maximum number/second it would mitigate this issue.
If this is possible, also having a custom capability to call out when this is detected would be great (like the space_left_action in the auditd.conf). Then a custom script could signal an administrator that this instance had happened.

Comment 1 LC Bruzenak 2010-05-25 15:31:41 UTC
So far all the extra audits have been AVCs if that matters.

Comment 2 Steve Grubb 2010-05-25 15:50:41 UTC
One question...where do all the events get held while waiting for transfer?

Comment 3 LC Bruzenak 2010-05-25 16:13:16 UTC
In the audisp queue? We changed the default size to 5120 from 512.
Or maybe I misunderstand the question.
We get the "queue is full - dropping event" message in the system message log.

Comment 4 Steve Grubb 2010-05-25 17:20:52 UTC
That would mean that it will have to have per-plugin bookkeeping so that it knows when it can safely retire a slot in the queue. For example, the selinux troubleshooter may not be overwhelmed and able to take events. And then what if you throw in network problems? At some point audispd's queue could get overwhelmed.

The "queue is full - dropping event" is actually an indication that audispd's queue is full. The audit daemon itself does not queue any events. This is because it has to be deterministic about how many events could be lost if it segfaults. If it has no queue, then only 1 event can be lost (the current one). It depends on the kernel's event queue - which has only 1 protection mechanism - panic the machine.

What you really might want is the log and forward model where the remote logger puts the event to disk and then tries to communicate with the aggregating server or you want some kind of reactive plugin that terminates apps/sessions that exceed some admin provided threshold.

Comment 5 LC Bruzenak 2010-05-25 17:51:04 UTC
I think we are doing the log and forward model. The local machine writes to disk (auditd)and tries to send to the aggregator (auditd->audispd->audisp-remote ... right?). 
You are right - the reactive plugin idea is really what I'm after with this. Even on the local machine where the event happens, I'd rather not allow unlimited audit event input. Something operating in this mode is really outside the desired event input anyway.
So then - if the process is terminated, ideally I'd like a script to be run which would allow me to communicate with a human (email, custom app alert, etc) that there was a problem. Maybe even a special audit event that would generate an IDS event similar to the ones already done with the special ids-file-hi.

Comment 6 John Poelstra 2010-06-11 16:59:14 UTC
Changing version to 13.  Fedora 10 is no longer supported.

Comment 7 Bug Zapper 2011-06-02 13:32:24 UTC
This message is a reminder that Fedora 13 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 13.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '13'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 13's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 13 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 

Comment 8 Bug Zapper 2011-06-27 16:38:33 UTC
Fedora 13 changed to end-of-life (EOL) status on 2011-06-25. Fedora 13 is 
no longer maintained, which means that it will not receive any further 
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of 
Fedora please feel free to reopen this bug against that version.

Thank you for reporting this bug and we are sorry it could not be fixed.

Note You need to log in before you can comment on or make changes to this bug.