Red Hat Bugzilla – Bug 595782
RFE: rate limit the auditors
Last modified: 2011-06-27 12:38:33 EDT
Description of problem:
A runaway AVC auditor can overwhelm the audisp-remote queue.
Version-Release number of selected component (if applicable):
It would be desirable to have a rate-limit configurable parameter to stop a process from doing a audit event flood. I have found that in the field, when a process gets into a state not seen in testing, it can produce prodigious amounts of audit data in a short time (600MB in 3 minutes for example).
If we could rate-limit the event input to some maximum number/second it would mitigate this issue.
If this is possible, also having a custom capability to call out when this is detected would be great (like the space_left_action in the auditd.conf). Then a custom script could signal an administrator that this instance had happened.
So far all the extra audits have been AVCs if that matters.
One question...where do all the events get held while waiting for transfer?
In the audisp queue? We changed the default size to 5120 from 512.
Or maybe I misunderstand the question.
We get the "queue is full - dropping event" message in the system message log.
That would mean that it will have to have per-plugin bookkeeping so that it knows when it can safely retire a slot in the queue. For example, the selinux troubleshooter may not be overwhelmed and able to take events. And then what if you throw in network problems? At some point audispd's queue could get overwhelmed.
The "queue is full - dropping event" is actually an indication that audispd's queue is full. The audit daemon itself does not queue any events. This is because it has to be deterministic about how many events could be lost if it segfaults. If it has no queue, then only 1 event can be lost (the current one). It depends on the kernel's event queue - which has only 1 protection mechanism - panic the machine.
What you really might want is the log and forward model where the remote logger puts the event to disk and then tries to communicate with the aggregating server or you want some kind of reactive plugin that terminates apps/sessions that exceed some admin provided threshold.
I think we are doing the log and forward model. The local machine writes to disk (auditd)and tries to send to the aggregator (auditd->audispd->audisp-remote ... right?).
You are right - the reactive plugin idea is really what I'm after with this. Even on the local machine where the event happens, I'd rather not allow unlimited audit event input. Something operating in this mode is really outside the desired event input anyway.
So then - if the process is terminated, ideally I'd like a script to be run which would allow me to communicate with a human (email, custom app alert, etc) that there was a problem. Maybe even a special audit event that would generate an IDS event similar to the ones already done with the special ids-file-hi.
Changing version to 13. Fedora 10 is no longer supported.
This message is a reminder that Fedora 13 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 13. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as WONTFIX if it remains open with a Fedora
'version' of '13'.
Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version'
to a later Fedora version prior to Fedora 13's end of life.
Bug Reporter: Thank you for reporting this issue and we are sorry that
we may not be able to fix it before Fedora 13 is end of life. If you
would still like to see this bug fixed and are able to reproduce it
against a later version of Fedora please change the 'version' of this
bug to the applicable version. If you are unable to change the version,
please add a comment here and someone will do it for you.
Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.
The process we are following is described here:
Fedora 13 changed to end-of-life (EOL) status on 2011-06-25. Fedora 13 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.
If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version.
Thank you for reporting this bug and we are sorry it could not be fixed.