Bug 688664

Summary:	Auditd disk_error_action tries to execute a program when the SYSLOG action was chosen
Product:	Red Hat Enterprise Linux 6	Reporter:	Eduard Benes <ebenes>
Component:	audit	Assignee:	Steve Grubb <sgrubb>
Status:	CLOSED ERRATA	QA Contact:	BaseOS QE Security Team <qe-baseos-security>
Severity:	medium	Docs Contact:
Priority:	high
Version:	6.1
Target Milestone:	rc
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	audit-2.1-1.el6	Doc Type:	Bug Fix
Doc Text:	A value of 'syslog' for the 'disk_error_action' parameter in 'auditd.conf' instructs auditd to issue a warning to syslog if an error is encountered when writing audit events to disk. If 'disk_error_action' was set to 'syslog', auditd always attempted to exec() a child process. Consequently, if a disk error was encountered (ie. a disk full error), auditd would attempt to exec() a null child process, and logging would not resume after the disk error was reported to syslog. In this update the child process is not called when the 'syslog' option is used, and logging continues as expected.	Story Points:	---
Clone Of:		Environment:
Last Closed:	2011-05-19 13:55:40 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	584498, 682670, 846801, 846802

Description Eduard Benes 2011-03-17 16:33:45 UTC

Description of problem:
Logging does not resume to audit.log after resume from SYSLOG on full disk action and user type messages continue to go to /var/log/messages. It should be noted that a record about successfull resume goes to audit.log eventually:

type=DAEMON_RESUME msg=audit(1300385209.456:8846): auditd resuming logging, sending auid=? pid=? subj=? res=success


Version-Release number of selected component (if applicable):
audit-2.0.6-1.el6.x86_64

How reproducible:
always


Steps to Reproduce:

1. Update a default /etc/aduitd/auditd.conf to use SYSLOG actions
-admin_space_left_action = SUSPEND
-disk_full_action = SUSPEND
-disk_error_action = SUSPEND
+admin_space_left_action = syslog 
+disk_full_action = syslog
+disk_error_action = syslog

2. Simulate a full disk where audit.log is located (more details later), and wait for a message in /var/log/messages:
<snip>
ar 17 13:06:25 dhcp-24-228 auditd[5523]: Audit daemon has no space left on logging partition
Mar 17 13:06:25 dhcp-24-228 auditd[5523]: Audit daemon has no space left on logging partition
Mar 17 13:06:25 dhcp-24-228 auditd[5523]: Record was not written to disk (No space left on device)
Mar 17 13:06:25 dhcp-24-228 auditd[5523]: write: Audit daemon detected an error writing an event to disk (No space left on device)
Mar 17 13:06:25 dhcp-24-228 auditd[5562]: Audit daemon failed to exec (null)
Mar 17 13:06:25 dhcp-24-228 auditd[5562]: The audit daemon is exiting.
Mar 17 13:06:25 dhcp-24-228 kernel: type=1305 audit(1300385185.779:403): audit_pid=0 old=5523 auid=501 ses=4 subj=unconfined_u:system_r:auditd_t:s0 res=1
Mar 17 13:06:25 dhcp-24-228 auditd[5523]: Record was not written to disk (No space left on device)
Mar 17 13:06:25 dhcp-24-228 auditd[5523]: write: Audit daemon detected an error writing an event to disk (No space left on device)
Mar 17 13:06:25 dhcp-24-228 auditd[5563]: Audit daemon failed to exec (null)
Mar 17 13:06:25 dhcp-24-228 auditd[5563]: The audit daemon is exiting.
Mar 17 13:06:25 dhcp-24-228 kernel: type=1305 audit(1300385185.846:404): audit_pid=0 old=0 auid=501 ses=4 subj=unconfined_u:system_r:auditd_t:s0 res=1
</snip> 
User messages (e.g. send by auditctl -m hello) now go to /v/l/messages

3. Make again some space on the logging disk (e.g. by removing the trash you've put the in prvious step), and trigger to resume the logging:
# service auditd resume

4. Send an user space message with 
# auditcl -m hello_world

Actual results:
Messages continue to end in /v/l/messages instead of audit.log after the resume.
Mar 17 13:07:02 dhcp-24-228 kernel: type=1005 audit(1300385222.277:411): user pid=5597 uid=0 auid=501 ses=4 subj=unconfined_u:unconfined_r:auditctl_t:s0-s0:c0.c1023 msg='hello_world exe="/sbin/auditctl" hostname=? addr=? terminal=pts/0 res=success'

Expected results:
Messages end in  audit.log after the resume.
type=USER msg=audit(1300385164.838:392): user pid=5533 uid=0 auid=501 ses=4 subj=unconfined_u:unconfined_r:auditctl_t:s0-s0:c0.c1023 msg='hello_world : exe="/sbin/auditctl" hostname=? addr=? terminal=pts/0 res=success'

Additional info:
This seems to happen only with SYSLOG action. Action SUSPEND works as expected and the logging is resumted to audit.log correctly.

Comment 1 Eduard Benes 2011-03-18 08:12:18 UTC

Client side message in /var/log/messages for syslog action when server signalling low disk space could be improved. Currently it logs something like this:
<snip>
  Mar 17 09:06:47 x86-64-v07 audisp-remote: remote disk low,
</snip>

Comment 2 Steve Grubb 2011-03-29 14:07:19 UTC

After reviewing the code, I have a couple things to report. The syslog option does not suspend logging. Choosing it means that you get a syslog message each failure. That said, the above log shows a real bug. It says that it cannot exec (null). That means that it was calling the safe_exec function which forks and tries to execute a child process and fails because the program is NULL. This seems to be what is messing things up.

Comment 3 Eduard Benes 2011-03-29 14:37:38 UTC

(In reply to comment #2)
> After reviewing the code, I have a couple things to report. The syslog option
> does not suspend logging. Choosing it means that you get a syslog message each
> failure. 

Yes, I should have used word "return" instead of resume in the bug report. 

Though there might be a possible issue with action SUSPEND in audisp-remote plugin is that it does not resume on SIGUSR2 e.g "service auditd resume", unless there is a different procedure for resuming the plugin. Could you please have a look into that? (I haven't tried to send it directly to the plugin yet, e.g kill -S usr2 audisp-remote).

> That said, the above log shows a real bug. It says that it cannot exec
> (null). That means that it was calling the safe_exec function which forks and
> tries to execute a child process and fails because the program is NULL. This
> seems to be what is messing things up.

Comment 4 Steve Grubb 2011-03-30 16:18:40 UTC

audit-2.1-1.el6 was build to resolve this problem.

Comment 6 Eduard Benes 2011-04-05 09:51:01 UTC

(In reply to comment #3)
> (In reply to comment #2)
> > After reviewing the code, I have a couple things to report. The syslog option
> > does not suspend logging. Choosing it means that you get a syslog message each
> > failure. 
> 
> Yes, I should have used word "return" instead of resume in the bug report. 
> 
> Though there might be a possible issue with action SUSPEND in audisp-remote
> plugin is that it does not resume on SIGUSR2 e.g "service auditd resume",
> unless there is a different procedure for resuming the plugin. Could you please
> have a look into that? (I haven't tried to send it directly to the plugin yet,
> e.g kill -S usr2 audisp-remote).

The audisp-remote plugin process will resume logging when it received a SIGUSR2 signal directly. The auditd/audispd does not send SIGUSR2 signals to running registered plugins.

Comment 11 errata-xmlrpc 2011-05-19 13:55:40 UTC

An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2011-0653.html