Bug 476380

Do you have anything written to syslog that would indicate why the audit daemon couldn't finish log rotation? The first thing it does is set the perms of audit.log, then it works from file with highest number to least, then finally opens a new file. It looks like it did the delete to make room for all the files, but wasn't able to rename .3 to .4.

This is in the logs:

Dec  4 13:35:40 free-spin auditd[1877]: Audit daemon rotating log files
Dec  4 13:35:40 free-spin auditd[1877]: Error rotating logs from /var/log/audit/audit.log.2 to /var/log/audit/audit.log.3 (Permission denied)
Dec  4 13:35:40 free-spin auditd[1877]: rotate: Audit daemon detected an error writing an event to disk (Permission denied)
Dec  4 13:35:40 free-spin auditd[1877]: Error rotating logs from /var/log/audit/audit.log.1 to /var/log/audit/audit.log.2 (Permission denied)
Dec  4 13:35:40 free-spin auditd[1877]: rotate: Audit daemon detected an error writing an event to disk (Permission denied)
Dec  4 13:35:40 free-spin auditd[17785]: Audit daemon failed to exec (null)
Dec  4 13:35:40 free-spin auditd[17785]: The audit daemon is exiting.
Dec  4 13:35:40 free-spin auditd[1877]: Error rotating logs from /var/log/audit/audit.log to /var/log/audit/audit.log.1 (Permission denied)
Dec  4 13:35:40 free-spin auditd[1877]: rotate2: Audit daemon detected an error writing an event to disk (Permission denied)
Dec  4 13:35:40 free-spin auditd[17786]: Audit daemon failed to exec (null)
Dec  4 13:35:40 free-spin auditd[17786]: The audit daemon is exiting.
Dec  4 13:35:40 free-spin auditd[1877]: Couldn't open log file /var/log/audit/audit.log (Permission denied)
Dec  4 13:35:40 free-spin auditd[1877]: Could not reopen a log after rotating.
Dec  4 13:35:40 free-spin auditd[1877]: reopen: Audit daemon detected an error writing an event to disk (Permission denied)
Dec  4 13:35:40 free-spin auditd[17787]: Audit daemon failed to exec (null)
Dec  4 13:35:40 free-spin auditd[17787]: The audit daemon is exiting.
Dec  4 13:35:40 free-spin auditd[17788]: Audit daemon failed to exec (null)
Dec  4 13:35:40 free-spin auditd[17788]: The audit daemon is exiting.
Dec  4 13:35:40 free-spin auditd[17785]: Error receiving audit netlink packet (No buffer space available)
Dec  4 13:35:40 free-spin auditd[17785]: Error setting audit daemon pid (No buffer space available)

But this is also in the middle of a yum update.  I think this may have been the selinux update that caused widespread and gratuitous denials.  I've since set it to "permissive".  Should audit have be able to avoid or recover from this state automatically?  Should I do "chmod -R u+w /var/log/audit" now?

Thanks for the log info. SE Linux could have done it. Auditd uses the rename syscall and about the only other failure mode is when something grabs up all the diskspace or something has the target file opened such that it cannot be overwritten.

As for whether auditd should recover...this is the only reported failure of rotation in the 3 years that code existed. So, I'll revisit the rotation code and see if we can reset the permissions and pretend it didn't happen so that next attempt to write we try to rotate again with hopefully better results.

I committed a change into upstream source code that chmod's the original log back to its permissions if it fails to rotate it.

https://fedorahosted.org/audit/changeset/215

The fix will be in the next major release.

audit-1.7.11-1.fc10 has been submitted as an update for Fedora 10.
http://admin.fedoraproject.org/updates/audit-1.7.11-1.fc10

Closing since the 1.7.11 package is now in F-10 stable. Thanks for reporting the problem.

I have audit-1.7.11-2.fc11.x86_64 installed now, and I am still getting auditd startup failure with the same error message, and ls -l /var/log/audit/ says:

-r-------- 1 root root 5243278 2008-12-04 13:35 audit.log
-r-------- 1 root root 5243057 2008-11-10 04:02 audit.log.1
-r-------- 1 root root 5243105 2008-08-13 04:57 audit.log.2
-r-------- 1 root root 5243186 2008-08-13 04:56 audit.log.3

The fix that was applied was to only fix a problem on log rotation so that you do not have this problem in the future. I'm not entirely sure that the audit daemon should open the permissions for writing on any file that its not continuously controlling. If the daemon has been stopped, the admin or a custom rotation script could have intentionally set perms for readonly. (Imagine someone using logrotate and it blowing up midflight.) They should fix that manually as well. But during rotation, they should not have any expectation of the files being released without looking at syslog. So, auditd now corrects the permissions if the log rotation its performing runs into any problems.

OK, I've fixed those permissions manually and auditd seems happy again.  Thanks for the preventive fix!