Bug 2207869
| Summary: | When audit has "admin_space_left_action = halt", the system is halted with no visible message on the console | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 9 | Reporter: | Renaud Métrich <rmetrich> |
| Component: | audit | Assignee: | Sergio Correia <scorreia> |
| Status: | CLOSED MIGRATED | QA Contact: | BaseOS QE Security Team <qe-baseos-security> |
| Severity: | high | Docs Contact: | |
| Priority: | high | ||
| Version: | 9.2 | CC: | alakatos, nbubakov, qguo, sgrubb |
| Target Milestone: | rc | Keywords: | MigratedToJIRA |
| Target Release: | --- | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2023-08-08 11:14:22 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
The audit event is sent to syslog with a priority of ALERT which is higher than CRIT. In testing on F38, logging with LOG_CONS is not honored. I suppose journald does not have ALERT or CONS capabilities? Even as root, no console messages in case capabilities were needed. Writing to /dev/console also does not send a message to the console. The wall command also seems non-functional. Not sure what else can be done. I did make a patch (upstream commit 0b118c7) to shutdown auditd when processing any runlevel change. Maybe that is noticeable? |
Description of problem: CIS level 2 hardening requires having "admin_space_left_action = halt" in the configuration. If for some reason the space gets low on /var/log/audit, it becomes impossible to boot the system, since it goes down immediately during the boot. Checking the console output, nowhere we can see an explicit message stating auditd is halting the system. Only a message is recorded in the journal, *assuming the journal is persistent, which is far from being the usual case*: -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< -------- May 17 08:58:23 vm-audit9 auditd[682]: Audit daemon is low on disk space for logging May 17 08:58:23 vm-audit9 auditd[682]: The audit daemon is now halting the system May 17 08:58:23 vm-audit9 auditd[682]: No plugins found, not dispatching events May 17 08:58:23 vm-audit9 systemd[1]: Started RPC Bind. May 17 08:58:23 vm-audit9 auditd[682]: Init complete, auditd 3.0.7 listening for events (startup state enable) : -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< -------- But definitely this message is not visible to admins on the console: -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< -------- Mounting /var/log/audit... [ 3.249105] XFS (vdb): Mounting V5 Filesystem [ 3.274115] XFS (vdb): Ending clean mount [ OK ] Mounted /var/log/audit. [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... : [ OK ] Removed slice Slice /system/modprobe. [ OK ] Removed slice Slice /system/systemd-hibernate-resume. [ OK ] Stopped target RPC Port Mapper. [ OK ] Closed LVM2 poll daemon socket. [ OK ] Closed Process Core Dump Socket. [ OK ] Started /usr/sbin/lvm vgch…y --autoactivation event rhel. Stopping Load/Save Random Seed... [ OK ] Removed slice Slice /system/getty. [ OK ] Removed slice Slice /system/serial-getty. [ OK ] Removed slice Slice /system/sshd-keygen. [ OK ] Stopped Generate network units from Kernel command line. [ OK ] Stopped target Path Units. [ OK ] Stopped target Slice Units. : [ OK ] Stopped target Local File Systems. Unmounting /boot... Unmounting /run/credentials/systemd-sysctl.service... Unmounting /run/credential…-tmpfiles-setup-dev.service... [ 3.471501] XFS (vda1): Unmounting Filesystem Unmounting /var/log/audit... : [ 3.641752] reboot: Power down -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< -------- All this makes it *really hard* to troubleshoot, and usually requires Red Hat Support to enter the game, *and them to struggle a lot*. We need to highlight the operation on the console and probably log the message in the journal as "CRIT" as well. Version-Release number of selected component (if applicable): All audit releases How reproducible: Always Steps to Reproduce: 1. Configure auditd to halt the system # sed -i "s/admin_space_left_action = .*/admin_space_left_action = halt/" /etc/audit/auditd.conf 2. Set up a dedicated mount for /var/log/audit (1GB) # mkfs.xfs /dev/vdb # echo "/dev/vdb /var/log/audit xfs defaults 0 0" >> /etc/fstab # service auditd stop # systemctl daemon-reload # mount /var/log/audit # restorecon -Frv /var/log/audit # service auditd start 3. Fill the mount point with wreck # dd if=/dev/random of=/var/log/audit/filler bs=1M count=970 4. Reboot Actual results: System goes down during boot Expected results: System goes down after printing a *large visible red message* on the console