Bug 983483
| Summary: | audisp-remote terminates immediately with a non-empty queue | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Ondrej Moriš <omoris> |
| Component: | audit | Assignee: | Steve Grubb <sgrubb> |
| Status: | CLOSED NOTABUG | QA Contact: | BaseOS QE Security Team <qe-baseos-security> |
| Severity: | high | Docs Contact: | |
| Priority: | high | ||
| Version: | 7.0 | ||
| Target Milestone: | beta | ||
| Target Release: | --- | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2013-07-16 22:52:22 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | |||
| Bug Blocks: | 717785 | ||
I'm not entirely sure that's a proper test for audisp-remote. What's happening is bash tears down the pipes between the processes as soon as the echo completes. When this happens it triggers the select and then it sees a term signal was received. It never had a chance to send the event. In real life, we have to worry about systemd shutting down the system. It doesn't give much time until it sends sigkill to everything. What we do is just stop under the assumption that we can process any outstanding events next time, but we need to exit so that audispd can exit so that auditd can exit. One of those might finish first causing more fun. If you really want to test like that, you might be better off just making a shell script that keeps stdout open a bit longer, like: #!/bin/sh echo $1 sleep 2 then ./slow-echo "ABCD" | audisp-remote Thanks Steve for your explanation and hint, both are very helpful and solves the problem. However, a new issues was revealed afterwards (Bug 985142). Since it was shown that this is not a bug, I am closing this bugzilla. |
Description of problem: Changeset 771 in audit introduced the following change in audisp/plugins/remote/audisp-remote.c: --- a/trunk/audisp/plugins/remote/audisp-remote.c +++ b/trunk/audisp/plugins/remote/audisp-remote.c @@ -553,6 +553,5 @@ if (sock > 0 && FD_ISSET(sock, &wfd)) { // If so, try to drain backlog - while (q_queue_length(queue) && !suspend && - !stop && transport_ok) + while (q_queue_length(queue)&& !suspend && transport_ok) send_one(queue); } In other words, once a stop request (controlled by term_handler) is set, audisp-remote plugin terminates even though some messages are left in the queue. Unfortunately this leads to the following behaviour: # echo "ABC" | audisp-remote # tail /var/log/messages ... Jul 11 11:26:14 cc-toe3 audisp-remote: Connected to 10.0.0.2 Jul 11 11:26:14 cc-toe3 audisp-remote: Audisp-remote started with queue_size: 0 Jul 11 11:26:14 cc-toe3 audisp-remote: audisp-remote is exiting on stop request, queue_size: 1 And the "ABC" message is not recorded (it is lost in the unemptied queue). It seems that a stop request is "somehow" set too fast for some reason. I am bit lost in details, see also Additional info. Version-Release number of selected component (if applicable): audit-2.3.1-4.el7.x86_64 How reproducible: 100% Steps to Reproduce: 1. configure auditd and audisp-remote plugin as follows: auditd.conf: log_group = root priority_boost = 3 tcp_listen_queue = 5 tcp_max_per_addr = 1 tcp_client_max_idle = 0 name_format = none enable_krb5 = no krb5_principal = auditd DISP_qos = lossy log_file = /var/log/audit/audit.log log_format = RAW num_logs = 5 max_log_file = 1024 max_log_file_action = IGNORE space_left = 2 space_left_action = IGNORE admin_space_left = 1 admin_space_left_action = IGNORE disk_full_action = IGNORE disk_error_action = IGNORE action_mail_acct = root flush = INCREMENTAL freq = 20 tcp_listen_port = 60 tcp_client_ports = 61 audisp-remote.conf: port = 60 transport = tcp queue_depth = 200 format = managed network_retry_time = 1 max_tries_per_record = 3 max_time_per_record = 5 heartbeat_timeout = 15 network_failure_action = IGNORE disk_error_action = IGNORE remote_ending_action = IGNORE generic_error_action = IGNORE generic_warning_action = IGNORE disk_full_action = IGNORE disk_low_action = IGNORE remote_server = 10.0.0.2 local_port = 61 mode = immediate 2. restart the auditd service 3. echo "ABC" | audisp-remote 3. see /var/log/audit/audit.log (message should be there) Actual results: Message is not recorded. Expected results: Message is recorded. Additional info: Previously it worked in a way that even with a stop request set, the queue was emptied. This is better than non-emptying the backlog queue, but it caused problem when a queue was over-filled with hundreds of messages as follows: ... Jul 11 11:41:08 cc-toe3 auditd[11883]: Audit daemon has no space left on logging partition Jul 11 11:41:08 cc-toe3 audisp-remote: remote logging stopping due to remote server is low on disk space ... From my point of view, the best solution might be to keep emptying the backlog queue even after the stop request with the assumption that the same messages are aggregated on dropped at same point (which has pitfalls too, but...).