Bug 1464589 - [RFE] Compress older audit logs
[RFE] Compress older audit logs
Status: NEW
Product: OpenShift Container Platform
Classification: Red Hat
Component: RFE (Show other bugs)
3.4.1
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Derek Carr
Xiaoli Tian
: OpsBlocker
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2017-06-23 16:50 EDT by bmorriso
Modified: 2017-10-13 10:34 EDT (History)
8 users (show)

See Also:
Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description bmorriso 2017-06-23 16:50:16 EDT
Description of problem:

Starting with OpenShift v3.4, we have started enabling the audit logging feature in our master-config.yaml:

auditConfig:
  enabled: true
  auditFilePath: /var/log/openshift-master-audit.log
  maximumFileRetentionDays: 14
  maximumFileSizeMegabytes: 100

Currently, the previous day's logs are stored uncompressed. On a busy cluster, we have seen >1GB of logs generated in a day, which takes up considerable storage space. It would be great if audit logging gzipped these older log files automatically. 

Version-Release number of selected component (if applicable):
v3.4.1.18

How reproducible:
always
Comment 1 Peter Portante 2017-08-31 23:29:10 EDT
Why not also use "maximumRetainedFiles: 20" so that you cap the total disk usage at 2 GB?
Comment 2 Peter Portante 2017-08-31 23:32:12 EDT
If instead we don't use an auditFilePath configuration, these AUDIT logs will go to stdout.  If we then use fluentd to collect logs from the master nodes as well, then all these audit logs will land in Elasticsearch under the .operations indices.

We could then ask the logging team to parse the audit logs to decorate those logs with metadata derived from the logs themselves to make them easier to search and correlate in Elasticsearch.

Note You need to log in before you can comment on or make changes to this bug.