Bug 1308872
| Summary: | Need protection from logs filling up the disk space | ||
|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Prateek Arora <parora> |
| Component: | openstack-tripleo | Assignee: | James Slagle <jslagle> |
| Status: | CLOSED NOTABUG | QA Contact: | yeylon <yeylon> |
| Severity: | low | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 7.0 (Kilo) | CC: | mburns, rhel-osp-director-maint, srevivo, yeylon |
| Target Milestone: | --- | ||
| Target Release: | 8.0 (Liberty) | ||
| Hardware: | All | ||
| OS: | All | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2016-02-23 10:02:56 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Description of problem: The environment had some problems and as such the amount of logs that were produced were in the range of 1300 Gigs in a very short span of time in an hour so. The logrotate utility is configured by cron to run daily and keep only 140M logs but like here in case the logrotate didnt have a chance to run and the customer was in a fix before that /var/log/neutron/*.log { rotate 14 size 10M missingok compress copytruncate } Version-Release number of selected component (if applicable): RHOS 7.0 How reproducible: Always Steps to Reproduce: 1.By having runaway logs on a RHOS deployment 2. 3. Actual results: The disk space filled out completely Expected results: Some defence mechanism to handle such things Additional info: The logs get compressed become very small like a 250 GB log got compressed to 4.4G, see below 656G openvswitch-agent.log 4.4G openvswitch-agent.log-20160121.gz ->~250GB 12G openvswitch-agent.log-20160122.gz ->~684GB 635G openvswitch-agent.log-20160123 3.2M openvswitch-agent.log-20160124 but if the disk does not have space for compression to succeed, the problem arises with huge logs and no disk to write to them or do anything/ The logrotate utility is working fime and the cron is configured to run it daily and it is doing as expected