Bug 1578731

Summary: Allow logrotate configuration to control log retention based on age
Product: Red Hat OpenStack Reporter: Nathan Kinder <nkinder>
Component: puppet-tripleoAssignee: RHOS Maint <rhos-maint>
Status: CLOSED CURRENTRELEASE QA Contact: Pavan <pkesavar>
Severity: high Docs Contact:
Priority: high    
Version: 13.0 (Queens)CC: agurenko, bdobreli, ealcaniz, hrybacki, jjoyce, josorior, jschluet, knoha, pablo.iranzo, pkesavar, slinaber, tvignaud
Target Milestone: z2Keywords: TestOnly, Triaged, ZStream
Target Release: 13.0 (Queens)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: puppet-tripleo-8.3.4-2.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1627740 (view as bug list) Environment:
Last Closed: 2018-11-13 11:46:04 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1627740    

Description Nathan Kinder 2018-05-16 10:10:45 UTC
Our current logrotate configuration in puppet-tripleo handles rotation based off of log size and number of rotated logs (rotate at 10M size and retain 14 logs).  For things such as GDPR, we need to be able to define the retention based on age as the logs may contain personally identifiable information that needs to be deleted on demand (within a window of time).  Ideally, we would only retain logs for <1M (2 weeks might be appropriate), with the option to configure the policy differently via Director.

Comment 1 Alex Schultz 2018-05-17 14:13:59 UTC
*** Bug 1578854 has been marked as a duplicate of this bug. ***

Comment 2 Bogdan Dobrelya 2018-05-22 07:56:43 UTC
I'm not happy with https://review.openstack.org/569805 and https://review.openstack.org/569806, and I have to tweak the maxsize defaults prolly. As is, the postrotate script purges all files in /var/log/containers, if there were *any* files rotated in /var/log/containers. So logfiles not reaching the maxsize (defaults to 10M) will be removed and never rotated. So tweaking should make those "any" files to become "all", which I think may be lowering the maxsize defaults to something low enough to put all files under rotation (logrotate has a hourly execution schedule).

I wish I could instead just access the files' *real* creation time, then I'd alter the postrotate script to purge only the files having its life-time expired. But that's not the case sadly and cannot be done w/o real creation time reported by FS drivers.

Comment 3 Bogdan Dobrelya 2018-05-22 08:43:39 UTC
Attempting to improve things in https://review.openstack.org/#/c/569938

Comment 4 Bogdan Dobrelya 2018-05-24 07:51:44 UTC
It should be fine now with https://review.openstack.org/#/c/570328 added on top.

Comment 5 Bogdan Dobrelya 2018-05-28 08:08:58 UTC
@Juan, the part of the issue addressing containerized logrotate and containerized services is done, patches in stable/queens. Please feel free to take it over for the remaining work - host services and host logroatate/journald configuration.

Comment 7 Keigo Noha 2018-07-19 00:02:11 UTC
Hello Bogdan,

Your comment to Juan is not have needinfo flag.
Could you add needinfo to Juan and ask Juan to proceed this bugzilla?

Best Regards,
Keigo Noha

Comment 13 Keigo Noha 2018-10-23 07:30:28 UTC
Hello Bogdan,

Could you proceed this bugzilla forward?
In the upstream, all related work seemed to be done.
Can we backport the fixes into RHOSP13?

Best Regards,
Keigo Noha

Comment 14 Keigo Noha 2018-10-31 05:09:18 UTC
Hello Bogdan,

It looks that the upstream fixes were merged and RHOSP13z3 candidate contains the fixes.
And puppet-tripleo-8.3.4-2.el7 seems to contain the fix in this bugzilla.
Is it correct? If yes, we can close this bugzilla.

Best Regards,
Keigo Noha

Comment 15 Bogdan Dobrelya 2018-11-05 12:20:23 UTC
Yes, looks correct.

Comment 16 Lon Hohberger 2018-11-06 11:44:05 UTC
According to our records, this should be resolved by puppet-tripleo-8.3.4-5.el7ost.  This build is available now.