Red Hat Bugzilla – Bug 1312490
Fluentd holds open log files that have been deleted
Last modified: 2017-12-12 21:29:21 EST
Created attachment 1130909 [details]
Description of problem: Fluentd is holding log files open that have been deleted. The inode locks stay in place for hours, possibly longer, if the fluentd container is not restarted.
Version-Release number of selected component (if applicable): 18.104.22.168
How reproducible: always, when metrics are on
Steps to Reproduce:
1. enable openshift metrics
2. rotate system logs
logs removed by logrotate are still open and taking space
fluentd should eventually let the files go
See attached lsof output
We ended up changing the fluentd image to set it to ignore gzipped/archived files, which has prevented issues with us running out of disk space.