Red Hat Bugzilla – Bug 469735
RFE: Improve handling of "huge" logs....
Last modified: 2008-11-07 02:02:34 EST
Description of problem:
Some rawhide kernels appear to spew huge amounts to /var/log/messages: https://bugzilla.redhat.com/show_bug.cgi?id=468896
We're talking about 5G or 2G or .... (limited only by filesystem space, reboot time, etc.).
After logrotate "moves" this file out of the way, logwatch seems to
1). copy the file to /var/cache/logwatch/....
2). launch quite a host of perl scripts against the copy (?)
3). appear to leave stuff in /var/cache/logwatch (per email messaage):
You have old files in your logwatch tmpdir (/var/cache/logwatch):
The directories listed above were most likely created by a
logwatch run that failed to complete successfully. If so, you
may delete these directories.
Is it possible to get logwatch to not copy such enormous files, either by linking to them or by establishing a max copy limit?
For reference, here are the "logrotated" version of /var/log/messages:
[root@tlondon log]# ls -l messages*
-rw------- 1 root root 526 2008-11-03 11:17 messages
-rw------- 1 root root 1801472 2008-10-26 04:02 messages-20081026
-rw------- 1 root root 2074722503 2008-10-28 14:30 messages-20081028
-rw------- 1 root root 5394885660 2008-11-01 17:40 messages-20081101
-rw------- 1 root root 731978 2008-11-03 11:08 messages-20081103
Version-Release number of selected component (if applicable):
Steps to Reproduce:
The old logs should be removed - they could stay in /var/log/messages only if logwatch was terminated/killed before it finished properly - then there is a log which informs users about remaining files.
logwatch in standard version uses only log files which are not rotated - if you use --archives option then logwatch parse rotated files (messages-date) too.
For now there is necessary to use this files because of logwatch use some scripts to modify the log formats before "service scripts" parse it.
There is no possibility to create some option which could add limit to this temporary files - there could not be a test which check whether the /var/log/ files exceed the limit after each line (it is too much time consuming), and I think there is no need to test it per file (if logwatch fills the whole disk space then it finishes its work, remove all files and wrote this fact to output).
If you think this kind of option would be really valuable for you - then we could ask upstream maintainers about their opinion.
How about a simple check that just estimates the size of the copied files (from the size of the /var/log/message* files) and sees if there is more than that amount of free space?
The files are preprocessing not only copied - so there is the difference between their size and size of /var/log/logfile size. Moreover if you does not use --archives only /var/log/message file is copied - not all /var/log/message*. If logwatch fill whole disk free space - it finishes. So I'm not sure whether this option is necessary/helpful.