Bug 469735

Summary: RFE: Improve handling of "huge" logs....
Product: [Fedora] Fedora Reporter: Tom London <selinux>
Component: logwatchAssignee: Ivana Varekova <varekova>
Status: CLOSED WONTFIX QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: low Docs Contact:
Priority: medium    
Version: rawhideCC: varekova
Target Milestone: ---Keywords: FutureFeature
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2008-11-06 16:14:17 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Tom London 2008-11-03 19:19:58 UTC
Description of problem:
Some rawhide kernels appear to spew huge amounts to /var/log/messages: https://bugzilla.redhat.com/show_bug.cgi?id=468896

We're talking about 5G or 2G or .... (limited only by filesystem space, reboot time, etc.).

After logrotate "moves" this file out of the way, logwatch seems to 
1). copy the file to /var/cache/logwatch/....
2). launch quite a host of perl scripts against the copy (?)
3). appear to leave stuff in /var/cache/logwatch (per email messaage):
/etc/cron.daily/0logwatch:

You have old files in your logwatch tmpdir (/var/cache/logwatch):
	logwatch.bo67Cx2i
The directories listed above were most likely created by a
logwatch run that failed to complete successfully.  If so, you
may delete these directories.

Is it possible to get logwatch to not copy such enormous files, either by linking to them or by establishing a max copy limit?

Other?

For reference, here are the "logrotated" version of /var/log/messages:

[root@tlondon log]# ls -l messages*
-rw------- 1 root root        526 2008-11-03 11:17 messages
-rw------- 1 root root    1801472 2008-10-26 04:02 messages-20081026
-rw------- 1 root root 2074722503 2008-10-28 14:30 messages-20081028
-rw------- 1 root root 5394885660 2008-11-01 17:40 messages-20081101
-rw------- 1 root root     731978 2008-11-03 11:08 messages-20081103
[root@tlondon log]# 


Version-Release number of selected component (if applicable):
logwatch-7.3.6-31.fc10.noarch

How reproducible:


Steps to Reproduce:
1.
2.
3.
  
Actual results:


Expected results:


Additional info:

Comment 1 Ivana Varekova 2008-11-06 16:14:17 UTC
The old logs should be removed - they could stay in /var/log/messages only if logwatch was terminated/killed before it finished properly - then there is a log which informs users about remaining files.

logwatch in standard version uses only log files which are not rotated - if you use --archives option then logwatch parse rotated files (messages-date) too. 

For now there is necessary to use this files because of logwatch use some scripts to modify the log formats before "service scripts" parse it. 
There is no possibility to create some option which could add limit to this temporary files - there could not be a test which check whether the /var/log/ files exceed the limit after each line (it is too much time consuming), and I think there is no need to test it per file (if logwatch fills the whole disk space then it finishes its work, remove all files and wrote this fact to output). 

If you think this kind of option would be really valuable for you - then we could ask upstream maintainers about their opinion.

Comment 2 Tom London 2008-11-06 16:22:57 UTC
How about a simple check that just estimates the size of the copied files (from the size of the /var/log/message* files) and sees if there is more than that amount of free space?

Comment 3 Ivana Varekova 2008-11-07 07:02:34 UTC
The files are preprocessing not only copied - so there is the difference between their size and size of /var/log/logfile size. Moreover if you does not use --archives only /var/log/message file is copied - not all /var/log/message*. If logwatch fill whole disk free space - it finishes. So I'm not sure whether this option is necessary/helpful.