Bug 469735 - RFE: Improve handling of "huge" logs....
RFE: Improve handling of "huge" logs....
Status: CLOSED WONTFIX
Product: Fedora
Classification: Fedora
Component: logwatch (Show other bugs)
rawhide
All Linux
medium Severity low
: ---
: ---
Assigned To: Ivana Varekova
Fedora Extras Quality Assurance
: FutureFeature
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2008-11-03 14:19 EST by Tom London
Modified: 2008-11-07 02:02 EST (History)
1 user (show)

See Also:
Fixed In Version:
Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2008-11-06 11:14:17 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Tom London 2008-11-03 14:19:58 EST
Description of problem:
Some rawhide kernels appear to spew huge amounts to /var/log/messages: https://bugzilla.redhat.com/show_bug.cgi?id=468896

We're talking about 5G or 2G or .... (limited only by filesystem space, reboot time, etc.).

After logrotate "moves" this file out of the way, logwatch seems to 
1). copy the file to /var/cache/logwatch/....
2). launch quite a host of perl scripts against the copy (?)
3). appear to leave stuff in /var/cache/logwatch (per email messaage):
/etc/cron.daily/0logwatch:

You have old files in your logwatch tmpdir (/var/cache/logwatch):
	logwatch.bo67Cx2i
The directories listed above were most likely created by a
logwatch run that failed to complete successfully.  If so, you
may delete these directories.

Is it possible to get logwatch to not copy such enormous files, either by linking to them or by establishing a max copy limit?

Other?

For reference, here are the "logrotated" version of /var/log/messages:

[root@tlondon log]# ls -l messages*
-rw------- 1 root root        526 2008-11-03 11:17 messages
-rw------- 1 root root    1801472 2008-10-26 04:02 messages-20081026
-rw------- 1 root root 2074722503 2008-10-28 14:30 messages-20081028
-rw------- 1 root root 5394885660 2008-11-01 17:40 messages-20081101
-rw------- 1 root root     731978 2008-11-03 11:08 messages-20081103
[root@tlondon log]# 


Version-Release number of selected component (if applicable):
logwatch-7.3.6-31.fc10.noarch

How reproducible:


Steps to Reproduce:
1.
2.
3.
  
Actual results:


Expected results:


Additional info:
Comment 1 Ivana Varekova 2008-11-06 11:14:17 EST
The old logs should be removed - they could stay in /var/log/messages only if logwatch was terminated/killed before it finished properly - then there is a log which informs users about remaining files.

logwatch in standard version uses only log files which are not rotated - if you use --archives option then logwatch parse rotated files (messages-date) too. 

For now there is necessary to use this files because of logwatch use some scripts to modify the log formats before "service scripts" parse it. 
There is no possibility to create some option which could add limit to this temporary files - there could not be a test which check whether the /var/log/ files exceed the limit after each line (it is too much time consuming), and I think there is no need to test it per file (if logwatch fills the whole disk space then it finishes its work, remove all files and wrote this fact to output). 

If you think this kind of option would be really valuable for you - then we could ask upstream maintainers about their opinion.
Comment 2 Tom London 2008-11-06 11:22:57 EST
How about a simple check that just estimates the size of the copied files (from the size of the /var/log/message* files) and sees if there is more than that amount of free space?
Comment 3 Ivana Varekova 2008-11-07 02:02:34 EST
The files are preprocessing not only copied - so there is the difference between their size and size of /var/log/logfile size. Moreover if you does not use --archives only /var/log/message file is copied - not all /var/log/message*. If logwatch fill whole disk free space - it finishes. So I'm not sure whether this option is necessary/helpful.

Note You need to log in before you can comment on or make changes to this bug.