Red Hat Bugzilla – Bug 40852
Logcheck errors because new egrep can't handle large pattern files
Last modified: 2008-05-01 11:38:00 EDT
I guess this is technically a problem with egrep, rather than logcheck -
but it's resulting in problems with logcheck. The version of egrep
distributed with RH7.1 seems not to be able to handle as large pattern
files (specified with the -f option) as it used to be able to. The files
which I use in /etc/logcheck are sufficiently large for it to bail out and
not bother checking my data at all.
I've attached a perl script which emulates grep for the purposes of
logcheck, and a trivial patch to /etc/logcheck/logcheck.conf to use it.
(Also patches /etc/cron.hourly/logcheck to make it nice logcheck, which
I've found to be helpful when I don't want the machine it's running on to
grind to a halt!)
Created attachment 18611 [details]
Perl script to act as grep replacement for logcheck
Created attachment 18612 [details]
Trivial patch to make logcheck use grep.pl
I don't want to incorporate this in. It creates a dependancy on perl through a
patch that is not a good idea. I am reassigning this and changing the component
to grep so that we can fix the actual problem (egrep).
I'm not seeing any logcheck problems on my rawhide installation (grep-2.5e-2),
assuming it's fixed.
Yeah, it does seem to be fixed in grep-2.5e-2. It's *incredibly* slow, though.
echo "Hello" | egrep -f /my/patternfile takes 45 seconds on my unloaded system,
where /my/patternfile is about 32K in size, as compared with 0.1 seconds for the
equivalent using the above grep.pl script.
(Might be worth assigning this back to Tim? There's no way I'm going to use
egrep in my logcheck script if it's going to take 45 seconds to check every
single line of my logs - it'll take all year to finish...)
Does pcregrep (from the pcre package) work better for you?
Ah, no, forget about that question, pcregrep doesn't handle file input at all.
Tim, grep is fixed; assigning this back to you because of the performance
issues. It's up to you to apply or WONTFIX this... ;)
Hmm. I would be much more comfortable with the patch if it were in python since
python is in the base set of dependancies for the distro while perl is not. Is
this an option for you?
Yep, I take your point completely. I don't speak Python - but I know a man who
does... Translation attached, though I'm sure there are cleaner ways of writing
the same thing. And it seems to be slightly faster than the Perl version.
Created attachment 18871 [details]
Translation of grep.pl into Python
OK. I have included the pgrep.py python script. logcheck-1.1.1-8 should show up
in rawhide once it's updated again. Until then, you can get it from
Great - thanks. Problem is, the patch to use pgrep.py is included, but the
corresponding %patch directive is missing from the .spec!
Fixed. You can find it in the same place (same release number).
Great - thanks. One last thing: I don't think /etc/logcheck/logcheck.conf
should be marked %config(noreplace). This has the effect that if someone has
altered that file, the new change to use pgrep.py doesn't make its way in, but
there's no indication to the user that it should have done.
It's fine to mark the other files in /etc/logcheck as %config(noreplace), but
logcheck.conf should just be marked %config.