Red Hat Bugzilla – Bug 40852
Logcheck errors because new egrep can't handle large pattern files
Last modified: 2008-05-01 11:38:00 EDT
I guess this is technically a problem with egrep, rather than logcheck - but it's resulting in problems with logcheck. The version of egrep distributed with RH7.1 seems not to be able to handle as large pattern files (specified with the -f option) as it used to be able to. The files which I use in /etc/logcheck are sufficiently large for it to bail out and not bother checking my data at all. I've attached a perl script which emulates grep for the purposes of logcheck, and a trivial patch to /etc/logcheck/logcheck.conf to use it. (Also patches /etc/cron.hourly/logcheck to make it nice logcheck, which I've found to be helpful when I don't want the machine it's running on to grind to a halt!) Cheers, Chris.
Created attachment 18611 [details] Perl script to act as grep replacement for logcheck
Created attachment 18612 [details] Trivial patch to make logcheck use grep.pl
I don't want to incorporate this in. It creates a dependancy on perl through a patch that is not a good idea. I am reassigning this and changing the component to grep so that we can fix the actual problem (egrep). Tim
I'm not seeing any logcheck problems on my rawhide installation (grep-2.5e-2), assuming it's fixed.
Yeah, it does seem to be fixed in grep-2.5e-2. It's *incredibly* slow, though. echo "Hello" | egrep -f /my/patternfile takes 45 seconds on my unloaded system, where /my/patternfile is about 32K in size, as compared with 0.1 seconds for the equivalent using the above grep.pl script. (Might be worth assigning this back to Tim? There's no way I'm going to use egrep in my logcheck script if it's going to take 45 seconds to check every single line of my logs - it'll take all year to finish...) Thanks, Chris.
Does pcregrep (from the pcre package) work better for you?
Ah, no, forget about that question, pcregrep doesn't handle file input at all.
Tim, grep is fixed; assigning this back to you because of the performance issues. It's up to you to apply or WONTFIX this... ;)
Hmm. I would be much more comfortable with the patch if it were in python since python is in the base set of dependancies for the distro while perl is not. Is this an option for you? Tim
Yep, I take your point completely. I don't speak Python - but I know a man who does... Translation attached, though I'm sure there are cleaner ways of writing the same thing. And it seems to be slightly faster than the Perl version. Thanks again, Chris.
Created attachment 18871 [details] Translation of grep.pl into Python
OK. I have included the pgrep.py python script. logcheck-1.1.1-8 should show up in rawhide once it's updated again. Until then, you can get it from http://people.redhat.com/timp/{RPMS,SRPMS} Tim
Great - thanks. Problem is, the patch to use pgrep.py is included, but the corresponding %patch directive is missing from the .spec!
Fixed. You can find it in the same place (same release number). Tim
Great - thanks. One last thing: I don't think /etc/logcheck/logcheck.conf should be marked %config(noreplace). This has the effect that if someone has altered that file, the new change to use pgrep.py doesn't make its way in, but there's no indication to the user that it should have done. It's fine to mark the other files in /etc/logcheck as %config(noreplace), but logcheck.conf should just be marked %config. Chris.