Bug 1048639
Summary: | Logwatch doesn't process fail2ban logs | ||||||||
---|---|---|---|---|---|---|---|---|---|
Product: | [Fedora] Fedora | Reporter: | Edward Kuns <eddie.kuns> | ||||||
Component: | logwatch | Assignee: | Jan Synacek <jsynacek> | ||||||
Status: | CLOSED ERRATA | QA Contact: | Fedora Extras Quality Assurance <extras-qa> | ||||||
Severity: | medium | Docs Contact: | |||||||
Priority: | medium | ||||||||
Version: | 22 | CC: | alanh, eddie.kuns, frank, gbcox, jsynacek, kevin.masaryk, legolegs, redhat, richardfearn, varekova, warlord | ||||||
Target Milestone: | --- | ||||||||
Target Release: | --- | ||||||||
Hardware: | noarch | ||||||||
OS: | Linux | ||||||||
Whiteboard: | |||||||||
Fixed In Version: | logwatch-7.4.1-5.20150731svn293.fc22 | Doc Type: | Bug Fix | ||||||
Doc Text: | Story Points: | --- | |||||||
Clone Of: | Environment: | ||||||||
Last Closed: | 2015-10-03 21:18:36 UTC | Type: | Bug | ||||||
Regression: | --- | Mount Type: | --- | ||||||
Documentation: | --- | CRM: | |||||||
Verified Versions: | Category: | --- | |||||||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
Cloudforms Team: | --- | Target Upstream Version: | |||||||
Embargoed: | |||||||||
Attachments: |
|
Description
Edward Kuns
2014-01-06 01:32:41 UTC
Created attachment 845873 [details]
Proposed patch to fix logwatch processing for fail2ban
The patch should apply just fine from /usr/share/logwatch -- I fixed these issues:
1) LogWatch expects fail2ban to have its own log file, when it logs (by default) to /var/log/messages
2) fail2ban logs to syslog using a variety of service names, and we want to process all of them.
3) There are some new lines output by fail2ban that we want to ignore, such as when it says a jail uses pyinotify. I added all of these lines that I have personally observed.
4) The "already banned" complaint is now output by fail2ban at INFO, not WARNING. There may be more instances of such changes, of course. I only changed the instance I have personally observed.
Thanks. Testing for F20. Works great for F20. Hope this patch is incorporated into the rpm. Thanks again! Updated to F20 as this also applies there... Edward, since you already wrote a patch, would you mind proposing it upstream (http://sourceforge.net/p/logwatch/mailman/logwatch-devel) ? Jan, Edward: Regarding the patch... I just installed the update fail2ban-0.9-2.fc20.noarch and now the messages are no longer going to /var/log/messages... now they go to /var/log/fail2ban.log which is where they probably should have been going in the first place. Looks like I missed the fact that logwatch is looking for "fail2ban" and not the new location which is "fail2ban.log". I made the change and now am testing. With the latest update the patch no longer works. This is also affecting me... Did a "yum downgrade fail2ban" to go back to fail2ban-0.9-0.3.git1f1a561.fc20.noarch from fail2ban-0.9-2.fc20.noarch and logwatcher is working again. I did a "yum changelog 3 fail2ban" before downgrading and didn't see much: fail2ban-0.9-2.fc20.noarch installed * Tue Mar 18 05:00:00 2014 Orion Poplawski <orion.com> - 0.9-2 - Use Fedora paths - Start after firewalld (bug #1067147) * Mon Mar 17 05:00:00 2014 Orion Poplawski <orion.com> - 0.9-1 - Update to 0.9 * Tue Sep 24 05:00:00 2013 Orion Poplawski <orion.com> - 0.9-0.3.git1f1a561 - Update to current 0.9 git branch - Rebase init patch, drop jail.d and notmp patch applied upstream Unfortunately, I am still on Fedora 19. I haven't had the free time to upgrade this system. This means I'm not able to look at what is going on with Fedora 20. However, I did supply my patch upstream as https://sourceforge.net/p/logwatch/patches/35/ That will handle the other improvements I made to the logwatch script for handling fail2ban logs. Fedora 19 is still on older versions of these packages: fail2ban-0.8.11-2.fc19.noarch logwatch-7.4.0-25.20130522svn140.fc19.noarch I just set up fail2ban, and noticed the lack of reporting in Logwatch. From looking at /usr/share/logwatch/scripts/services/fail2ban, the issue appears that it's expecting messages preceded by WARNING rather than NOTICE. Example fail2ban line: 2014-06-22 14:40:29,084 fail2ban.server.actions[6938]: NOTICE [postfix-sasl] Ban xx.xx.xx.xx Logwatch is looking for a line staring with WARNING: } elsif ( my ($Service,$Action,$Host) = ($ThisLine =~ m/WARNING:?\s\[?(.*?)[]:]?\s(Ban|Unban)[^\.]* (\S+)/)) { So it doesn't match and doesn't get reported. If I change this to NOTICE, it works. @Alan, thanks very much for the circumvention. Worked perfectly! @Edward, did you put that into your patch that you submitted upstream? There was an update to logwatch: logwatch-7.4.0-33.20140704svn198.fc20.noarch Which meant I needed to reapply the change from @Alex described above. Created upstream ticket: https://sourceforge.net/p/logwatch/bugs/37/ Edward, the patch you suggested was accepted and incorporated upstream and will be available in v7.4.1 - refer to the upstream ticket: http://sourceforge.net/p/logwatch/patches/35/ for more information. Alan's patch is also required in addition to what Edward reported. The ticket I created reporting Alan's change was accepted and incorporated upstream and will be available in v7.4.1 - refer to the upstream ticket: https://sourceforge.net/p/logwatch/bugs/37/ for more information. Please, try https://admin.fedoraproject.org/updates/logwatch-7.4.1-1.20140924svn242.fc20 if it resolves the issue. No, it isn't working. I'll go back upstream and ask them what happened. Created attachment 948123 [details]
Proposed patch
I went through the issues with the current release (logwatch-7.4.1-2.20140924svn242) and came up with a patch.
In the config file, OnlySerice only works with syslogd-style loglines, but fail2ban's log has a different date format so nothing matches. Also, the empty regex in (|\..+) (before the pipe) will match anything.
I changed it to use OnlyContains, and match "fail2ban" not followed by a '-'.
In the script file, I added some additional matches to keep informational lines from showing up as unmatched. These mismatches don't print at --detail low, but will at higher --detail levels.
Service names were showing up with a leading square bracket, like "[dovecot". This is due to a missing + in line 86.
"Already banned" messages can show up with a NOTICE tag. so I added that.
Thanks for the patch Alan, works great. Are you going to propose upstream or do you want me to? Sure, go ahead and submit it. Created upstream ticket: https://sourceforge.net/p/logwatch/patches/38/ Just adding a "me too". I tried this patch and it works for me. (although I'd kind of like to see a list of the banned IPs, but a summary is okay -- and I can see the list from the sshd errors). Running FC20 (I'll try FC21 shortly). (In reply to Derek Atkins from comment #22) You can display the IPs by setting "Detail = 5" in conf/services/fail2ban.conf. This bug is also present in FC21. This message is a reminder that Fedora 20 is nearing its end of life. Approximately 4 (four) weeks from now Fedora will stop maintaining and issuing updates for Fedora 20. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a Fedora 'version' of '20'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 20 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete. Carry forward to F22. Could anyone please try again with the latest package? It's built for F23, but it should work in F22 as well. http://koji.fedoraproject.org/koji/buildinfo?buildID=674481 (In reply to Jan Synacek from comment #26) > Could anyone please try again with the latest package? It's built for F23, > but it should work in F22 as well. > > http://koji.fedoraproject.org/koji/buildinfo?buildID=674481 Thanks, I'll give it a whirl. Hey Jan, It works! This can now be closed. Thanks much! logwatch-7.4.1-5.20150731svn293.fc22 has been submitted as an update to Fedora 22. https://bodhi.fedoraproject.org/updates/FEDORA-2015-14203 logwatch-7.4.1-5.20150731svn293.fc22 has been pushed to the Fedora 22 testing repository. If problems still persist, please make note of it in this bug report.\nIf you want to test the update, you can install it with \n su -c 'yum --enablerepo=updates-testing update logwatch'. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2015-14203 logwatch-7.4.1-5.20150731svn293.fc22 has been pushed to the Fedora 22 stable repository. If problems still persist, please make note of it in this bug report. The version logwatch-7.4.1-5.20150731svn293.fc22 is imperfect. It still outputs service names with extra "[": --------------------- fail2ban-messages Begin ------------------------ Banned services with Fail2Ban: Bans:Unbans [sshd: [ 17:1 ] ---------------------- fail2ban-messages End ------------------------- Its happens because of wrong regex in /usr/share/logwatch/scripts/services/fail2ban on line 87: m/(?:WARNING|NOTICE):?\s\[?(.*?)[]:]?\s(Ban|Unban)[^\.]* (\S+)/) it should be m/(?:WARNING|NOTICE):?\s+\[?(.*?)[]:]?\s(Ban|Unban)[^\.]* (\S+)/) The patch by Alan Hamilton does have that extra plus signg but for some reason it ended missing from logwatch-7.4.1-5.20150731svn293.fc22. |