Description of problem: Logwatch doesn't process the fail2ban logs. In Fedora 19, fail2ban is set up to use syslog, and its messages go to /var/log/messages. Logwatch assumes that the fail2ban logs will be found in /var/log/fail2ban. After fixing this, I found many unknown messages, so I fixed those as well. Version-Release number of selected component (if applicable): logwatch-7.4.0-25.20130522svn140.fc19.noarch fail2ban-0.8.11-2.fc19.noarch How reproducible: 100% Steps to Reproduce: 1. Configure fail2ban in the default Fedora 19 manner 2. Wait for fail2ban to ban something, or force it to do so 3. Wait for the next logwatch execution, or manually run logwatch Actual results: No fail2ban analysis in the logwatch output. Expected results: I expect to see fail2ban analysis in the logwatch output. Additional info: I will attach the patch I used to fix this. I don't know how different Fedora's fail2ban is from the default, so I don't know whether or not these patches apply to the generic logwatch distribution or not. But they definitely apply to Fedora, as it configures these applications.
Created attachment 845873 [details] Proposed patch to fix logwatch processing for fail2ban The patch should apply just fine from /usr/share/logwatch -- I fixed these issues: 1) LogWatch expects fail2ban to have its own log file, when it logs (by default) to /var/log/messages 2) fail2ban logs to syslog using a variety of service names, and we want to process all of them. 3) There are some new lines output by fail2ban that we want to ignore, such as when it says a jail uses pyinotify. I added all of these lines that I have personally observed. 4) The "already banned" complaint is now output by fail2ban at INFO, not WARNING. There may be more instances of such changes, of course. I only changed the instance I have personally observed.
Thanks. Testing for F20.
Works great for F20. Hope this patch is incorporated into the rpm. Thanks again!
Updated to F20 as this also applies there...
Edward, since you already wrote a patch, would you mind proposing it upstream (http://sourceforge.net/p/logwatch/mailman/logwatch-devel) ?
Jan, Edward: Regarding the patch... I just installed the update fail2ban-0.9-2.fc20.noarch and now the messages are no longer going to /var/log/messages... now they go to /var/log/fail2ban.log which is where they probably should have been going in the first place.
Looks like I missed the fact that logwatch is looking for "fail2ban" and not the new location which is "fail2ban.log". I made the change and now am testing.
With the latest update the patch no longer works.
This is also affecting me... Did a "yum downgrade fail2ban" to go back to fail2ban-0.9-0.3.git1f1a561.fc20.noarch from fail2ban-0.9-2.fc20.noarch and logwatcher is working again. I did a "yum changelog 3 fail2ban" before downgrading and didn't see much: fail2ban-0.9-2.fc20.noarch installed * Tue Mar 18 05:00:00 2014 Orion Poplawski <orion.com> - 0.9-2 - Use Fedora paths - Start after firewalld (bug #1067147) * Mon Mar 17 05:00:00 2014 Orion Poplawski <orion.com> - 0.9-1 - Update to 0.9 * Tue Sep 24 05:00:00 2013 Orion Poplawski <orion.com> - 0.9-0.3.git1f1a561 - Update to current 0.9 git branch - Rebase init patch, drop jail.d and notmp patch applied upstream
Unfortunately, I am still on Fedora 19. I haven't had the free time to upgrade this system. This means I'm not able to look at what is going on with Fedora 20. However, I did supply my patch upstream as https://sourceforge.net/p/logwatch/patches/35/ That will handle the other improvements I made to the logwatch script for handling fail2ban logs. Fedora 19 is still on older versions of these packages: fail2ban-0.8.11-2.fc19.noarch logwatch-7.4.0-25.20130522svn140.fc19.noarch
I just set up fail2ban, and noticed the lack of reporting in Logwatch. From looking at /usr/share/logwatch/scripts/services/fail2ban, the issue appears that it's expecting messages preceded by WARNING rather than NOTICE. Example fail2ban line: 2014-06-22 14:40:29,084 fail2ban.server.actions[6938]: NOTICE [postfix-sasl] Ban xx.xx.xx.xx Logwatch is looking for a line staring with WARNING: } elsif ( my ($Service,$Action,$Host) = ($ThisLine =~ m/WARNING:?\s\[?(.*?)[]:]?\s(Ban|Unban)[^\.]* (\S+)/)) { So it doesn't match and doesn't get reported. If I change this to NOTICE, it works.
@Alan, thanks very much for the circumvention. Worked perfectly! @Edward, did you put that into your patch that you submitted upstream?
There was an update to logwatch: logwatch-7.4.0-33.20140704svn198.fc20.noarch Which meant I needed to reapply the change from @Alex described above.
Created upstream ticket: https://sourceforge.net/p/logwatch/bugs/37/
Edward, the patch you suggested was accepted and incorporated upstream and will be available in v7.4.1 - refer to the upstream ticket: http://sourceforge.net/p/logwatch/patches/35/ for more information. Alan's patch is also required in addition to what Edward reported. The ticket I created reporting Alan's change was accepted and incorporated upstream and will be available in v7.4.1 - refer to the upstream ticket: https://sourceforge.net/p/logwatch/bugs/37/ for more information.
Please, try https://admin.fedoraproject.org/updates/logwatch-7.4.1-1.20140924svn242.fc20 if it resolves the issue.
No, it isn't working. I'll go back upstream and ask them what happened.
Created attachment 948123 [details] Proposed patch I went through the issues with the current release (logwatch-7.4.1-2.20140924svn242) and came up with a patch. In the config file, OnlySerice only works with syslogd-style loglines, but fail2ban's log has a different date format so nothing matches. Also, the empty regex in (|\..+) (before the pipe) will match anything. I changed it to use OnlyContains, and match "fail2ban" not followed by a '-'. In the script file, I added some additional matches to keep informational lines from showing up as unmatched. These mismatches don't print at --detail low, but will at higher --detail levels. Service names were showing up with a leading square bracket, like "[dovecot". This is due to a missing + in line 86. "Already banned" messages can show up with a NOTICE tag. so I added that.
Thanks for the patch Alan, works great. Are you going to propose upstream or do you want me to?
Sure, go ahead and submit it.
Created upstream ticket: https://sourceforge.net/p/logwatch/patches/38/
Just adding a "me too". I tried this patch and it works for me. (although I'd kind of like to see a list of the banned IPs, but a summary is okay -- and I can see the list from the sshd errors). Running FC20 (I'll try FC21 shortly).
(In reply to Derek Atkins from comment #22) You can display the IPs by setting "Detail = 5" in conf/services/fail2ban.conf. This bug is also present in FC21.
This message is a reminder that Fedora 20 is nearing its end of life. Approximately 4 (four) weeks from now Fedora will stop maintaining and issuing updates for Fedora 20. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a Fedora 'version' of '20'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 20 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete.
Carry forward to F22.
Could anyone please try again with the latest package? It's built for F23, but it should work in F22 as well. http://koji.fedoraproject.org/koji/buildinfo?buildID=674481
(In reply to Jan Synacek from comment #26) > Could anyone please try again with the latest package? It's built for F23, > but it should work in F22 as well. > > http://koji.fedoraproject.org/koji/buildinfo?buildID=674481 Thanks, I'll give it a whirl.
Hey Jan, It works! This can now be closed. Thanks much!
logwatch-7.4.1-5.20150731svn293.fc22 has been submitted as an update to Fedora 22. https://bodhi.fedoraproject.org/updates/FEDORA-2015-14203
logwatch-7.4.1-5.20150731svn293.fc22 has been pushed to the Fedora 22 testing repository. If problems still persist, please make note of it in this bug report.\nIf you want to test the update, you can install it with \n su -c 'yum --enablerepo=updates-testing update logwatch'. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2015-14203
logwatch-7.4.1-5.20150731svn293.fc22 has been pushed to the Fedora 22 stable repository. If problems still persist, please make note of it in this bug report.
The version logwatch-7.4.1-5.20150731svn293.fc22 is imperfect. It still outputs service names with extra "[": --------------------- fail2ban-messages Begin ------------------------ Banned services with Fail2Ban: Bans:Unbans [sshd: [ 17:1 ] ---------------------- fail2ban-messages End ------------------------- Its happens because of wrong regex in /usr/share/logwatch/scripts/services/fail2ban on line 87: m/(?:WARNING|NOTICE):?\s\[?(.*?)[]:]?\s(Ban|Unban)[^\.]* (\S+)/) it should be m/(?:WARNING|NOTICE):?\s+\[?(.*?)[]:]?\s(Ban|Unban)[^\.]* (\S+)/) The patch by Alan Hamilton does have that extra plus signg but for some reason it ended missing from logwatch-7.4.1-5.20150731svn293.fc22.