|Summary:||RFE: Logwatch parse Journal|
|Product:||[Fedora] Fedora||Reporter:||Frank Murphy <frankly3d>|
|Component:||logwatch||Assignee:||Jan Synacek <jsynacek>|
|Status:||CLOSED RAWHIDE||QA Contact:||Fedora Extras Quality Assurance <extras-qa>|
|Version:||rawhide||CC:||amessina, andrew.kavalov, bugzilla.redhat, covex, dominik, frank, gbcox, glandvador, goeran, herrold, iav, jrd-rhbz, jsynacek, mattdm, neil, ol+redhat, paul.destefano-redhat2, richardfearn, sdmau63ndf, tibbs, twaugh, uckelman, vchelban, zbyszek|
|Fixed In Version:||logwatch-7.5.0-1.fc30||Doc Type:||Bug Fix|
|Doc Text:||Story Points:||---|
|Last Closed:||2019-01-04 11:22:27 UTC||Type:||Bug|
|oVirt Team:||---||RHEL 7.3 requirements from Atomic Host:|
Description Frank Murphy 2012-10-10 10:27:00 UTC
Description of problem: "As far as I know logwatch has not been patched to parse and use journal. Try filing an RFE against logwatch for that JBG " https://lists.fedoraproject.org/pipermail/devel/2012-October/172445.html
Comment 1 Jan Synacek 2012-11-28 13:09:17 UTC
AFAIK journal uses its own binary format. Do any Perl wrappers for journal's C API exist? Logwatch could probably call 'journalctl -o export' and then process the export, but that's ugly and would be slow on large logs.
Comment 2 Lennart Poettering 2013-07-15 11:27:37 UTC
Wherever you use fopen("/var/log/messages", "r") you can pretty much instead of use popen("journalctl", "r"). It will provide you with the same text output, in the same formatting.
Comment 3 Thomas Jürges 2013-08-19 11:51:38 UTC
If I am correct in interpreting your reply, Lennart, then no Perl wrappers exist. Is that a correct interpretation? And instead of simply fopening a file you propose to start a new process to read the contents of the log? Sounds like taking a sledgehammer to crack a nut to me. A simple fopen seems to me to be pretty basic and error handling is quite simple. What happened to the "KISS" priciple?
Comment 4 Zbigniew Jędrzejewski-Szmek 2013-10-11 12:52:26 UTC
(In reply to Thomas Jürges from comment #3) > If I am correct in interpreting your reply, Lennart, then no Perl wrappers > exist. Is that a correct interpretation? Yes. > And instead of simply fopening a file you propose to start a new process to > read the contents of the log? Sounds like taking a sledgehammer to crack a > nut to me. A simple fopen seems to me to be pretty basic and error handling > is quite simple. What happened to the "KISS" priciple? popen() also has easy error handling :) Any progress here?
Comment 5 Adam Pribyl 2014-04-04 12:53:40 UTC
There is a patch to logwatch now: http://sourceforge.net/p/logwatch/patches/34/ but maybe we can have systemd-journald-logwatchd?
Comment 6 Jan Synacek 2014-09-24 06:29:22 UTC
After additional thought, it really makes no sense to extend logwatch to parse journal. The clean and, IMHO, right solution is to write a new tool that preferably uses journal bindings to process the journal.
Comment 7 R P Herrold 2014-09-24 19:26:19 UTC
Jan Synacek as you WONTFIX within logwatch, because you personally deem another approach (systemd / journal) preferable, would you consider transferring maintainership of Logwatch, to me or another willing to make the requested chances within the Logwatch package?
Comment 8 Jan Synacek 2014-09-25 07:33:55 UTC
R P Herrold there is nothing holding you back from writing a patch, proposing it upstream and then posting it here for inclusion.
Comment 9 R P Herrold 2014-09-25 16:13:52 UTC
Comment 10 Jan Synacek 2014-09-26 05:52:16 UTC
(In reply to R P Herrold from comment #9) > What is the objection to it? Believe it or not, I completely forgot about it, even though it was written one comment above mine when I wontfixed this bug. I'm sorry for that (sleep deprived?). My only objection to it is that it's not merged upstream yet, even though it's been amended and tested. If you really want to take responsibility and apply it in Fedora, apply for the commit privileges and I'll make you a comaintainer.
Comment 11 Frank Crawford 2014-09-27 12:10:54 UTC
It is also worth noting that upstream appears to be interested in merging in some patches to handle systemd journals.
Comment 12 R P Herrold 2014-09-29 20:26:56 UTC
My fedoraproject ID is: herrold I presently serve as a co-maintainer on another package, and am in 'packager' group already I am uncertain what additional Fedora group (if any) you are seeking to have me in. Please advise
Comment 13 Jan Synacek 2014-09-30 06:50:03 UTC
After login, you should be able to apply for commit privileges at https://admin.fedoraproject.org/pkgdb/package/logwatch/.
Comment 14 Jason Tibbitts 2015-03-26 18:46:44 UTC
I wonder what's happening here. denyhosts is switching to logging to the journal by default in F22, which means that people will stop getting denyhosts info in their logwatch output. I don't want to switch away from that but on the other hand people might complain about not getting the report. Not sure if there's any way out other than just implementing some kind of journal interface.
Comment 15 Matthew Miller 2015-03-26 19:38:21 UTC
Jason, I don't think that's necessarily a problem — denyhost may provide journal logs but if you've got rsyslogd installed to get traditional text logs, the output should appear there as well, and logwatch will work as normal.
Comment 16 Jason Tibbitts 2015-03-26 20:06:27 UTC
Actually, that's not the case. Denyhosts never logged to syslog by default. It wrote directly to /var/log/logwatch. Now it logs to the journal (optionally, but on Fedora I'd like to make it the default). I suppose with a syslog daemon the logs will show up in /var/log/messages but I do not believe that logwatch looks there. And of course the whole reason I implemented journal access for denyhosts was to get away from the requirement that the user install a syslog daemon.
Comment 17 Tim Waugh 2015-10-05 16:03:56 UTC
FWIW, I've written a small utility along the same lines: https://github.com/twaugh/journal-brief
Comment 18 Paul DeStefano 2016-06-27 07:37:17 UTC
I think it was Waugh who linked to this from another forum, but this feature has been added upstream, albeit experimentally. https://sourceforge.net/p/logwatch/patches/34/ It should also be noted, however, that journal-brief is packaged in Fedora, now.
Comment 19 Jan Synacek 2016-11-30 08:11:11 UTC
Comment 20 Fedora Update System 2016-11-30 08:12:21 UTC
logwatch-7.4.3-3.fc25 has been submitted as an update to Fedora 25. https://bodhi.fedoraproject.org/updates/FEDORA-2016-580c613e16
Comment 21 Dominik 'Rathann' Mierzejewski 2016-11-30 11:56:38 UTC
Please backport to F24 as well.
Comment 22 Fedora Update System 2016-11-30 14:05:26 UTC
logwatch-7.4.3-3.fc24 has been submitted as an update to Fedora 24. https://bodhi.fedoraproject.org/updates/FEDORA-2016-94ea5244ba
Comment 23 Fedora Update System 2016-12-03 04:32:27 UTC
logwatch-7.4.3-3.fc25 has been pushed to the Fedora 25 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-580c613e16
Comment 24 Fedora Update System 2016-12-03 05:40:24 UTC
logwatch-7.4.3-3.fc24 has been pushed to the Fedora 24 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-94ea5244ba
Comment 25 Fedora Update System 2016-12-12 23:56:35 UTC
logwatch-7.4.3-3.fc25 has been pushed to the Fedora 25 stable repository. If problems still persist, please make note of it in this bug report.
Comment 26 Fedora Update System 2016-12-13 02:24:58 UTC
logwatch-7.4.3-3.fc24 has been pushed to the Fedora 24 stable repository. If problems still persist, please make note of it in this bug report.
Comment 27 Gerald Cox 2017-07-11 20:18:44 UTC
Reopening...as I mentioned on the developer list, if you are required to do a new install of F25, (which means reformatting partition, etc.) logwatch doesn't work. Since I have been doing upgrades for many years, I suppose that is why I haven't had any issue until now. I still had the old log format on my system, and from the comments it appears that logwatch is suppose to support the journal now. Is there a runtime requisite missing from the rpm?
Comment 28 Göran Uddeborg 2017-07-11 20:27:24 UTC
As I understand it, the dependencies are fine. Logwatch by default looks in various files in /var/log. On a Fedora system it should instead run various journalctl commands to get the corresponding information. The current logwatch version is able to do that. What remains is to add configuration files overriding any entries in /usr/share/logwatch/default.conf/logfiles for all log files that have been replaced by the journal. That is probably most of them, but not quite all. Httpd, for example, still logs to files in /var/log/httpd. The "proper" place to put those overrides would be /usr/share/logwatch/dist.conf/logfiles. It is a bit of work to figure out the correct override in each case, and it is not always trivial. I'm working on overrides on my own systems, and there are still some expected sections missing from the reports I get.
Comment 29 Gerald Cox 2017-07-11 20:52:31 UTC
Thanks Göran - I just found this: https://bbs.archlinux.org/viewtopic.php?id=227516 I'll try out it's instructions, but I find interesting what the author stated, which is: "Ever since systemd-journald replaced rsyslog - the logwatch package had more or less become useless.... Recently logwatch version 7.4.3-3 was released with support for journalctl ... This version however does not implement conf file changes. So default logwatch still remains useless." Here: https://fedoraproject.org/wiki/Changes/NoDefaultSyslog Logwatch was listed as a dependency... and if one would read that announcement one would come away with the impression that it would be taken care of... AND FOUR YEARS LATER it still doesn't work. If you're lucky enough to have been doing system upgrades you're fine, but as soon as you do a re-install, forget it. Users shouldn't have to go through contortions to get software that is posted in the production repositories to work.
Comment 30 Göran Uddeborg 2017-07-11 21:14:39 UTC
The difference between upgrades and re-installs is most likely if you run rsyslog or not. If you after doing the re-installation also do "dnf install rsyslog" and "systemctl enable rsyslog", you still get all the log files, and logwatch would behave as before. I agree, though, this ought to be provided, since rsyslog is no longer installed by default. Someone "just" has to do the work. The Arch page seems to be a good start.
Comment 31 Gerald Cox 2017-07-11 21:30:15 UTC
Yes, that is why I asked about run time requisites. If rsyslog is still needed for logwatch, then it should be installed if you install logwatch. If using the journal is the direction Fedora wants to go, then someone should work with upstream to get it done, and in the meantime add the required files to the rpm. Just doing a ¯\_(ツ)_/¯ isn't appropriate - especially for four years. I ran a test using the info from Arch, and that seems to be working.
Comment 32 Göran Uddeborg 2017-08-22 19:37:52 UTC
Created attachment 1316775 [details] Some logwatch configurations to read from the journal. Hello again, For anyone interested, I attach a set of configurations I've created to make logwatch read from the journal. It is probably only a start; it fixes things for the services I run on my systems. As I understand it, files like these could go into /usr/share/logwatch/dist.conf Three files in the attachment require some explanation: /etc/logwatch/scripts/logfiles/cron/applydate overrides the default by changing the time pattern. The leading zero on the date is different between the cron log files, and the output from journalctl. This change will not be backward compatible with reading from the /var/log/cron file. /etc/logwatch/conf/empty is there because logwatch is designed so it needs SOME actual file to read. Processing overhead will be minimized if it is an empty file. /dev/null isn't good enough, it seems a regular file is needed. /etc/logwatch/conf/logfiles/mysql.conf doesn't have to do with journaling, but the change from MySQL to MariaDB in Fedora. The log files changed place with it. While not really related to this bugzilla, I thought it was worth mentioning while adding those files.
Comment 33 Anthony Messina 2017-08-24 23:25:42 UTC
(In reply to Göran Uddeborg from comment #32) > Created attachment 1316775 [details] > Some logwatch configurations to read from the journal. > > Hello again, > > For anyone interested, I attach a set of configurations I've created to make > logwatch read from the journal. It is probably only a start; it fixes > things for the services I run on my systems. > > As I understand it, files like these could go into > /usr/share/logwatch/dist.conf > > Three files in the attachment require some explanation: > > /etc/logwatch/scripts/logfiles/cron/applydate overrides the default by > changing the time pattern. The leading zero on the date is different > between the cron log files, and the output from journalctl. This change > will not be backward compatible with reading from the /var/log/cron file. > > /etc/logwatch/conf/empty is there because logwatch is designed so it needs > SOME actual file to read. Processing overhead will be minimized if it is an > empty file. /dev/null isn't good enough, it seems a regular file is needed. > > /etc/logwatch/conf/logfiles/mysql.conf doesn't have to do with journaling, > but the change from MySQL to MariaDB in Fedora. The log files changed place > with it. While not really related to this bugzilla, I thought it was worth > mentioning while adding those files. Thanks for getting it started. I had been trying a different approach with moderate (and incomplete) success... cat /etc/logwatch/conf/override.conf # Configuration overrides for specific logfiles/services may be placed here. # Logfiles logfiles/cron: Archive = logfiles/cron: LogFile = /dev/null logfiles/cron: *JournalCtl = "--output=short _COMM=crond" logfiles/kernel: Archive = logfiles/kernel: LogFile = /dev/null logfiles/kernel: *JournalCtl = "--output=short _TRANSPORT=kernel" logfiles/maillog: Archive = logfiles/maillog: LogFile = /dev/null logfiles/maillog: *JournalCtl = "--output=short" logfiles/messages: Archive = logfiles/messages: LogFile = /dev/null logfiles/messages: *JournalCtl = "--output=short" logfiles/secure: Archive = logfiles/secure: LogFile = /dev/null logfiles/secure: *JournalCtl = "--output=short SYSLOG_FACILITY=10" # Services services/iptables: LogFile = services/iptables: LogFile = kernel services/kernel: LogFile = services/kernel: LogFile = kernel