Bug 864872 - RFE: Logwatch parse Journal
RFE: Logwatch parse Journal
Status: NEW
Product: Fedora
Classification: Fedora
Component: logwatch (Show other bugs)
rawhide
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Jan Synacek
Fedora Extras Quality Assurance
: Reopened
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-10-10 06:27 EDT by Frank Murphy
Modified: 2017-07-11 17:30 EDT (History)
25 users (show)

See Also:
Fixed In Version: logwatch-7.4.3-3.fc26 logwatch-7.4.3-3.fc25 logwatch-7.4.3-3.fc24
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-12-12 18:56:35 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Frank Murphy 2012-10-10 06:27:00 EDT
Description of problem: 


"As far as I know logwatch has not been patched to parse and use journal.

Try filing an RFE against logwatch for that

JBG "

https://lists.fedoraproject.org/pipermail/devel/2012-October/172445.html
Comment 1 Jan Synacek 2012-11-28 08:09:17 EST
AFAIK journal uses its own binary format. Do any Perl wrappers for journal's C API exist? Logwatch could probably call 'journalctl -o export' and then process the export, but that's ugly and would be slow on large logs.
Comment 2 Lennart Poettering 2013-07-15 07:27:37 EDT
Wherever you use fopen("/var/log/messages", "r") you can pretty much instead of use popen("journalctl", "r"). It will provide you with the same text output, in the same formatting.
Comment 3 Thomas Jürges 2013-08-19 07:51:38 EDT
If I am correct in interpreting your reply, Lennart, then no Perl wrappers exist.  Is that a correct interpretation?

And instead of simply fopening a file you propose to start a new process to read the contents of the log?  Sounds like taking a sledgehammer to crack a nut to me.  A simple fopen seems to me to be pretty basic and error handling is quite simple.  What happened to the "KISS" priciple?
Comment 4 Zbigniew Jędrzejewski-Szmek 2013-10-11 08:52:26 EDT
(In reply to Thomas Jürges from comment #3)
> If I am correct in interpreting your reply, Lennart, then no Perl wrappers
> exist.  Is that a correct interpretation?
Yes.

> And instead of simply fopening a file you propose to start a new process to
> read the contents of the log?  Sounds like taking a sledgehammer to crack a
> nut to me.  A simple fopen seems to me to be pretty basic and error handling
> is quite simple.  What happened to the "KISS" priciple?
popen() also has easy error handling :)

Any progress here?
Comment 5 Adam Pribyl 2014-04-04 08:53:40 EDT
There is a patch to logwatch now:

http://sourceforge.net/p/logwatch/patches/34/

but maybe we can have systemd-journald-logwatchd?
Comment 6 Jan Synacek 2014-09-24 02:29:22 EDT
After additional thought, it really makes no sense to extend logwatch to parse journal. The clean and, IMHO, right solution is to write a new tool that preferably uses journal bindings to process the journal.
Comment 7 R P Herrold 2014-09-24 15:26:19 EDT
Jan Synacek

as you WONTFIX within logwatch, because you personally deem another approach (systemd / journal) preferable, would you consider transferring maintainership of Logwatch, to me or another willing to make the requested chances within the Logwatch package?
Comment 8 Jan Synacek 2014-09-25 03:33:55 EDT
R P Herrold

there is nothing holding you back from writing a patch, proposing it upstream and then posting it here for inclusion.
Comment 9 R P Herrold 2014-09-25 12:13:52 EDT
Comment 5 mentions just such a patch which you seek in comment 8 and was not responded to

It appears to be tested and was amended

What is the objection to it?
Comment 10 Jan Synacek 2014-09-26 01:52:16 EDT
(In reply to R P Herrold from comment #9)
> What is the objection to it?

Believe it or not, I completely forgot about it, even though it was written one comment above mine when I wontfixed this bug. I'm sorry for that (sleep deprived?).

My only objection to it is that it's not merged upstream yet, even though it's been amended and tested. If you really want to take responsibility and apply it in Fedora, apply for the commit privileges and I'll make you a comaintainer.
Comment 11 Frank Crawford 2014-09-27 08:10:54 EDT
It is also worth noting that upstream appears to be interested in merging in some patches to handle systemd journals.
Comment 12 R P Herrold 2014-09-29 16:26:56 EDT
My fedoraproject ID is: herrold

I presently serve as a co-maintainer on another package, and am in 'packager'  group already
   
I am uncertain what additional Fedora group (if any) you are seeking to have me in.  Please advise
Comment 13 Jan Synacek 2014-09-30 02:50:03 EDT
After login, you should be able to apply for commit privileges at https://admin.fedoraproject.org/pkgdb/package/logwatch/.
Comment 14 Jason Tibbitts 2015-03-26 14:46:44 EDT
I wonder what's happening here.

denyhosts is switching to logging to the journal by default in F22, which means that people will stop getting denyhosts info in their logwatch output.  I don't want to switch away from that but on the other hand people might complain about not getting the report.  Not sure if there's any way out other than just implementing some kind of journal interface.
Comment 15 Matthew Miller 2015-03-26 15:38:21 EDT
Jason, I don't think that's necessarily a problem — denyhost may provide journal logs but if you've got rsyslogd installed to get traditional text logs, the output should appear there as well, and logwatch will work as normal.
Comment 16 Jason Tibbitts 2015-03-26 16:06:27 EDT
Actually, that's not the case.

Denyhosts never logged to syslog by default.  It wrote directly to /var/log/logwatch.  Now it logs to the journal (optionally, but on Fedora I'd like to make it the default).  I suppose with a syslog daemon the logs will show up in /var/log/messages but I do not believe that logwatch looks there.  And of course the whole reason I implemented journal access for denyhosts was to get away from the requirement that the user install a syslog daemon.
Comment 17 Tim Waugh 2015-10-05 12:03:56 EDT
FWIW, I've written a small utility along the same lines:
  https://github.com/twaugh/journal-brief
Comment 18 Paul DeStefano 2016-06-27 03:37:17 EDT
I think it was Waugh who linked to this from another forum, but this feature has been added upstream, albeit experimentally.

https://sourceforge.net/p/logwatch/patches/34/

It should also be noted, however, that journal-brief is packaged in Fedora, now.
Comment 20 Fedora Update System 2016-11-30 03:12:21 EST
logwatch-7.4.3-3.fc25 has been submitted as an update to Fedora 25. https://bodhi.fedoraproject.org/updates/FEDORA-2016-580c613e16
Comment 21 Dominik 'Rathann' Mierzejewski 2016-11-30 06:56:38 EST
Please backport to F24 as well.
Comment 22 Fedora Update System 2016-11-30 09:05:26 EST
logwatch-7.4.3-3.fc24 has been submitted as an update to Fedora 24. https://bodhi.fedoraproject.org/updates/FEDORA-2016-94ea5244ba
Comment 23 Fedora Update System 2016-12-02 23:32:27 EST
logwatch-7.4.3-3.fc25 has been pushed to the Fedora 25 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-580c613e16
Comment 24 Fedora Update System 2016-12-03 00:40:24 EST
logwatch-7.4.3-3.fc24 has been pushed to the Fedora 24 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-94ea5244ba
Comment 25 Fedora Update System 2016-12-12 18:56:35 EST
logwatch-7.4.3-3.fc25 has been pushed to the Fedora 25 stable repository. If problems still persist, please make note of it in this bug report.
Comment 26 Fedora Update System 2016-12-12 21:24:58 EST
logwatch-7.4.3-3.fc24 has been pushed to the Fedora 24 stable repository. If problems still persist, please make note of it in this bug report.
Comment 27 Gerald Cox 2017-07-11 16:18:44 EDT
Reopening...as I mentioned on the developer list, if you are required to do a new install of F25, (which means reformatting partition, etc.) logwatch doesn't work.  Since I have been doing upgrades for many years, I suppose that is why I haven't had any issue until now.  I still had the old log format on my system, and from the comments it appears that logwatch is suppose to support the journal now.  

Is there a runtime requisite missing from the rpm?
Comment 28 Göran Uddeborg 2017-07-11 16:27:24 EDT
As I understand it, the dependencies are fine.

Logwatch by default looks in various files in /var/log.  On a Fedora system it should instead run various journalctl commands to get the corresponding information.  The current logwatch version is able to do that.  What remains is to add configuration files overriding any entries in /usr/share/logwatch/default.conf/logfiles for all log files that have been replaced by the journal.  That is probably most of them, but not quite all.  Httpd, for example, still logs to files in /var/log/httpd.

The "proper" place to put those overrides would be /usr/share/logwatch/dist.conf/logfiles.  It is a bit of work to figure out the correct override in each case, and it is not always trivial.  I'm working on overrides on my own systems, and there are still some expected sections missing from the reports I get.
Comment 29 Gerald Cox 2017-07-11 16:52:31 EDT
Thanks Göran - I just found this:

https://bbs.archlinux.org/viewtopic.php?id=227516

I'll try out it's instructions, but I find interesting what the author stated, which is:
"Ever since systemd-journald replaced rsyslog - the logwatch package had more or less become useless....
Recently logwatch version 7.4.3-3 was released with support for journalctl
...
This version however does not implement conf file changes. So default logwatch still remains useless."

Here:
https://fedoraproject.org/wiki/Changes/NoDefaultSyslog

Logwatch was listed as a dependency... and if one would read that announcement one would come away with the impression that it would be taken care of... AND FOUR YEARS LATER it still doesn't work.  

If you're lucky enough to have been doing system upgrades you're fine, but as soon as you do a re-install, forget it.  

Users shouldn't have to go through contortions to get software that is posted in the production repositories to work.
Comment 30 Göran Uddeborg 2017-07-11 17:14:39 EDT
The difference between upgrades and re-installs is most likely if you run rsyslog or not.  If you after doing the re-installation also do "dnf install rsyslog" and "systemctl enable rsyslog", you still get all the log files, and logwatch would behave as before.

I agree, though, this ought to be provided, since rsyslog is no longer installed by default.  Someone "just" has to do the work.  The Arch page seems to be a good start.
Comment 31 Gerald Cox 2017-07-11 17:30:15 EDT
Yes, that is why I asked about run time requisites.  If rsyslog is still needed for logwatch, then it should be installed if you install logwatch.  If using the journal is the direction Fedora wants to go, then someone should work with upstream to get it done, and in the meantime add the required files to the rpm.    

Just doing a ¯\_(ツ)_/¯ isn't appropriate - especially for four years.  

I ran a test using the info from Arch, and that seems to be working.

Note You need to log in before you can comment on or make changes to this bug.