Bug 1026804 - journald syslog forwarding doesn't work
Summary: journald syslog forwarding doesn't work
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: rsyslog
Version: 20
Hardware: All
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Tomas Heinrich
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: 1038136
TreeView+ depends on / blocked
 
Reported: 2013-11-05 13:24 UTC by Richard Marko
Modified: 2016-09-20 04:53 UTC (History)
16 users (show)

Fixed In Version: rsyslog-7.4.8-1.fc20
Clone Of:
: 1038136 (view as bug list)
Environment:
Last Closed: 2014-02-24 12:35:33 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Richard Marko 2013-11-05 13:24:06 UTC
Description of problem:
Some of the messages seem to appear only in journal and not in syslog.

Version-Release number of selected component (if applicable):
systemd-208-4.fc20.x86_64

How reproducible:
Always

Steps to Reproduce:
0. yum -y install systemd-python
1. python -c 'from systemd import journal; journal.send("vv")'
2. grep vv /var/log/messages
3. journalctl -n 20 | grep vv

Actual results:
vv message is visible only in journalctl output

Expected results:
vv message present in both places

Additional info:
ForwardToSyslog=yes in /etc/systemd/journald.conf, log msg priority doesn't matter

Related to bug 955118 ?

Comment 1 Tomas Heinrich 2013-12-04 15:59:46 UTC
(In reply to Richard Marko from comment #0)
> Description of problem:
> Some of the messages seem to appear only in journal and not in syslog.

This might even be a bug in rsyslog, not sure yet.

> Additional info:
> ForwardToSyslog=yes in /etc/systemd/journald.conf, log msg priority doesn't
> matter

These settings are useful for a generic syslog daemon but rsyslog doesn't need them because of...

> Related to bug 955118 ?

... this bug. I've just closed it, the change happened quite some time ago.

By default, rsyslog doesn't read from the socket (/run/systemd/journal/syslog) at all and directly calls  journal's api.

I'll try to reproduce the symptoms and perhaps change the component if the findings suggest that it's a problem on rsyslog's side.

Comment 2 Lennart Poettering 2013-12-13 02:06:01 UTC
If the data ends up in the journal, but rsyslog doesn't get it then this suggests a problem in rsyslog since rsyslog is nowadays pulling it rather than journald pushing it. Reassigning.

Comment 3 Jakub Filak 2014-01-23 10:36:28 UTC
Is there any progress on this issue?

Comment 5 Tomas Heinrich 2014-01-23 10:58:48 UTC
(In reply to Jakub Filak from comment #3)
> Is there any progress on this issue?

Actually there is. There's likely to be a fix for this soon, but no promises.

Comment 6 Tomas Heinrich 2014-02-02 22:31:47 UTC
Here's a scratch build to test:
http://koji.fedoraproject.org/koji/taskinfo?taskID=6482262

In this version, _all_ messages from journald are pulled into rsyslog. I'm not yet sure we really want this behavior, which made me to think again about your report:

Why do you assume the messages should appear in syslog at all?

The forwarding mechanism works for messages written to /dev/log, but you are using journal's native API. What made you think these messages should go to syslog?

Comment 7 Richard Marko 2014-02-03 11:53:45 UTC
(In reply to Tomas Heinrich from comment #6)
> Here's a scratch build to test:
> http://koji.fedoraproject.org/koji/taskinfo?taskID=6482262
> 
> In this version, _all_ messages from journald are pulled into rsyslog. I'm
> not yet sure we really want this behavior, which made me to think again
> about your report:

Indeed, works fine.

> 
> Why do you assume the messages should appear in syslog at all?
> 
> The forwarding mechanism works for messages written to /dev/log, but you are
> using journal's native API. What made you think these messages should go to
> syslog?

Having to look for log messages in two places seems suboptimal to me. IIRC rsyslog is still required for sending logs over network so I would expect all logs to end in rsyslog if it's installed.

Comment 8 Tomas Heinrich 2014-02-03 12:37:59 UTC
(In reply to Richard Marko from comment #7)
> (In reply to Tomas Heinrich from comment #6)
> > Here's a scratch build to test:
> > http://koji.fedoraproject.org/koji/taskinfo?taskID=6482262
> > 
> > In this version, _all_ messages from journald are pulled into rsyslog. I'm
> > not yet sure we really want this behavior, which made me to think again
> > about your report:
> 
> Indeed, works fine.

I've found a small error in my patch - the default facility should be set to a different value.

> > Why do you assume the messages should appear in syslog at all?
> > 
> > The forwarding mechanism works for messages written to /dev/log, but you are
> > using journal's native API. What made you think these messages should go to
> > syslog?
> 
> Having to look for log messages in two places seems suboptimal to me.

I agree.

> IIRC
> rsyslog is still required for sending logs over network so I would expect
> all logs to end in rsyslog if it's installed.

And all systemd logs (logged through syslog(3) and written to /dev/log) end up in rsyslog.

The message you send to journald can't be directly mapped to syslog as it has no facility and severity. Isn't possible for you to simply call syslog()? Then the message would end up in both journald and rsyslog.
Other possibility is to log a proper syslog message by filling the necessary fields, e.g.:
$ python -c "from systemd import journal; journal.send('fo0 msg', SYSLOG_FACILITY='11', PRIORITY='4')"

This is a borderline case. At the very least this behavior should be configurable but I'm not yet convinced this should be the default.

Comment 9 Richard Marko 2014-02-03 13:11:36 UTC
(In reply to Tomas Heinrich from comment #8)
> (In reply to Richard Marko from comment #7)
> > (In reply to Tomas Heinrich from comment #6)
> Other possibility is to log a proper syslog message by filling the necessary
> fields, e.g.:
> $ python -c "from systemd import journal; journal.send('fo0 msg',
> SYSLOG_FACILITY='11', PRIORITY='4')"
> 

This looks good, I want to use journal due to additional fields like CODE_FILE, _LINE, _FUNC.

Is SYSLOG_FACILITY='11' the right one for abrtd?

> This is a borderline case. At the very least this behavior should be
> configurable but I'm not yet convinced this should be the default.

I'm not fully aware of the consequences of enabling this by default so I'll leave this up to you. If the facility workaround is enough even if this is disabled by default then I don't really care but I guess more people will hit this sooner or later.

Comment 10 Tomas Heinrich 2014-02-03 13:30:06 UTC
(In reply to Richard Marko from comment #9)
> Is SYSLOG_FACILITY='11' the right one for abrtd?

See syslog(3) and /usr/include/sys/syslog.h for the mapping of names of facilities/severities and numbers. For some reason, journald only allows numbers.

I'd probably go with user.notice, which is the default in logger(1), if you don't have any special needs. Numerically:
SYSLOG_FACILITY='1'
PRIORITY='5'

> > This is a borderline case. At the very least this behavior should be
> > configurable but I'm not yet convinced this should be the default.
> 
> I'm not fully aware of the consequences of enabling this by default so I'll
> leave this up to you. If the facility workaround is enough even if this is
> disabled by default then I don't really care but I guess more people will
> hit this sooner or later.

I'll bring this up for discussion. I concur that this will catch some people by surprise.

Comment 11 Fedora Update System 2014-02-23 17:52:06 UTC
rsyslog-7.4.8-1.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/rsyslog-7.4.8-1.fc20

Comment 12 Fedora Update System 2014-02-24 12:35:33 UTC
rsyslog-7.4.8-1.fc20 has been pushed to the Fedora 20 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.