Bug 1026804

(In reply to Richard Marko from comment #0)
> Description of problem:
> Some of the messages seem to appear only in journal and not in syslog.

This might even be a bug in rsyslog, not sure yet.

> Additional info:
> ForwardToSyslog=yes in /etc/systemd/journald.conf, log msg priority doesn't
> matter

These settings are useful for a generic syslog daemon but rsyslog doesn't need them because of...

> Related to bug 955118 ?

... this bug. I've just closed it, the change happened quite some time ago.

By default, rsyslog doesn't read from the socket (/run/systemd/journal/syslog) at all and directly calls  journal's api.

I'll try to reproduce the symptoms and perhaps change the component if the findings suggest that it's a problem on rsyslog's side.

If the data ends up in the journal, but rsyslog doesn't get it then this suggests a problem in rsyslog since rsyslog is nowadays pulling it rather than journald pushing it. Reassigning.

Is there any progress on this issue?

(In reply to Jakub Filak from comment #3)
> Is there any progress on this issue?

Actually there is. There's likely to be a fix for this soon, but no promises.

Here's a scratch build to test:
http://koji.fedoraproject.org/koji/taskinfo?taskID=6482262

In this version, _all_ messages from journald are pulled into rsyslog. I'm not yet sure we really want this behavior, which made me to think again about your report:

Why do you assume the messages should appear in syslog at all?

The forwarding mechanism works for messages written to /dev/log, but you are using journal's native API. What made you think these messages should go to syslog?

(In reply to Tomas Heinrich from comment #6)
> Here's a scratch build to test:
> http://koji.fedoraproject.org/koji/taskinfo?taskID=6482262
> 
> In this version, _all_ messages from journald are pulled into rsyslog. I'm
> not yet sure we really want this behavior, which made me to think again
> about your report:

Indeed, works fine.

> 
> Why do you assume the messages should appear in syslog at all?
> 
> The forwarding mechanism works for messages written to /dev/log, but you are
> using journal's native API. What made you think these messages should go to
> syslog?

Having to look for log messages in two places seems suboptimal to me. IIRC rsyslog is still required for sending logs over network so I would expect all logs to end in rsyslog if it's installed.

(In reply to Richard Marko from comment #7)
> (In reply to Tomas Heinrich from comment #6)
> > Here's a scratch build to test:
> > http://koji.fedoraproject.org/koji/taskinfo?taskID=6482262
> > 
> > In this version, _all_ messages from journald are pulled into rsyslog. I'm
> > not yet sure we really want this behavior, which made me to think again
> > about your report:
> 
> Indeed, works fine.

I've found a small error in my patch - the default facility should be set to a different value.

> > Why do you assume the messages should appear in syslog at all?
> > 
> > The forwarding mechanism works for messages written to /dev/log, but you are
> > using journal's native API. What made you think these messages should go to
> > syslog?
> 
> Having to look for log messages in two places seems suboptimal to me.

I agree.

> IIRC
> rsyslog is still required for sending logs over network so I would expect
> all logs to end in rsyslog if it's installed.

And all systemd logs (logged through syslog(3) and written to /dev/log) end up in rsyslog.

The message you send to journald can't be directly mapped to syslog as it has no facility and severity. Isn't possible for you to simply call syslog()? Then the message would end up in both journald and rsyslog.
Other possibility is to log a proper syslog message by filling the necessary fields, e.g.:
$ python -c "from systemd import journal; journal.send('fo0 msg', SYSLOG_FACILITY='11', PRIORITY='4')"

This is a borderline case. At the very least this behavior should be configurable but I'm not yet convinced this should be the default.

(In reply to Tomas Heinrich from comment #8)
> (In reply to Richard Marko from comment #7)
> > (In reply to Tomas Heinrich from comment #6)
> Other possibility is to log a proper syslog message by filling the necessary
> fields, e.g.:
> $ python -c "from systemd import journal; journal.send('fo0 msg',
> SYSLOG_FACILITY='11', PRIORITY='4')"
> 

This looks good, I want to use journal due to additional fields like CODE_FILE, _LINE, _FUNC.

Is SYSLOG_FACILITY='11' the right one for abrtd?

> This is a borderline case. At the very least this behavior should be
> configurable but I'm not yet convinced this should be the default.

I'm not fully aware of the consequences of enabling this by default so I'll leave this up to you. If the facility workaround is enough even if this is disabled by default then I don't really care but I guess more people will hit this sooner or later.

(In reply to Richard Marko from comment #9)
> Is SYSLOG_FACILITY='11' the right one for abrtd?

See syslog(3) and /usr/include/sys/syslog.h for the mapping of names of facilities/severities and numbers. For some reason, journald only allows numbers.

I'd probably go with user.notice, which is the default in logger(1), if you don't have any special needs. Numerically:
SYSLOG_FACILITY='1'
PRIORITY='5'

> > This is a borderline case. At the very least this behavior should be
> > configurable but I'm not yet convinced this should be the default.
> 
> I'm not fully aware of the consequences of enabling this by default so I'll
> leave this up to you. If the facility workaround is enough even if this is
> disabled by default then I don't really care but I guess more people will
> hit this sooner or later.

I'll bring this up for discussion. I concur that this will catch some people by surprise.

rsyslog-7.4.8-1.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/rsyslog-7.4.8-1.fc20

rsyslog-7.4.8-1.fc20 has been pushed to the Fedora 20 stable repository.  If problems still persist, please make note of it in this bug report.