Bug 1295501 - [spec] fail to start when exporting to remote server
[spec] fail to start when exporting to remote server
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: systemd (Show other bugs)
Unspecified Unspecified
medium Severity medium
: rc
: ---
Assigned To: systemd-maint
Radka Skvarilova
: Patch
Depends On:
  Show dependency treegraph
Reported: 2016-01-04 12:36 EST by M. Scherer
Modified: 2018-02-23 10:17 EST (History)
8 users (show)

See Also:
Fixed In Version: systemd-219-50.el7
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2018-02-23 10:17:28 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
Proposed spec file patch to create needed files (2.37 KB, patch)
2017-08-18 07:33 EDT, Jan Rybar
no flags Details | Diff

External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 2778121 None None None 2016-11-23 09:36 EST
Red Hat Bugzilla 1193143 None None None 2016-12-02 09:11 EST

  None (edit)
Description M. Scherer 2016-01-04 12:36:00 EST
Description of problem:
Trying to start systemd-journal-upload, it fail with the following error message:

janv. 04 17:23:54 freeipa.example.org systemd-journal-upload[22353]: Cannot create parent directory of state file /var/lib/systemd/journal-upload/state: Permission denied

The configuration file is rather standard:

$cat /etc/systemd/journal-upload.conf

Version-Release number of selected component (if applicable):

How reproducible:
each time

Steps to Reproduce:
1. setup systemd to export remotely the log
2. try to start it ( service systemd-journal-upload start )

Actual results:
it fail with the error message I gave

Expected results:
it should work.

Additional info:

It seems a directory /var/lib/systemd/journal-upload owned by  systemd-journal-upload is missing. Creating it fix the problem.
Comment 2 M. Scherer 2016-01-04 12:46:48 EST
as a side note, this is fixed by http://pkgs.fedoraproject.org/cgit/systemd.git/commit/?id=ffc19a0568815d02408a29a44f39ffd4b2a152e2 in Fedora.
Comment 4 Zdenek Pytela 2016-12-02 09:11:59 EST
As a workaround, run the following commands:

mkdir /var/lib/systemd/journal-upload
chown systemd-journal-upload: /var/lib/systemd/journal-upload

Seems to be fixed in F21+.
Comment 5 Ferry 2017-03-07 06:42:39 EST
Are you sure it's fixed in Fedora? Tried to get this working reliably in Fedora 24 last week (all updates installed). Ran into the following issues and don't have it fully (reliably) working yet:

1) The state file didn't automatically get created and permissions needed to be set. Seeing the link you provided has fixes from end of 2015 I doubt they're included in Fedora 24 at least as I ran into the same thing.

2) When running as a service (ran fine as root) SElinux blocks systemd-journal-upload from TCP traffic and it crashes because it can't reach the remote thus. Error from systemd-journal-upload is very non-descriptive:

Mar 07 12:19:59 <hostname> systemd[1]: Started Journal Remote Upload Service.
Mar 07 12:19:59 <hostname> systemd[1]: systemd-journal-upload.service: Main process exited, code=exited, status=1/FAILURE
Mar 07 12:19:59 <hostname> systemd[1]: systemd-journal-upload.service: Unit entered failed state.
Mar 07 12:19:59 <hostname> systemd[1]: systemd-journal-upload.service: Failed with result 'exit-code'.

3) The service is very error prone. By default it's not configured to automatically restart and unlike what one (well I at least) would expect for a log upload service, which might be *very* important in many environments, the systemd-journal-upload process just dies if it can't reach the remote end. If it's down for a couple of minutes, doing updates, network issues, whatever, systemd-journal-upload just dies and that's it. Maybe you'll get your logs if it reboots (if they're persistent), maybe if some user pays attention and manually restarts it (hoping they aren't rotated yet), making it a very far from reliable service in the default setup.
Comment 7 Jan Rybar 2017-08-18 07:33 EDT
Created attachment 1315172 [details]
Proposed spec file patch to create needed files

Change in spec file allows creation of vital files during installation.
Also the description of the journal-gatewayd package was extended by contents list.
Comment 8 Jan Rybar 2017-12-04 07:05:08 EST

it seems that the directory /var/lib/systemd/journal-upload is created automatically by tmpfiles.d (there exist a configuration file for that already in RHEL), but it needs to run tmpfiles.d daemon again (e.g. by rebooting).
I can imagine a scenario where a user tries to start the journal-upload daemon right after installation of package systemd-gateway, hence the directory is not created yet and the daemon fails.

To fix the problem, we are going to create the directory during installation of the package.

Meanwhile there is similar problem with systemd-journal-remote which demands more delicate solution and there shall be a new bugzilla opened for it.
Comment 9 Lukáš Nykrýn 2018-02-08 08:45:18 EST
I forget about this one and now there are only blocker bugs accepted for 7.5. SO let's move it to 7.6.
Comment 10 Lukáš Nykrýn 2018-02-23 10:17:28 EST
Actually, I did not forget about it and the fix is there since December.

Note You need to log in before you can comment on or make changes to this bug.