Red Hat Bugzilla – Bug 1295501
[spec] fail to start when exporting to remote server
Last modified: 2018-01-01 20:17:24 EST
Description of problem:
Trying to start systemd-journal-upload, it fail with the following error message:
janv. 04 17:23:54 freeipa.example.org systemd-journal-upload: Cannot create parent directory of state file /var/lib/systemd/journal-upload/state: Permission denied
The configuration file is rather standard:
Version-Release number of selected component (if applicable):
Steps to Reproduce:
1. setup systemd to export remotely the log
2. try to start it ( service systemd-journal-upload start )
it fail with the error message I gave
it should work.
It seems a directory /var/lib/systemd/journal-upload owned by systemd-journal-upload is missing. Creating it fix the problem.
as a side note, this is fixed by http://pkgs.fedoraproject.org/cgit/systemd.git/commit/?id=ffc19a0568815d02408a29a44f39ffd4b2a152e2 in Fedora.
As a workaround, run the following commands:
chown systemd-journal-upload: /var/lib/systemd/journal-upload
Seems to be fixed in F21+.
Are you sure it's fixed in Fedora? Tried to get this working reliably in Fedora 24 last week (all updates installed). Ran into the following issues and don't have it fully (reliably) working yet:
1) The state file didn't automatically get created and permissions needed to be set. Seeing the link you provided has fixes from end of 2015 I doubt they're included in Fedora 24 at least as I ran into the same thing.
2) When running as a service (ran fine as root) SElinux blocks systemd-journal-upload from TCP traffic and it crashes because it can't reach the remote thus. Error from systemd-journal-upload is very non-descriptive:
Mar 07 12:19:59 <hostname> systemd: Started Journal Remote Upload Service.
Mar 07 12:19:59 <hostname> systemd: systemd-journal-upload.service: Main process exited, code=exited, status=1/FAILURE
Mar 07 12:19:59 <hostname> systemd: systemd-journal-upload.service: Unit entered failed state.
Mar 07 12:19:59 <hostname> systemd: systemd-journal-upload.service: Failed with result 'exit-code'.
3) The service is very error prone. By default it's not configured to automatically restart and unlike what one (well I at least) would expect for a log upload service, which might be *very* important in many environments, the systemd-journal-upload process just dies if it can't reach the remote end. If it's down for a couple of minutes, doing updates, network issues, whatever, systemd-journal-upload just dies and that's it. Maybe you'll get your logs if it reboots (if they're persistent), maybe if some user pays attention and manually restarts it (hoping they aren't rotated yet), making it a very far from reliable service in the default setup.
Created attachment 1315172 [details]
Proposed spec file patch to create needed files
Change in spec file allows creation of vital files during installation.
Also the description of the journal-gatewayd package was extended by contents list.
it seems that the directory /var/lib/systemd/journal-upload is created automatically by tmpfiles.d (there exist a configuration file for that already in RHEL), but it needs to run tmpfiles.d daemon again (e.g. by rebooting).
I can imagine a scenario where a user tries to start the journal-upload daemon right after installation of package systemd-gateway, hence the directory is not created yet and the daemon fails.
To fix the problem, we are going to create the directory during installation of the package.
Meanwhile there is similar problem with systemd-journal-remote which demands more delicate solution and there shall be a new bugzilla opened for it.