Bug 1388049

Summary: Job for systemd-journald.service failed because a fatal signal was delivered causing the control process to dump core
Product: Red Hat Enterprise Linux 7 Reporter: muahao <muahao>
Component: systemdAssignee: systemd-maint
Status: CLOSED DUPLICATE QA Contact: qe-baseos-daemons
Severity: medium Docs Contact:
Priority: unspecified    
Version: 7.2CC: dtardon, nduffy, systemd-maint-list
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-02-01 14:38:09 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description muahao 2016-10-24 10:10:56 UTC 
Description of problem:

Hello ,I got a problem,I want to modify systemd-journald's behavior from auto to persistent,because journal log file used my memory already 4G.

But I found, when I modify /etc/systemd/journal.conf from "auto" to "pesistent", and restart systemd-journald,and the directory /run/log/journald was still exist! and use my memory 4G still, and the directory /var/log/journald was made ,and journal log file drop into /var/log/journald with expectation;

But when I restart systemd-journald some errors occured:

```
[root@muahao:/home/ahao.mah]
#systemctl restart systemd-journald
Job for systemd-journald.service failed because a fatal signal was delivered causing the control process to dump core. See "systemctl status systemd-journald.service" and "journalctl -xe" for details.
```


So, my first question is this systemd's logic?

But, when /run/log/journal/ used 4G and I clear the log files at this directory /run/log/journal/ by command # journalctl --vacuum-time=2minutes,and then only "system.journal" file exist at /run/log/journal/  directory ,and I restart systemd-journald ,then /run/log/journal/ will be delete;

So, my second question is which behavior should be systemd's logic?
But ,I think when modify from "auto" to "persistent",the log file exist at  /run/log/journal/ should be flushed to disk(/var/log/journal ),and then ,the old journal log file which exist at /run/log/journal/ should be delete 

Isn't it?




How reproducible:


#cat /etc/issue
Alibaba Group Enterprise Linux Server release 7.2 (Paladin)
Kernel \r on an \m



```
[root@muahao:/home/ahao.mah]
#ll /run/log/journal/ad1f3bcdccf14b6b9ac6bc40e73d50fb/
total 4210688
-rw-r-----+ 1 root systemd-journal 117440512 Oct  8 10:07 system
-rw-r-----+ 1 root systemd-journal 117440512 Oct  8 21:13 system
-rw-r-----+ 1 root systemd-journal 117440512 Oct  9 08:14 system
-rw-r-----+ 1 root systemd-journal 117440512 Oct  9 19:18 system
-rw-r-----+ 1 root systemd-journal 117440512 Oct 10 06:19 system
-rw-r-----+ 1 root systemd-journal 117440512 Oct 10 17:23 system
-rw-r-----+ 1 root systemd-journal 117440512 Oct 11 04:27 system
-rw-r-----+ 1 root systemd-journal 117440512 Oct 11 15:30 system
-rw-r-----+ 1 root systemd-journal 117440512 Oct 12 02:34 system
-rw-r-----+ 1 root systemd-journal 117440512 Oct 12 13:36 system
-rw-r-----+ 1 root systemd-journal 117440512 Oct 13 00:47 system
-rw-r-----+ 1 root systemd-journal 117440512 Oct 13 12:00 system
-rw-r-----+ 1 root systemd-journal 117440512 Oct 13 23:16 system
-rw-r-----+ 1 root systemd-journal 117440512 Oct 14 10:29 system
-rw-r-----+ 1 root systemd-journal 117440512 Oct 14 21:55 system
-rw-r-----+ 1 root systemd-journal 125829120 Oct 15 10:00 system
-rw-r-----+ 1 root systemd-journal 125829120 Oct 15 22:13 system
-rw-r-----+ 1 root systemd-journal 125829120 Oct 16 10:02 system
-rw-r-----+ 1 root systemd-journal 125829120 Oct 16 21:47 system
-rw-r-----+ 1 root systemd-journal 125829120 Oct 17 09:30 system
-rw-r-----+ 1 root systemd-journal 125829120 Oct 17 21:13 system
-rw-r-----+ 1 root systemd-journal 125829120 Oct 18 08:54 system
-rw-r-----+ 1 root systemd-journal 125829120 Oct 18 20:39 system
-rw-r-----+ 1 root systemd-journal 125829120 Oct 19 08:19 system
-rw-r-----+ 1 root systemd-journal 125829120 Oct 19 19:56 system

..
..


```


```
[root@muahao:/home/ahao.mah]
#du -sh /run/log/journal/ad1f3bcdccf14b6b9ac6bc40e73d50fb/
4.1G	/run/log/journal/ad1f3bcdccf14b6b9ac6bc40e73d50fb/

```

```
[root@muahao:/home/ahao.mah]
#ll  /var/log/journal
ls: cannot access /var/log/journal: No such file or directory


```

Storage=auto change to Storage=persistent

```
[root@muahao:/home/ahao.mah]
#vim /etc/systemd/journald.conf
[Journal]

Storage=persistent
#Compress=yes
```

```
[root@muahao:/home/ahao.mah]
#systemctl restart systemd-journald
Job for systemd-journald.service failed because a fatal signal was delivered causing the control process to dump core. See "systemctl status systemd-journald.service" and "journalctl -xe" for details.
```

```
[root@muahao:/home/ahao.mah]
#du -sh /run/log/journal/
4.1G	/run/log/journal/
```

```
[root@muahao:/home/ahao.mah]
#du -sh /var/log/journal/
1.6G	/var/log/journal/

```
Comment 3 Noel Duffy 2018-03-05 22:06:30 UTC 
On 7.4, with SELinux in Permissive mode, I see these log lines:

Mar  6 09:57:23 ferdia systemd: Stopping Flush Journal to Persistent Storage...
Mar  6 09:57:23 ferdia journal: Journal stopped
Mar  6 09:57:23 ferdia journal: Permanent journal is using 8.0M (max allowed 408.6M, trying to leave 612.9M 
free of 3.3G available → current limit 408.6M).
Mar  6 09:57:23 ferdia journal: Permanent journal is using 8.0M (max allowed 408.6M, trying to leave 612.9M 
free of 3.3G available → current limit 408.6M).
Mar  6 09:57:23 ferdia journal: Journal stopped
Mar  6 09:57:23 ferdia journal: Permanent journal is using 8.0M (max allowed 408.6M, trying to leave 612.9M 
free of 3.3G available → current limit 408.6M).
Mar  6 09:57:23 ferdia journal: Permanent journal is using 8.0M (max allowed 408.6M, trying to leave 612.9M 
free of 3.3G available → current limit 408.6M).
Mar  6 09:57:23 ferdia journal: Journal stopped
Mar  6 09:57:23 ferdia journal: Permanent journal is using 8.0M (max allowed 408.6M, trying to leave 612.9M 
free of 3.3G available → current limit 408.6M).
Mar  6 09:57:23 ferdia journal: Permanent journal is using 8.0M (max allowed 408.6M, trying to leave 612.9M 
free of 3.3G available → current limit 408.6M).
Mar  6 09:57:23 ferdia journal: Journal stopped
Mar  6 09:57:23 ferdia journal: Permanent journal is using 8.0M (max allowed 408.6M, trying to leave 612.9M 
free of 3.3G available → current limit 408.6M).
Mar  6 09:57:23 ferdia journal: Permanent journal is using 8.0M (max allowed 408.6M, trying to leave 612.9M 
free of 3.3G available → current limit 408.6M).
Mar  6 09:57:23 ferdia journal: Permanent journal is using 8.0M (max allowed 408.6M, trying to leave 612.9M 
free of 3.3G available → current limit 408.6M).
Mar  6 09:57:23 ferdia journal: Permanent journal is using 8.0M (max allowed 408.6M, trying to leave 612.9M 
free of 3.3G available → current limit 408.6M).
Mar  6 09:57:23 ferdia journal: Permanent journal is using 8.0M (max allowed 408.6M, trying to leave 612.9M 
free of 3.3G available → current limit 408.6M).
Mar  6 09:57:23 ferdia journal: Permanent journal is using 8.0M (max allowed 408.6M, trying to leave 612.9M 
free of 3.3G available → current limit 408.6M).
Mar  6 09:57:23 ferdia journal: Permanent journal is using 8.0M (max allowed 408.6M, trying to leave 612.9M 
free of 3.3G available → current limit 408.6M).
Mar  6 09:57:23 ferdia journal: Permanent journal is using 8.0M (max allowed 408.6M, trying to leave 612.9M 
free of 3.3G available → current limit 408.6M).
Mar  6 09:57:23 ferdia journal: Permanent journal is using 8.0M (max allowed 408.6M, trying to leave 612.9M 
free of 3.3G available → current limit 408.6M).
Mar  6 09:57:23 ferdia journal: Permanent journal is using 8.0M (max allowed 408.6M, trying to leave 612.9M 
free of 3.3G available → current limit 408.6M).
Mar  6 09:57:38 ferdia journal: Permanent journal is using 354.4M (max allowed 408.6M, trying to leave 612.9
M free of 3.0G available → current limit 408.6M).
Mar  6 09:57:38 ferdia systemd-journald[474]: Received SIGTERM from PID 1 (systemd).
Mar  6 09:57:38 ferdia systemd-journald[6164]: Failed to set file attributes: Operation not supported
Mar  6 09:57:38 ferdia systemd-journald[6164]: Failed to create new runtime journal: No such file or directory
Mar  6 09:57:38 ferdia systemd-journald[6164]: Failed to set file attributes: Operation not supported
Mar  6 09:57:38 ferdia systemd-journald[6164]: Failed to set file attributes: Operation not supported
Mar  6 09:57:38 ferdia systemd-journald[6164]: Failed to set file attributes: Operation not supported
Mar  6 09:57:38 ferdia systemd-journald[6164]: Failed to set file attributes: Operation not supported
Mar  6 09:57:38 ferdia systemd-journald[6164]: Failed to set file attributes: Operation not supported
Mar  6 09:57:38 ferdia systemd-journald[6164]: Failed to set file attributes: Operation not supported
Mar  6 09:57:38 ferdia systemd-journald[6164]: Assertion 'f' failed at src/journal/journal-file.c:132, function journal_file_close(). Aborting.
Mar  6 09:57:38 ferdia journal: Journal started

So the question is what attribute operation is not supported? My file system is xfs, with mount options (rw,nosuid,nodev,noexec,relatime,seclabel,attr2,inode64,noquota)

# rpm -qa|grep systemd
systemd-219-42.el7_4.7.x86_64
systemd-sysv-219-42.el7_4.7.x86_64
systemd-python-219-42.el7_4.7.x86_64
systemd-libs-219-42.el7_4.7.x86_64
Comment 4 David Tardon 2019-02-01 14:38:09 UTC 

*** This bug has been marked as a duplicate of bug 1399612 ***