Bug 1713358

Summary:	Rsyslog holds open deleted files, which consumes all free disk space
Product:	Red Hat Enterprise Linux 7	Reporter:	matousejem
Component:	rsyslog	Assignee:	Jiří Vymazal <jvymazal>
Status:	CLOSED DUPLICATE	QA Contact:	BaseOS QE Security Team <qe-baseos-security>
Severity:	low	Docs Contact:
Priority:	unspecified
Version:	7.6	CC:	1224412750, dapospis, ingo.meldau, julien, jvymazal, rmeggins
Target Milestone:	rc
Target Release:	---
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2019-05-29 11:07:44 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description matousejem 2019-05-23 13:16:37 UTC

Systemd (journal) creates many files similar to `/var/log/journal/d6d244379c464a13a38ae1b05cab0c0a/user-22263`.
Later they are deleted.

### Expected behavior
When a file is deleted from disk, rsyslogd should close it prior to disk is full.

### Actual behavior
From the output of `lsof /var/log` you can see that rsyslog holds open many files. For example:
```
Apr 23 12:25:06 rsyslogd   62213     root 1469r   REG 253,11  8388608  920711 /var/log/journal/d6d244379c464a13a38ae1b05cab0c0a/user-16328 (deleted)
Apr 23 12:25:06 rsyslogd   62213     root 1480r   REG 253,11  8388608  917891 /var/log/journal/d6d244379c464a13a38ae1b05cab0c0a/user-19660 (deleted)
Apr 23 12:25:06 rsyslogd   62213     root 1487r   REG 253,11  8388608  920573 /var/log/journal/d6d244379c464a13a38ae1b05cab0c0a/user-22769 (deleted)
...
...
```
This results to full disk.

### Steps to reproduce the behavior

This is `/etc/systemd/journald.conf`
```
[Journal]
SystemMaxUse=5G
SystemKeepFree=5G
```

### Environment
- rsyslog version: 8.24.0-34
- platform: RHEL 7

### Configuration
`/etc/rsyslog.conf`
```
$ModLoad imuxsock # provides support for local system logging (e.g. via logger command)
$ModLoad imjournal # provides access to the systemd journal
$WorkDirectory /var/lib/rsyslog
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
$OmitLocalLogging on
$IMJournalStateFile imjournal.state
*.info;mail.none;authpriv.none;cron.none;local1.none     /var/log/messages
authpriv.*                                              /var/log/secure
mail.*                                                  -/var/log/maillog
cron.*                                                  /var/log/cron
*.emerg                                                 :omusrmsg:*
uucp,news.crit                                          /var/log/spooler
local7.*                                                /var/log/boot.log
$MaxOpenFiles 4096
local1.*                                                /var/log/dovecot/syslog
*.info                                                  @147.251.48.140
$SystemLogSocketName /run/systemd/journal/syslog
```

Comment 2 Jiří Vymazal 2019-05-29 11:07:44 UTC


*** This bug has been marked as a duplicate of bug 1595840 ***

Comment 3 1224412750 2020-09-24 06:41:18 UTC

(In reply to Jiří Vymazal from comment #2)
> 
> *** This bug has been marked as a duplicate of bug 1595840 ***

Hello, jvymazal,
I have faced the same problem recently and have been troubled for a long time. I asked for help in the rsyslog and systemd communities, but did not get the root cause. 
BUG1595840 requires internal permissions to access. I would love to know the current status of this problem and how to solve it. 
Hope you can help me. 

thank you very much

Comment 4 Buto 2022-10-25 12:15:57 UTC

Hi, 
Is there any news to this issue ?
I also can't see the original bug as mentioned because of permissions.
Thnaks