Bug 983230 - systemd-journald leaks buffer_head slab objects
systemd-journald leaks buffer_head slab objects
Product: Fedora
Classification: Fedora
Component: kernel (Show other bugs)
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Kernel Maintainer List
Fedora Extras Quality Assurance
Depends On:
  Show dependency treegraph
Reported: 2013-07-10 14:58 EDT by Phil Oester
Modified: 2013-07-12 14:53 EDT (History)
14 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2013-07-11 13:43:14 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description Phil Oester 2013-07-10 14:58:25 EDT
Description of problem:
If systemd-journald is running, and some process is doing printk's, buffer_head slab objects increase and never decrease

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1. Make sure systemd-journald is running (but rsyslog.service can be stopped)
2. Start a process to cause printks.  I am using this:
while : ; do modprobe nf_conntrack ; modprobe -r nf_conntrack ; done
which simply prints out the nf_conntrack version on each module load cycle
3. In another window, launch slabtop
4. Note buffer_head count increasing in slabtop
5. Stop systemd-journald service, and note that buffer_head count stops increasing
Comment 1 Michal Schmidt 2013-07-11 05:16:43 EDT
(In reply to Phil Oester from comment #0)
> If systemd-journald is running, and some process is doing printk's,
> buffer_head slab objects increase and never decrease

Does it have to be printk? Do you not observe the same phenomenon using logs originating in userspace? Such as:

while :; do
  for i in `seq 1000`; do
    logger Hello
  slabtop -o | grep buffer_head

More importantly: Does the slab grow completely out of bounds? Does it eventually exhaust all RAM and cause an Out-Of-Memory condition? I don't think it would do that. It's supposed to be reclaimable when there's memory pressure.
Does it shrink when you do "echo 3 > /proc/sys/vm/drop_caches"?

What filesystem is your /var/log/journal on? ext4?

I think this is NOTABUG, but I am reassigning to kernel first (because if it were a real bug, it would be in the kernel).
Comment 2 Phil Oester 2013-07-11 13:43:14 EDT
I booted the box with less ram (mem=), and it does not OOM.  

Was not aware that systemd-journald was writing binary logfile data to disk.  I assumed with rsyslogd shutdown, all disk logging activity should be halted (like the "old days").  Now apparently you have to stop two separate logging services to stop disk logging.  

So this buffer_head usage appears to be normal caching, and this report can be closed as NOTABUG.
Comment 3 Michal Schmidt 2013-07-12 14:53:27 EDT
OK, thanks for testing this.

I strongly recommend against disabling systemd-journald. Rather, in /etc/systemd/journald.conf set "Storage=" to "volatile", or "none". See man journald.conf for details.

Note You need to log in before you can comment on or make changes to this bug.