Bug 1545345 - fluentd memory use grows until OOM with glibc malloc
Summary: fluentd memory use grows until OOM with glibc malloc
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: glibc
Version: 8.0
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: 8.0
Assignee: glibc team
QA Contact: qe-baseos-tools-bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-02-14 17:17 UTC by Rich Megginson
Modified: 2023-07-18 14:30 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-04-28 07:28:24 UTC
Type: Bug
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Rich Megginson 2018-02-14 17:17:52 UTC
Description of problem:
When using fluentd (0.12.x using ruby 2.0 in RHEL7), the memory usage (as measured by ps h -p $fluentdpid -o rss) grows steadily until it runs out of memory and crashes or is OOMkilled.  This seems mostly exacerbated when using fluentd as a listener (using the secure_forward protocol) of other fluentd collectors.

Version-Release number of selected component (if applicable):
RHEL 7 - ruby 2.0 - fluentd 0.12.x

How reproducible:
Every time

Steps to Reproduce:
1. Deploy OpenShift + logging + mux component
2. Put a load on fluentd e.g. logger -f file_with_many_lines
3. Measure rss of the fluentd mux

Actual results:


Expected results:
I would expect the memory usage to eventually plateau instead of growing constantly.

Additional info:
The mux case is the easiest and fastest way to show the memory growth.

If using OCP 3.7 or later, the fluentd is bundled with jemalloc - set the environment variable USE_JEMALLOC=false to disable jemalloc and use glibc malloc.

Comment 2 Carlos O'Donell 2018-02-14 17:30:42 UTC
Thanks Rich, we will work with you on this issue to see what we can determine about fluentd's particular use case that doesn't lead to stable memory usage over time.

Comment 5 Carlos O'Donell 2020-04-28 16:52:22 UTC
I'm moving this bug to RHEL8, but that shouldn't matter here, this is a tracking bug for the purposes of reviewing the workload issues with malloc.

Comment 13 RHEL Program Management 2022-04-28 07:28:24 UTC
After evaluating this issue, there are no plans to address it further or fix it in an upcoming release.  Therefore, it is being closed.  If plans change such that this issue will be fixed in an upcoming release, then the bug can be reopened.


Note You need to log in before you can comment on or make changes to this bug.