Bug 1765646 - Fluentd single pod logging performance regression 3.x
Summary: Fluentd single pod logging performance regression 3.x
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Logging
Version: 3.11.0
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 3.11.z
Assignee: Jeff Cantrill
QA Contact: Anping Li
URL:
Whiteboard:
Depends On: 1741663
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-10-25 16:22 UTC by German Parente
Modified: 2024-02-04 04:25 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-04-14 19:24:41 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift openshift-ansible pull 11999 0 'None' closed Bug 1765646: Fluentd single pod logging performance regression 3.x 2021-01-06 02:47:33 UTC
Github openshift origin-aggregated-logging pull 1779 0 'None' closed Bug 1765646: Fluentd single pod logging performance regression 3.x 2021-01-06 02:46:56 UTC
Red Hat Knowledge Base (Solution) 4766981 0 None None None 2020-01-23 16:34:52 UTC
Red Hat Product Errata RHBA-2019:4050 0 None None None 2019-12-16 11:57:26 UTC

Internal Links: 1780560

Description German Parente 2019-10-25 16:22:07 UTC
Description of problem:

This is just a clone of the 4.x bugzilla

Bug 1741663 - Fluentd single pod logging performance regression in 4.x

but for the 3.11

Version-Release number of selected component (if applicable): 3.11

it seems that in the mentioned bugzilla, the idea was to expose

rotate_wait 

parameter to fluentd.

Comment 5 Rich Megginson 2019-11-05 20:33:54 UTC
https://github.com/openshift/openshift-ansible/pull/11999 adds ansible support as well

Comment 6 Rich Megginson 2019-11-15 23:34:36 UTC
https://github.com/openshift/origin-aggregated-logging/pull/1779 has merged

Comment 8 Anping Li 2019-12-03 02:08:28 UTC
Verified. Once we enable openshift_logging_fluentd_refresh_interval=6&openshift_logging_fluentd_rotate_wait=6. The Containers logs are can be controlled by them.

sh-4.2# env |grep CONTAINER_LOGS_
CONTAINER_LOGS_REFRESH_INTERVAL=6
CONTAINER_LOGS_ROTATE_WAIT=6


sh-4.2# grep -r refresh_interval
dynamic/input-docker-default-docker.conf:  refresh_interval 6
sh-4.2# grep -r rotate_wait     
dynamic/input-docker-default-docker.conf:  rotate_wait 6

Comment 10 errata-xmlrpc 2019-12-16 11:57:10 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:4050

Comment 13 German Parente 2020-04-07 07:58:39 UTC
Hi,

I am re-opening this bug because the customer is still hitting it, even if less frequently. Customer environment:

pods running for more than 3 weeks of an application that writes 2G of logs in less than 2days.
Docker options are --log-opt max-size=2G --log-opt max-file=2

Comment 16 Red Hat Bugzilla 2024-02-04 04:25:26 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days


Note You need to log in before you can comment on or make changes to this bug.