Bug 1557431

Summary:	[fluentd] take forward + secure_forward outputs into account when computing NUM_OUTPUTS
Product:	OpenShift Container Platform	Reporter:	François Cami <fcami>
Component:	Logging	Assignee:	Jeff Cantrill <jcantril>
Status:	CLOSED ERRATA	QA Contact:	Anping Li <anli>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	3.5.0	CC:	aos-bugs, nhosoi, rmeggins
Target Milestone:	---
Target Release:	3.6.z
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:	Cause: The fluentd file buffer size is calculated in the fluentd start script run.sh at the start up time. In the calculation, the script gathers the number of outputs, then the available disk space is divided by the gathered number. When gathering the number of outputs, it missed the case for the non-secure forward. If it happens to be configured, the number of outputs is one less than actual. Consequence: When the non-secure forward is configured, the calculated file buffer size is larger than the expected size, which might cause the disk full on the filesystem for the fluentd file buffer. Fix: The non-secure forward case is taken account. Result: The file buffer size is correctly calculated when the non-secure forward is configured.	Story Points:	---
Clone Of:		Environment:
Last Closed:	2018-05-17 07:58:35 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description François Cami 2018-03-16 14:56:11 UTC

In run.sh, specifically line 218:

https://github.com/openshift/origin-aggregated-logging/blob/master/fluentd/run.sh#L218

sec_forward_files=$( grep -l "@type *secure_forward" ${CFG_DIR}/*/* 2> /dev/null || : )

and line 222:

https://github.com/openshift/origin-aggregated-logging/blob/master/fluentd/run.sh#L222

grep "@type *secure_forward" $afile | while read -r line; do


we only take the secure_forward plugin into account, not the forward plugin.

Changing the two afore-mentioned lines to:
sec_forward_files=$( grep -l "@type *forward" ${CFG_DIR}/*/* 2> /dev/null || : )
and:
grep "@type *forward" $afile | while read -r line; do
takes into account both secure_forward and forward, computing the value of NUM_OUTPUTS properly.

Comment 3 François Cami 2018-03-16 16:24:10 UTC

Upstream PR: https://github.com/openshift/origin-aggregated-logging/pull/1021

Comment 6 Jeff Cantrill 2018-04-27 14:48:40 UTC

Backported to 3.6 in https://github.com/openshift/origin-aggregated-logging/pull/1030

Comment 9 Jeff Cantrill 2018-05-04 13:01:51 UTC

Noriko,

Please update the doc text for this issue

Comment 10 Noriko Hosoi 2018-05-04 17:17:42 UTC

(In reply to Jeff Cantrill from comment #9)
> Noriko,
> 
> Please update the doc text for this issue

Done!

Comment 11 Anping Li 2018-05-09 01:56:19 UTC

Verified with openshift3/logging-fluentd:v3.6.173.0.117-1

Comment 14 errata-xmlrpc 2018-05-17 07:58:35 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:1579