Bug 1492188 - Add keepalive, max_retry_wait parameters to fluentd secure_forward configuration
Summary: Add keepalive, max_retry_wait parameters to fluentd secure_forward configuration
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: ovirt-engine-metrics
Classification: oVirt
Component: Generic
Version: unspecified
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ovirt-4.1.7
: 1.0.7.1
Assignee: Shirly Radco
QA Contact: Lukas Svaty
URL:
Whiteboard:
Depends On:
Blocks: 1475135 1493030
TreeView+ depends on / blocked
 
Reported: 2017-09-15 16:58 UTC by Rich Megginson
Modified: 2017-11-14 11:43 UTC (History)
2 users (show)

Fixed In Version: ovirt-engine-metrics-1.0.7.1
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-11-13 12:26:40 UTC
oVirt Team: Metrics
rule-engine: ovirt-4.1+


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 82329 0 master MERGED fluentd: set buffer configurations 2017-09-28 19:46:36 UTC
oVirt gerrit 82330 0 ovirt-engine-metrics-4.1 MERGED fluentd: set buffer configurations 2017-09-28 19:56:48 UTC
oVirt gerrit 82422 0 master MERGED fluentd: set buffer configurations 2017-10-01 11:33:35 UTC
oVirt gerrit 82423 0 ovirt-engine-metrics-4.1 MERGED fluentd: fix bug in buffer configurations 2017-10-01 11:42:19 UTC

Description Rich Megginson 2017-09-15 16:58:38 UTC
Description of problem:
When fluentd secure_forward connects to a remote secure_forward listener through a proxy, the proxy cannot load balance the connections until they attempt to reconnect.  There is a parameter `keepalive` which can be used to force fluentd secure_forward to reconnect periodically to help with load balancing.

https://github.com/tagomoris/fluent-plugin-secure-forward#secureforwardoutput

I suggest using a value of 5 minutes (300 seconds), and we can tune from there.

I guess this would need to be added to your ansible playbooks/templates for fluentd.

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 1 Rich Megginson 2017-09-25 02:05:51 UTC
Also set `max_retry_wait 300`
In my testing of ovirt -> logging, if there are connection problems, the ovirt fluentd will keep exponentially backing off until it is waiting for several hours.  I recommend a max_retry_wait of 300.

Comment 2 Lukas Svaty 2017-10-19 15:44:57 UTC
missing in ovirt-engine-metrics-1.0.7-1.el7ev.noarch

Comment 3 Sandro Bonazzola 2017-10-21 06:14:31 UTC
Lukas, note this has been fixed in 1.0.7.1 not 1.0.7-1

Comment 4 Lukas Svaty 2017-10-25 10:21:43 UTC
[root@/ ]# grep -R max_retry /etc/fluentd/
/etc/fluentd/config.d/30-source-forward.conf:  max_retry_wait 300s
/etc/fluentd/config.d/30-source-forward.conf:  max_retry_wait 300s

verified in ovirt-engine-metrics-1.0.7.1-1.el7ev.noarch


Note You need to log in before you can comment on or make changes to this bug.