Bug 1567991 - Fluentd: complete record sent to ES
Summary: Fluentd: complete record sent to ES
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Logging
Version: 3.5.0
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
: ---
Assignee: Jeff Cantrill
QA Contact: Anping Li
Depends On:
TreeView+ depends on / blocked
Reported: 2018-04-16 14:18 UTC by François Cami
Modified: 2018-06-26 08:45 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Last Closed: 2018-04-18 14:13:48 UTC
Target Upstream Version:

Attachments (Terms of Use)

Description François Cami 2018-04-16 14:18:29 UTC
Description of problem:
Fluentd now adds the complete record to ES, adding uneeded noise and space usage to the ES cluster.

Version-Release number of selected component (if applicable):
logging-fluentd v3.

Comment 4 Rich Megginson 2018-04-16 17:09:03 UTC
What is the severity of this problem?  Is it worth fixing in 3.5, if the problem has been fixed in 3.6 and later and can be resolved by upgrading?

Comment 5 François Cami 2018-04-17 08:31:11 UTC
I don't think a fix for 3.5 is warranted at this point.

Comment 6 Jeff Cantrill 2018-04-18 14:13:48 UTC
Closing based on #c5 an fixed in 3.6 and later

Comment 7 Rich Megginson 2018-04-18 14:17:55 UTC
(In reply to François Cami from comment #5)
> I don't think a fix for 3.5 is warranted at this point.

Just to confirm: the record should look like https://bugzilla.redhat.com/show_bug.cgi?id=1567991#c1 _except_ that the "message" field should look like this:

    "message":" [13/Apr/2018:16:00:36 +0200] GET /CCC/ HTTP/1.1 200 320 0"

instead of containing the embedded JSON document

I don't suppose you could test this on OCP 3.6 or later, to see if it is fixed there?  The code is quite different in 3.6 and later.

Comment 8 François Cami 2018-04-18 14:22:50 UTC
Rich, it will be tested in the following weeks, on 3.6+.

Comment 9 François Cami 2018-06-26 08:45:27 UTC
Tested and fixed in 3.9.

Note You need to log in before you can comment on or make changes to this bug.