Bug 1686946 - Allow MERGE_JSON_LOG=true for indexing of JSON payload fields
Summary: Allow MERGE_JSON_LOG=true for indexing of JSON payload fields
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Logging
Version: 3.10.0
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: 3.10.z
Assignee: Rich Megginson
QA Contact: Anping Li
URL:
Whiteboard:
Depends On: 1685243
Blocks: 1686947
TreeView+ depends on / blocked
 
Reported: 2019-03-08 17:53 UTC by Rich Megginson
Modified: 2020-01-31 18:51 UTC (History)
9 users (show)

Fixed In Version: openshift3/logging-fluentd:v3.10.126-1
Doc Type: Bug Fix
Doc Text:
Cause: Using MERGE_JSON_LOG=true can create fields in the record which will cause schema and syntax violations in Elasticsearch. It can also create too many fields for Elasticsearch to handle without severe performance problems. Consequence: Fluentd reports error 400 sending records to Elasticsearch. Elasticsearch performance degrades. Fix: Allow users who experience these problems to tune their Fluentd to accomodate their log record fields. Result: Logs are ingested into Fluentd with no errors. Elasticsearch performance does not degrade.
Clone Of: 1685243
: 1686947 (view as bug list)
Environment:
Last Closed: 2019-04-09 23:40:44 UTC
Target Upstream Version:
Embargoed:
smunilla: needinfo-

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift origin-aggregated-logging pull 1556 0 None closed [release-3.10] Bug 1685243 - Allow MERGE_JSON_LOG=true for indexing of JSON payload fields 2020-01-31 18:51:19 UTC
Red Hat Product Errata RHBA-2019:0620 0 None None None 2019-04-09 23:40:47 UTC

Comment 1 Rich Megginson 2019-03-12 18:20:22 UTC 
https://github.com/openshift/origin-aggregated-logging/pull/1556
Comment 2 Rich Megginson 2019-03-12 20:07:15 UTC 
merged upstream https://github.com/openshift/origin-aggregated-logging/commit/02dd621fa3add25c6c686994be211faa64f72140
Comment 3 Rich Megginson 2019-03-13 02:00:47 UTC 
The fix is implemented in rubygem-fluent-plugin-viaq_data_model-0.0.18-1.el7 - please verify that the fluentd image has this version (or later).
Comment 9 Anping Li 2019-03-22 09:31:01 UTC 
Move to Modified status. as the bug isn't fix in the release package v3.10.127-1.  list in https://errata.devel.redhat.com/advisory/40290
Comment 11 Anping Li 2019-03-26 08:47:56 UTC 
The fix works well on fluentd:v3.10.127
Comment 13 errata-xmlrpc 2019-04-09 23:40:44 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:0620
Note You need to log in before you can comment on or make changes to this bug.