Bug 1686946 - Allow MERGE_JSON_LOG=true for indexing of JSON payload fields
Summary: Allow MERGE_JSON_LOG=true for indexing of JSON payload fields
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Logging
Version: 3.10.0
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
: 3.10.z
Assignee: Rich Megginson
QA Contact: Anping Li
Depends On: 1685243
Blocks: 1686947
TreeView+ depends on / blocked
Reported: 2019-03-08 17:53 UTC by Rich Megginson
Modified: 2020-01-31 18:51 UTC (History)
9 users (show)

Fixed In Version: openshift3/logging-fluentd:v3.10.126-1
Doc Type: Bug Fix
Doc Text:
Cause: Using MERGE_JSON_LOG=true can create fields in the record which will cause schema and syntax violations in Elasticsearch. It can also create too many fields for Elasticsearch to handle without severe performance problems. Consequence: Fluentd reports error 400 sending records to Elasticsearch. Elasticsearch performance degrades. Fix: Allow users who experience these problems to tune their Fluentd to accomodate their log record fields. Result: Logs are ingested into Fluentd with no errors. Elasticsearch performance does not degrade.
Clone Of: 1685243
: 1686947 (view as bug list)
Last Closed: 2019-04-09 23:40:44 UTC
Target Upstream Version:
smunilla: needinfo-

Attachments (Terms of Use)

System ID Priority Status Summary Last Updated
Github openshift origin-aggregated-logging pull 1556 None closed [release-3.10] Bug 1685243 - Allow MERGE_JSON_LOG=true for indexing of JSON payload fields 2020-01-31 18:51:19 UTC
Red Hat Product Errata RHBA-2019:0620 None None None 2019-04-09 23:40:47 UTC

Comment 3 Rich Megginson 2019-03-13 02:00:47 UTC
The fix is implemented in rubygem-fluent-plugin-viaq_data_model-0.0.18-1.el7 - please verify that the fluentd image has this version (or later).

Comment 9 Anping Li 2019-03-22 09:31:01 UTC
Move to Modified status. as the bug isn't fix in the release package v3.10.127-1.  list in https://errata.devel.redhat.com/advisory/40290

Comment 11 Anping Li 2019-03-26 08:47:56 UTC
The fix works well on fluentd:v3.10.127

Comment 13 errata-xmlrpc 2019-04-09 23:40:44 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.