Bug 1324996 - JSON message fields are getting overwritten
Summary: JSON message fields are getting overwritten
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Logging
Version: 3.1.0
Hardware: Unspecified
OS: Unspecified
medium
high
Target Milestone: ---
: ---
Assignee: ewolinet
QA Contact: chunchen
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-04-07 19:32 UTC by Ryan Howe
Modified: 2019-12-16 05:37 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-05-19 20:12:46 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
message field shown in kibana (167.50 KB, image/png)
2016-05-16 03:19 UTC, chunchen
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2016:1094 0 normal SHIPPED_LIVE Important: Red Hat OpenShift Enterprise 3.2 security update 2016-05-20 00:12:27 UTC

Description Ryan Howe 2016-04-07 19:32:01 UTC
Description of problem: 

While logging out in JSON from camel app via Log4j2 JSONLayout [1] but when looking in Kibana, the "message" field is empty but everything else is imported and works fine.

    {"timeMillis":1459853347608,"thread":"main","level":"INFO","loggerName":"org.apache.camel.spring.SpringCamelContext","message":"Total 1 routes, of which 1 is started.","endOfBatch":false,"loggerFqcn":"org.apache.logging.slf4j.Log4jLogger"}

[1]https://logging.apache.org/log4j/2.x/manual/layouts.html#JSONLayout


Culprit:
https://github.com/openshift/origin-aggregated-logging/blob/master/fluentd/configs.d/filter/k8s_record_transform.conf#L6

Overwrites the message field, regardless if it's set or not. 



Version-Release number of selected component (if applicable):
3.1.1.6

Addition Info: 

http://post-office.corp.redhat.com/archives/openshift-sme/2016-April/msg00236.html

Comment 1 Luke Meyer 2016-04-08 15:02:28 UTC
"log" is elsewhere set based on several other potential fields, but not "message" specifically. It should be a simple patch to make it based on "message" as well if that's present. In general, we should probably make an effort to normalize any meaningful fields before removing them.

Comment 2 Iuliia Ievstignieieva 2016-05-11 09:02:04 UTC
Hi team,

is there any news on this bugzilla please?


Regards,


Julia Ievstignieieva

Operations Support Team lead


GSS EMEA

Comment 5 Troy Dawson 2016-05-13 19:12:12 UTC
Should be in image openshift3/logging-fluentd:3.2.0-8

Comment 7 chunchen 2016-05-16 03:17:59 UTC
It's fixed, tested with openshift3/logging-fluentd:3.2.0-8(ImageId: e1a07eb99c), please refer to the test result in attachment named "message field shown in kibana".

Comment 8 chunchen 2016-05-16 03:19:41 UTC
Created attachment 1157786 [details]
message field shown in kibana

Comment 10 errata-xmlrpc 2016-05-19 20:12:46 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2016:1094


Note You need to log in before you can comment on or make changes to this bug.