Bug 1294168 - RFE : multi-line logging
RFE : multi-line logging
Status: NEW
Product: OpenShift Container Platform
Classification: Red Hat
Component: RFE (Show other bugs)
Unspecified Unspecified
unspecified Severity urgent
: ---
: 3.10.0
Assigned To: Rich Megginson
Junqi Zhao
: 1378592 (view as bug list)
Depends On:
  Show dependency treegraph
Reported: 2015-12-25 02:52 EST by Miheer Salunke
Modified: 2018-06-20 03:01 EDT (History)
31 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 2197831 None None None 2018-04-13 11:35 EDT

  None (edit)
Description Miheer Salunke 2015-12-25 02:52:12 EST
1. Proposed title of this feature request  
Multi-line logging
3. What is the nature and description of the request?  
We have tested the 3.1 centralized logging feature and saw that (like in v2) you do not treat multiline logging messages. We would like to have some way of handling multiline messages, and out-of-the-box support for this in the JBOSS image.
4. Why does the customer need this? (List the business requirements here)  
In v2, we used rsyslog to store the data in a file. Now you provide an elasticsearch/kibana access, but this does not work properly if you do not support multiline messages. Main use case are java exceptions : if you cannot keep the stacktrace as a whole, your metrics in kibana are useless and when you search on a given exception, you do not have the contaxt
5. How would the customer like to achieve this? (List the functional requirements here)  
Instead of using docker-like logging to stdout and treating that line based, you should be able to cope with logging in a multiline capable format (like JSON). The transport/storage/acces is ok, but then the log message is preserved, and not broken up into multiple independent lines.
6. For each functional requirement listed, specify how Red Hat and the customer can test to confirm the requirement is successfully implemented.  
Provoke a stacktrace in a JBOSS image and make sure it is logged as a single event. The result in kibana should show the complete sctacktrace, not a single line of it.
7. Is there already an existing RFE upstream or in Red Hat Bugzilla?

8. Does the customer have any specific timeline dependencies and which release would they like to target (i.e. RHEL5, RHEL6)?  
Openshift Enterprise 3.2
10. List any affected packages or components.  
Openshift Enterprise 3.2 / JBOSS EAP Openshift image
11. Would the customer be able to assist in testing this functionality if implemented? 
Yes. We can provide workloads where we can validate the behaviour.
Comment 2 Rich Megginson 2016-03-28 14:01:48 EDT
Can the customer change their log4j configuration to send stack traces as a single data blob as "stack_trace" field metadata of the log entry?
Comment 3 Rich Megginson 2016-03-28 14:06:21 EDT
Is this a request for multi-line logging in general, or specifically to log and view Java exceptions correctly?  If it is the latter, please change the bug summary to be "RFE: Java exception logging and viewing in Kibana"

Note that multi-line logging in general is extremely difficult, and will require different fluentd configuration for each type of multi-line log.  For example, parsing and viewing a multi-line Java exception is a lot different than a Python exception.
Comment 4 Luke Meyer 2016-03-28 14:36:32 EDT
There is a corresponding github issue https://github.com/openshift/origin-aggregated-logging/issues/28
Comment 12 Dan McPherson 2016-09-22 17:33:11 EDT
*** Bug 1378592 has been marked as a duplicate of this bug. ***
Comment 15 Jeff Cantrill 2016-10-27 15:14:56 EDT
The 3.0 images include (but is disabled) a log formatter to output log entries as JSON.  An example is provided here: https://github.com/knrc/openshift-examples/tree/json-logging-eap6/json-logging-eap6 .  Additional documentation is the deliverable for: https://trello.com/c/k617Elhi which is to advise applications how to produce log messages.  There is nothing we are able to do from the server side to handle the various permutations of logs that will come through the system.
Comment 25 markv 2018-06-15 04:21:18 EDT

Note You need to log in before you can comment on or make changes to this bug.