Bug 1740263
| Summary: | enable the parsing of docker log-driver=json-file continuation lines using fluent concat plugin | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | David Caldwell <dcaldwel> |
| Component: | Logging | Assignee: | Rich Megginson <rmeggins> |
| Status: | CLOSED ERRATA | QA Contact: | Anping Li <anli> |
| Severity: | high | Docs Contact: | |
| Priority: | high | ||
| Version: | 3.11.0 | CC: | aos-bugs, gferrazs, jcantril, jjerezro, rmeggins, zhigwang |
| Target Milestone: | --- | ||
| Target Release: | 3.11.z | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: |
Cause: Fluentd did not look for continuation log lines from the docker json-file or journald log driver.
Consequence: Very long log lines were split in Elasticsearch and Kibana.
Fix: Change Fluentd to look for continuation log lines and reconstruct them before sending to Elasticsearch. We used a Fluentd plugin called fluent-plugin-concat to do this. We had to backport the 2.4.0 version to Fluentd 0.12.43 in order to work with OCP 3.11.
Result: Very long log lines are not split in Elasticsearch. The entire log can be viewed in Kibana.
|
Story Points: | --- |
| Clone Of: | Environment: | ||
| Last Closed: | 2019-09-03 15:56:03 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
|
Comment 1
Rich Megginson
2019-08-14 15:15:34 UTC
Verified in 3.11.141. USE_MULTILINE_JSON - default false - if set to true, and you are using docker with --log-driver=json-file (which is the default), Fluentd will reconstruct whole log records from the docker log partial fragments Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:2580 *** Bug 1792051 has been marked as a duplicate of this bug. *** |