This problem is not really due to an defect in Fluentd.
When OpenShift logs to Fluent, the default behaviour is to consider
log messages written to stdout as INFO and stderr as ERROR.
The problem I see with this behaviour is that many programs log
diagnostic information, which would have log level INFO or lower, to
stderr, following the guidelines set out in the POSIX specification
or the C stdlib. Stdout is reserved for the normal output of the
program, which I believe would not contain any log messages, even in
the case of a web server or any other network deamon.
This is not an easy problem. To properly handle the log stream coming
from OpenShift we would need to implement a custom parser for *each*
type of application. However, the current situation generates a large
amount of noise that makes monitoring and interpreting the logs
correctly quite hard. Anything at the ERROR level should be cause for
concern and addressed accordingly. A health probe succeeding is
clearly not something that show up when scanning for these problems.
fluentd - fix is in https://github.com/ViaQ/fluent-plugin-viaq_data_model 0.0.19 https://rubygems.org/downloads/fluent-plugin-viaq_data_model-0.0.19.gem
next steps - update 4.2 source code with new gem, rebuild image
backport 4.2 changes to 4.1, rebuild 4.1 image
3.11 - update gem version in upstream Dockerfile - rebuild rubygem rpm downstream - rebuild downstream image
The downstream rpm work has not been done for 3.11
new rpm built and tagged for 3.11
brew tag-build rhaos-3.11-rhel-7-candidate rubygem-fluent-plugin-viaq_data_model-0.0.19-1.el7
Need 3.11 compose built and fluentd image rebuild
sorry, wrong bz - this bz confused me because it is marked Version: 3.11.0 but Target Release: 4.2.0 - will clone for 3.11
Unknow are used as level. tested using openshift/ose-logging-fluentd:201907222219 and openshift/ose-logging-rsyslo:201907222219
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory, and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.