Document URL: https://docs.openshift.com/container-platform/4.2/logging/config/cluster-logging-external.html#cluster-logging-fluentd-external_cluster-logging-external Section Number and Name: Configuring Fluentd to send logs to an external log aggregator Describe the issue: The documentation states: You can configure Fluentd to send a copy of its logs to an external log aggregator, and not the default Elasticsearch, using the out_forward plug-in. However, with the configuration which we have, the logs will be sent to both external aggregators as well as internal Elasticsearch. Suggestions for improvement: Modify the documentation and mention, below configuration will send the logs to both external aggregators as well as internal Elasticsearch. Additional information: This is what we have in the configuration files: https://github.com/openshift/origin-aggregated-logging/blob/release-4.2/fluentd/configs.d/user/secure-forward.conf#L1-L3 Though secure-forward.conf is using @type forward, the output-operations.conf file still has @type copy, and this file will be processed first to send the logs to internal ES and then the copy would be sent to the aggregator mentioned in the secure-forward.conf file. https://github.com/openshift/origin-aggregated-logging/blob/release-4.2/fluentd/configs.d/openshift/output-operations.conf
Jeff -- Is this expected behavior? >Though secure-forward.conf is using @type forward, the output-operations.conf file still has @type copy, and this file will be processed first to send the >logs to internal ES and then the copy would be sent to the aggregator mentioned in the secure-forward.conf file.
Set to Modified to determine if the described behavior is correct. If so, the docs change could apply to 4.3+
Rolfe -- Would you mind taking over this BZ? It is listed as 4.2. Need to determine if it applies to 4.4+. Jeff -- Is this expected behavior? >Though secure-forward.conf is using @type forward, the output-operations.conf file still has @type copy, and this file will be processed first to send the >logs to internal ES and then the copy would be sent to the aggregator mentioned in the secure-forward.conf file.
Tracking this bz in JIRA at https://issues.redhat.com/browse/RHDEVDOCS-2594
Thanks for reporting this issue. However, we don't maintain doc versions 4.4 and earlier. I don't see that content or variants of in in later releases. A separate issue, https://issues.redhat.com/browse/LOG-1172, has been resolved in the current release and is being cherry-picked to previous releases. This bug forced users to choose between using the Fluentd forward protocol "to send logs to destinations outside of your OpenShift Container Platform cluster INSTEAD OF the default Elasticsearch log store by creating a configuration file and config map." (ALLCAPS emphasis mine) [1] Now, users do not have to make an either/or choice; they can do both. I need to update a couple of items in the docs to reflect this change. To avoid confusion, I'm handling those changes with https://issues.redhat.com/browse/RHDEVDOCS-2858. [1]https://docs.openshift.com/container-platform/4.7/logging/cluster-logging-external.html#cluster-logging-collector-legacy-fluentd_cluster-logging-external Thanks again for your help.