Description of problem:
As per documentation, we can configure ClusterLogForwarder to send project-specific logs to external/internal log aggregators.
If only single project logs need to be forwarded, the configuration works fine.
But if we need to forward multiple project logs to different destinations, the fluentd configuration is messed up.
For example, forwarding logs from two different projects to two different external elasticsearch instances, the logs from both the projects are forwarded to both the elasticsearch instances, due to the fluentd configuration getting messed up as below:
# A log source matcher may be null if no pipeline wants that type of log.
<match **_default_** **_kube-*_** **_openshift-*_** **_openshift_** journal.** system.var.log**>
<match kubernetes.**_dev-apple_** kubernetes.**_dev-ocp_** >
@label @_APPLICATION <<<<< Here combined logs are labelled as APPLICATION logs
<match kubernetes.** >
<match linux-audit.log** k8s-audit.log** openshift-audit.log**>
# Relabel specific sources (e.g. logs.apps) to multiple pipelines <<< Then we have this section, I believe this section should be merged/corrected with above section
Version-Release number of selected component (if applicable):
Steps to Reproduce:
1. Deploy an OCP 4.6 cluster and install cluster-logging stack on it.
2. Configure clusterlogforwarder to forward logs from two projects to different elasticsearch instance.
3. Check the logs on external elasticsearch instances, it will contain logs from both the projects
Logs from both the project are available on both elasticsearch instances
Logs from one project to go to the first external elasticsearch instance and the second projects logs to the second elasticsearch instance.
 Example CLF configuration:
- name: elasticsearch-ocp
- name: elasticsearch-apple
- name: input-namespaces-ocp
- name: input-namespaces-apple
- name: logs-namespaces-ocp
- name: logs-namespaces-apple
Note: before fixing this, check the progress of https://github.com/openshift/cluster-logging-operator/pull/865
That PR can fix this bug as a side-effect.
The fix is actively being worked on now.
Here's the pull request on GitHub, I think you should be able to see it: https://github.com/openshift/cluster-logging-operator/pull/955
Moved to modified as PR merged
Verified on clusterlogging.4.6.0-202104161407.p0
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory (Important: OpenShift Container Platform 4.6.26 security and extras update), and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.
*** Bug 1953646 has been marked as a duplicate of this bug. ***