Bug 1905615

Summary: can't forward all application log to internal(OCP ES) while forwarding logs from specific projects to external aggregator
Product: OpenShift Container Platform Reporter: Mani <mmohan>
Component: LoggingAssignee: Jeff Cantrill <jcantril>
Status: CLOSED ERRATA QA Contact: Anping Li <anli>
Severity: medium Docs Contact:
Priority: medium    
Version: 4.6CC: achakrat, anli, aos-bugs, benjamin.merot, periklis, qitang, stwalter
Target Milestone: ---   
Target Release: 4.6.z   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: logging-core
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Cause: The cluster-logging-operator generated the collector configuration incorrectly Consequence: Log messages were not correctly routing the the desired destinations Fix: Correct the config generator Result: Logs from specific namespaces can be selectively targeted to desired outputs using the cluster log forwarding resource
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-05-12 12:16:04 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1921263, 1925361, 1928949    
Bug Blocks:    
Attachments:
Description Flags
fluent.conf from master
none
fluent.conf from 4.6 gen none

Description Mani 2020-12-08 16:20:40 UTC
Description of problem:

While sending specific namespace logs to an external aggregator as well as all application namespace logs to internal OCP elastic search(default). Internal ES  could see only the specific namespace(test) logs.


Version-Release number of selected component (if applicable):

4.6
How reproducible:


Steps to Reproduce:
1. Configure Cluster log forwarder with any external aggregator and also include the internal elastic search.

~~~
apiVersion: logging.openshift.io/v1
kind: ClusterLogForwarder
metadata:
  name: instance
  namespace: openshift-logging
spec:
  inputs:
    - application:
        namespaces: 
         - test
      name: test-app-logs
  outputs:
   - name: rsyslog
     type: syslog 
     syslog: 
       facility: local0
       rfc: RFC3164
       payloadKey: message
       severity: informational
     url: 'tcp://<syslog-ip>:514'
  pipelines:
    - inputRefs:
        - application
        - audit
        - infrastructure
      name: application-audit-infrastructure-logs
      outputRefs:
        - default
    - inputRefs:
        - test-app-logs
      name: forward-to-syslog
      outputRefs:
        - rsyslog
~~~



2.
3.

Actual results:

Internal Elastic search can only see the specific namespace

Expected results:

All the application namespace log should available internal elastic search while sending specific namespace to external

Additional info:

Comment 1 Benjamin Merot 2021-01-27 10:21:47 UTC
Could this situation have something to do with the way retention is configured according to https://docs.openshift.com/container-platform/4.6/logging/config/cluster-logging-log-store.html#cluster-logging-elasticsearch-retention_cluster-logging-store

If you do not specify a retention policy for all three log sources, only logs from the sources with a retention policy are stored. For example, if you set a retention policy for the infrastructure and applicaiton logs, but do not set a retention policy for audit logs, the audit logs will not be retained and there will be no audit- index in Elasticsearch or Kibana.

Comment 2 Jeff Cantrill 2021-02-10 18:07:19 UTC
This issue is actually combination of the following:

* https://bugzilla.redhat.com/show_bug.cgi?id=1925361
* https://bugzilla.redhat.com/show_bug.cgi?id=1921263

Comment 4 Jeff Cantrill 2021-03-17 23:50:48 UTC
*** Bug 1900804 has been marked as a duplicate of this bug. ***

Comment 8 Anping Li 2021-04-20 08:46:04 UTC
The pod logs in test namespace can not sent to default ES. The other pods logs can be sent to default ES.

apiVersion: logging.openshift.io/v1
kind: ClusterLogForwarder
metadata:
  name: instance
  namespace: openshift-logging
spec:
  inputs:
    - application:
        namespaces: 
         - test
      name: test-app-logs
  outputs:
   - name: rsyslog
     type: syslog 
     syslog: 
       facility: local0
       severity: informational
     url: 'udp://rsyslogserver.openshift-logging.svc:514'
  pipelines:
    - inputRefs:
        - application
        - audit
        - infrastructure
      name: application-audit-infrastructure-logs
      outputRefs:
        - default
    - inputRefs:
        - test-app-logs
      name: forward-to-syslog
      outputRefs:
        - rsyslog

Comment 10 Jeff Cantrill 2021-04-28 19:04:02 UTC
Created attachment 1776873 [details]
fluent.conf from master

Comment 11 Jeff Cantrill 2021-04-28 19:04:32 UTC
Created attachment 1776874 [details]
fluent.conf from 4.6 gen

Comment 12 Jeff Cantrill 2021-04-28 19:08:06 UTC
@anli, Based on using our config generators for 4.6 and master, the configurations are almost identical.  I have attached those generated conf to this issue.  Your 4.6 fluentd config should be similar to #c11.  Please reconfirm your testing as it was confirmed as verified in both of:
* https://issues.redhat.com/browse/LOG-1224
* https://issues.redhat.com/browse/LOG-1222

Comment 13 Anping Li 2021-04-29 06:10:58 UTC
Yes, that is confused, The fluentd.conf looks good. I will close this one as it do fix the issue Description. And file a new bug if I can find provide more detail.

Comment 16 errata-xmlrpc 2021-05-12 12:16:04 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.6.28 extras update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:1489