Bug 1862367 - Couldn't gather the infra logs when enabled both ns filter inputref and infrastructure
Summary: Couldn't gather the infra logs when enabled both ns filter inputref and infra...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Logging
Version: 4.6
Hardware: Unspecified
OS: Unspecified
urgent
high
Target Milestone: ---
: 4.6.0
Assignee: Vimal Kumar
QA Contact: Anping Li
URL:
Whiteboard: logging-core
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-07-31 09:05 UTC by Anping Li
Modified: 2020-10-27 15:09 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-10-27 15:09:31 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
fluent.conf (13.84 KB, text/plain)
2020-07-31 09:19 UTC, Anping Li
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Github openshift cluster-logging-operator pull 675 0 None closed Bug 1862367: fixed application namespace filtering 2020-09-17 02:54:09 UTC
Red Hat Product Errata RHBA-2020:4198 0 None None None 2020-10-27 15:09:47 UTC

Description Anping Li 2020-07-31 09:05:15 UTC
Description of problem:

When enabled both ns filter inputref and infrastructure. I couldn't find any docs in the infra-00001 indices. 

The <source> path should include logs from projects 'kube-', 'openshift-', 'defaut' and 'openshift'/
<source>
  @type tail
  @id container-input
  path /var/log/containers/*_project1_*.log, /var/log/containers/*_project2_*.log

It is strange the journald logs wasn't sent out.


Version-Release number of selected component (if applicable):
4.6

How reproducible:
always

Steps to Reproduce:
1) Deploy clusterlogging
2) Send filtered and infra project logs to default 

apiVersion: logging.openshift.io/v1
kind: ClusterLogForwarder
metadata:
  name: instance
  namespace: openshift-logging
spec:
  outputs:
    - name: fluentd-created-by-user
      type: fluentdForward
      secret:
         name: 'fluentdserver'
  pipelines:
    - name: forward-to-fluentd-server
      inputRefs:
      - infrastructure
      - application
      - audit
      outputRefs:
      - fluentd-created-by-user

3) check the docs in Elasticsearch

$ oc exec -c elasticsearch elasticsearch-cdm-0e77m86s-1-7c7bfc66df-fv8g5 -- es_util '--query=_cat/indices?v'
health status index        uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   infra-000001 9xwkMPfQQeyyUBQbkUIiNw   1   0          0            0       261b           261b
green  open   .security    hsT3Gpw4QiuC23dJcV4jvg   1   0          5            0     29.9kb         29.9kb
green  open   audit-000001 CIcwYm1MSe-f_xjgqlcOag   1   0          0            0       261b           261b
green  open   app-000001   Y17UcO3QRyKQ9ZTx_Ttz_g   1   0       1361            0      715kb          715kb


Actual Result:

No docs in infra-000001

Comment 1 Anping Li 2020-07-31 09:19:42 UTC
Created attachment 1703077 [details]
fluent.conf

Please ignore the ClusterLogForwarder CR in step 2. the correct CLF is as below.

{
    "apiVersion": "logging.openshift.io/v1",
    "kind": "ClusterLogForwarder",
    "metadata": {
        "name": "instance",
        "namespace": "openshift-logging",
    },
    "spec": {
        "inputs": [
            {
                "application": {
                    "namespaces": [
                        "project1",
                        "project2"
                    ]
                },
                "name": "inputest"
            }
        ],
        "pipelines": [
            {
                "inputRefs": [
                    "inputest",
                    "infrastructure"
                ],
                "name": "test-app",
                "outputRefs": [
                    "default"
                ]
            }
        ]
    }
}

Comment 4 Anping Li 2020-08-27 07:50:40 UTC
Verified on clusterlogging.4.6.0-202008262209.p0

Comment 6 errata-xmlrpc 2020-10-27 15:09:31 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.6.1 extras update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:4198


Note You need to log in before you can comment on or make changes to this bug.