Bug 1857094

Summary: Should not create ClusterLogForwarder without specifying `url` in the output.
Product: OpenShift Container Platform Reporter: Qiaoling Tang <qitang>
Component: LoggingAssignee: IgorKarpukhin <ikarpukh>
Status: CLOSED ERRATA QA Contact: Qiaoling Tang <qitang>
Severity: medium Docs Contact:
Priority: medium    
Version: 4.6CC: aos-bugs, ewolinet, ikarpukh
Target Milestone: ---   
Target Release: 4.6.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: logging-core
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-10-27 16:14:38 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Qiaoling Tang 2020-07-15 06:40:52 UTC
Description of problem:
Creating the ClusterLogForwarder CR without specifying `url` in the output, the CR could be created successfully. 

$ oc get clf instance -oyaml
spec:
  outputs:
  - name: fluentd-created-by-user
    secret:
      name: fluentdserver
    type: fluentdForward
  pipelines:
  - inputRefs:
    - infrastructure
    - application
    - audit
    name: forward-to-external-fluentd
    outputRefs:
    - fluentd-created-by-user
status:
  conditions:
  - lastTransitionTime: "2020-07-15T06:23:58Z"
    message: 'all pipelines invalid: [forward-to-external-fluentd]'
    reason: Invalid
    status: "False"
    type: Ready
  outputs:
    fluentd-created-by-user:
    - lastTransitionTime: "2020-07-15T06:23:58Z"
      message: 'output "fluentd-created-by-user": missing URL'
      reason: Invalid
      status: "False"
      type: Ready
  pipelines:
    forward-to-external-fluentd:
    - lastTransitionTime: "2020-07-15T06:23:58Z"
      message: 'invalid: unrecognized outputs: [fluentd-created-by-user], no valid
        outputs'
      reason: Invalid
      status: "False"
      type: Ready

The fluent.conf in cm/fluentd turn to empty after the CR is created, and the CLO updates all the fluentd pods:
CLO logs:
time="2020-07-15T06:23:58Z" level=warning msg="Unable to generate log configuration: No recognized input types: []"
time="2020-07-15T06:23:58Z" level=info msg="Collector container EnvVar change found, updating \"fluentd\""
time="2020-07-15T06:23:58Z" level=info msg="Collector volumes change found, updating \"fluentd\""
time="2020-07-15T06:23:58Z" level=info msg="Collector container volumemounts change found, updating \"fluentd\""
time="2020-07-15T06:23:58Z" level=warning msg="Unable to generate log configuration: No recognized input types: []"
time="2020-07-15T06:23:58Z" level=warning msg="Unable to generate log configuration: No recognized input types: []"
time="2020-07-15T06:23:58Z" level=info msg="Updating status of Fluentd"
time="2020-07-15T06:23:59Z" level=warning msg="Unable to generate log configuration: No recognized input types: []"
time="2020-07-15T06:24:01Z" level=warning msg="Unable to generate log configuration: No recognized input types: []"
time="2020-07-15T06:24:01Z" level=info msg="Updating status of Fluentd"
time="2020-07-15T06:24:02Z" level=warning msg="Unable to generate log configuration: No recognized input types: []"
time="2020-07-15T06:24:02Z" level=info msg="Updating status of Fluentd"

Version-Release number of selected component (if applicable):
quay.io/openshift/origin-cluster-logging-operator@sha256:0e01abc6a8a5a1f6c86d467891faab48ae8fea411ebd49638b684438e35d92d3


How reproducible:
Always

Steps to Reproduce:
1. deploy logging
2. create ClusterLogForwarder CR instance with:
apiVersion: logging.openshift.io/v1
kind: ClusterLogForwarder
metadata:
  name: instance
  namespace: openshift-logging
spec:
  outputs:
  - name: fluentd-created-by-user
    type: fluentdForward
    secret:
      name: fluentdserver
  pipelines:
  - name: forward-to-external-fluentd
    inputRefs:
    - infrastructure
    - application
    - audit
    outputRefs:
    - fluentd-created-by-user

3. check the fluentd pods

Actual results:


Expected results:
The `url` should be a required value, users couldn't create CLF CR without `url`.

Additional info:

Comment 4 Qiaoling Tang 2020-08-04 05:52:03 UTC
Tested with ose-cluster-logging-operator-v4.6.0-202008040110.p0, the issue isn't fixed, I still could create CLF without specifying `url` in the `outputs`.

$ oc get clf -oyaml
apiVersion: v1
items:
- apiVersion: logging.openshift.io/v1
  kind: ClusterLogForwarder
  metadata:
    creationTimestamp: "2020-08-04T05:47:14Z"
    generation: 1
    managedFields:
    - apiVersion: logging.openshift.io/v1
      fieldsType: FieldsV1
      fieldsV1:
        f:spec:
          f:outputs: {}
        f:status:
          .: {}
          f:conditions: {}
      manager: cluster-logging-operator
      operation: Update
      time: "2020-08-04T05:47:14Z"
    - apiVersion: logging.openshift.io/v1
      fieldsType: FieldsV1
      fieldsV1:
        f:spec:
          .: {}
          f:pipelines: {}
      manager: oc
      operation: Update
      time: "2020-08-04T05:47:14Z"
    name: instance
    namespace: openshift-logging
    resourceVersion: "200018"
    selfLink: /apis/logging.openshift.io/v1/namespaces/openshift-logging/clusterlogforwarders/instance
    uid: 734d16d8-0fa3-4be6-9222-583f63221411
  spec:
    outputs:
    - name: fluentd-created-by-user
      secret:
        name: fluentdserver
      type: fluentdForward
    pipelines:
    - inputRefs:
      - infrastructure
      - application
      - audit
      name: forward-to-fluentd-server
      outputRefs:
      - fluentd-created-by-user
  status:
    conditions:
    - lastTransitionTime: "2020-08-04T05:47:14Z"
      status: "True"
      type: Ready
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""

Comment 5 IgorKarpukhin 2020-08-20 14:51:03 UTC
This field is optional and can be empty if there is an output-type field with alternative connection information. But I guess I can add additional check for fluentd...

Comment 6 IgorKarpukhin 2020-08-21 14:34:48 UTC
This PR solves that: https://github.com/openshift/cluster-logging-operator/pull/668 I made URL field mandatory for the fluentdForwarder, elasticsearch and syslog

Comment 8 Qiaoling Tang 2020-08-25 01:48:42 UTC
Verified with clusterlogging.4.6.0-202008241350.p0

Comment 10 errata-xmlrpc 2020-10-27 16:14:38 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.6 GA Images), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:4196