Bug 1842043

Summary: Lot of proxy config reconciliation error before clusterlogging CR are provided
Product: OpenShift Container Platform Reporter: Anping Li <anli>
Component: LoggingAssignee: Periklis Tsirakidis <periklis>
Status: CLOSED ERRATA QA Contact: Anping Li <anli>
Severity: low Docs Contact:
Priority: unspecified    
Version: 4.5CC: aos-bugs, periklis
Target Milestone: ---   
Target Release: 4.6.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-10-27 16:01:56 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1844329    

Description Anping Li 2020-05-30 12:51:11 UTC
Description of problem:
There are lots of 'proxy config reconciliation' error before the clusterlogging instance are created. The EO should be in green status if no CR is given.  if we do want to show the 'proxy config reconciliation' messages. it should be a warning message.



oc logs elasticsearch-operator-6586fc7bc8-hckz5
{"level":"info","ts":1590842610.5373578,"logger":"cmd","msg":"Go Version: go1.13.4"}
{"level":"info","ts":1590842610.5373883,"logger":"cmd","msg":"Go OS/Arch: linux/amd64"}
{"level":"info","ts":1590842610.5373933,"logger":"cmd","msg":"Version of operator-sdk: v0.8.2"}
{"level":"info","ts":1590842610.53796,"logger":"leader","msg":"Trying to become the leader."}
{"level":"info","ts":1590842610.7530456,"logger":"leader","msg":"No pre-existing lock was found."}
{"level":"info","ts":1590842610.7605128,"logger":"leader","msg":"Became the leader."}
{"level":"info","ts":1590842610.943608,"logger":"cmd","msg":"Registering Components."}
{"level":"info","ts":1590842610.9441252,"logger":"kubebuilder.controller","msg":"Starting EventSource","controller":"kibana-controller","source":"kind source: /, Kind="}
{"level":"info","ts":1590842610.9442868,"logger":"kubebuilder.controller","msg":"Starting EventSource","controller":"elasticsearch-controller","source":"kind source: /, Kind="}
{"level":"info","ts":1590842610.9444468,"logger":"kubebuilder.controller","msg":"Starting EventSource","controller":"proxyconfig-controller","source":"kind source: /, Kind="}
{"level":"info","ts":1590842610.9445403,"logger":"kubebuilder.controller","msg":"Starting EventSource","controller":"kibanasecret-controller","source":"kind source: /, Kind="}
{"level":"info","ts":1590842610.944699,"logger":"kubebuilder.controller","msg":"Starting EventSource","controller":"trustedcabundle-controller","source":"kind source: /, Kind="}
{"level":"info","ts":1590842611.161969,"logger":"metrics","msg":"Metrics Service object created","Service.Name":"elasticsearch-operator","Service.Namespace":"openshift-operators-redhat"}
{"level":"info","ts":1590842611.1620052,"logger":"cmd","msg":"This operator no longer honors the image specified by the custom resources so that it is able to properly coordinate the configuration with the image."}
{"level":"info","ts":1590842611.1620126,"logger":"cmd","msg":"Starting the Cmd."}
{"level":"info","ts":1590842611.962236,"logger":"kubebuilder.controller","msg":"Starting Controller","controller":"kibana-controller"}
{"level":"info","ts":1590842611.9622269,"logger":"kubebuilder.controller","msg":"Starting Controller","controller":"elasticsearch-controller"}
{"level":"info","ts":1590842611.9622416,"logger":"kubebuilder.controller","msg":"Starting Controller","controller":"kibanasecret-controller"}
{"level":"info","ts":1590842611.9622273,"logger":"kubebuilder.controller","msg":"Starting Controller","controller":"trustedcabundle-controller"}
{"level":"info","ts":1590842611.962234,"logger":"kubebuilder.controller","msg":"Starting Controller","controller":"proxyconfig-controller"}
{"level":"info","ts":1590842612.062404,"logger":"kubebuilder.controller","msg":"Starting workers","controller":"kibanasecret-controller","worker count":1}
{"level":"info","ts":1590842612.0624251,"logger":"kubebuilder.controller","msg":"Starting workers","controller":"kibana-controller","worker count":1}
{"level":"info","ts":1590842612.0623937,"logger":"kubebuilder.controller","msg":"Starting workers","controller":"proxyconfig-controller","worker count":1}
{"level":"info","ts":1590842612.062401,"logger":"kubebuilder.controller","msg":"Starting workers","controller":"elasticsearch-controller","worker count":1}
{"level":"info","ts":1590842612.0624096,"logger":"kubebuilder.controller","msg":"Starting workers","controller":"trustedcabundle-controller","worker count":1}
{"level":"error","ts":1590842612.0625966,"logger":"kubebuilder.controller","msg":"Reconciler error","controller":"proxyconfig-controller","request":"/cluster","error":"skipping proxy config reconciliation in \"\": failed to find elasticsearch instance in \"\": empty result set","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/src/github.com/openshift/elasticsearch-operator/vendor/github.com/go-logr/zapr/zapr.go:128\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/src/github.com/openshift/elasticsearch-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:217\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1\n\t/go/src/github.com/openshift/elasticsearch-operator/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:158\nk8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\t/go/src/github.com/openshift/elasticsearch-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/src/github.com/openshift/elasticsearch-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:134\nk8s.io/apimachinery/pkg/util/wait.Until\n\t/go/src/github.com/openshift/elasticsearch-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88"}


Version-Release number of selected component (if applicable):
elasticsearch-operator.4.5.0-202005291637

How reproducible:
always

Steps to Reproduce:
1. Deploy clusterlogging and elasiticsearch operators.
2. Check the EO pod logs.
3.

Actual results:


Expected results:


Additional info:

Comment 1 Anping Li 2020-06-02 06:57:44 UTC
Reboot CLO pod should be a workaround.

Comment 2 Periklis Tsirakidis 2020-06-02 07:18:32 UTC
@Anping

Nope the errors are per se indicators that we cannot reconcile a Kibana CR in an Elasticsearch is not present. However, you are right they should be omitted for the proxyconfig reconciliation.

Comment 5 Anping Li 2020-06-08 11:53:55 UTC
Verified on 4.6.0

Comment 7 errata-xmlrpc 2020-10-27 16:01:56 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.6 GA Images), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:4196