Description of problem: After deploying logging stack, fluentd and kibana are not able to resolve elasticsearch hostname Fluentd logs : 2021-03-28T04:25:16.095607233Z 2021-03-28 04:25:16 +0000 [warn]: [default] failed to flush the buffer. retry_time=3 next_retry_seconds=2021-03-28 04:25:20 +0000 chunk="5be8a1fd530465ad135bed371a2c729a" error_class=Fluent::Plugin::ElasticsearchOutput::RecoverableRequestFailure error="could not push logs to Elasticsearch cluster ({:host=>\"elasticsearch.openshift-logging.svc.cluster.local\", :port=>9200, :scheme=>\"https\"}): Couldn't resolve host name" Kibana logs: 2021-03-28T04:24:50.963666757Z {"type":"log","@timestamp":"2021-03-28T04:24:50Z","tags":["warning","elasticsearch","admin"],"pid":117,"message":"No living connections"} 2021-03-28T04:24:53.469808289Z {"type":"log","@timestamp":"2021-03-28T04:24:53Z","tags":["warning","elasticsearch","admin"],"pid":117,"message":"Unable to revive connection: https://elasticsearch.openshift-logging.svc.cluster.local:9200/"} Version-Release number of selected component (if applicable): 4.7 How reproducible: 100% Steps to Reproduce: 1. Use internal domain for RHOCP 4.7 2. Install Red Hat OpenShift Logging 3. Check fluentd and kibana pod logs Actual results: Fluentd and kibana are not able to resolve elasticsearch's hostname Expected results: Fluentd and kibana should be able to resolve elasticsearch hostname Additional info:
Hi Team Can you share the steps to workaround the issue ? I will get it updated in the KCS solution.
Change the clusterdomain to dev.test in two steps 1) update clusterDomain in nodes /etc/kubernetes/kubelet.conf 2) update CoreDNS configmap to serve the new clusterDomain. Verified on clusterlogging.4.6.0-202104300142.p0,elasticsearch-operator.4.6.0-202104302046.p0.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (OpenShift Container Platform 4.6.28 extras update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2021:1489