Bug 1475119

Summary:	Elastic search not reachable due to [WARN ][com.floragunn.searchguard.ssl.transport.SearchGuardSSLNettyTransport] [Torrent] exception caught on transport layer [[id: 0x019a5237]], closing connection java.net.NoRouteToHostException: No route to host
Product:	OpenShift Container Platform	Reporter:	Miheer Salunke <misalunk>
Component:	Logging	Assignee:	Jeff Cantrill <jcantril>
Status:	CLOSED INSUFFICIENT_DATA	QA Contact:	Xia Zhao <xiazhao>
Severity:	urgent	Docs Contact:
Priority:	urgent
Version:	3.4.1	CC:	aos-bugs, jcantril, jwozniak, misalunk, nhosoi, pportant, rmeggins
Target Milestone:	---
Target Release:	3.4.z
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2017-10-10 19:15:56 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Miheer Salunke 2017-07-26 06:05:06 UTC

Created attachment 1304595 [details]
logging stack logs and details of the logging pods

Description of problem:

EFK stack not working. Elastic search id not reachable it seems. "plugin:elasticsearch Unable to connect to Elasticsearch at https://logging-es:9200."

in es logs I see ->

...[2017-07-24 02:10:41,109][WARN ][com.floragunn.searchguard.ssl.transport.SearchGuardSSLNettyTransport] [Torrent] exception caught on transport layer [[id: 0x019a5237]], closing connection
java.net.NoRouteToHostException: No route to host

May related to networking ? Any suggestions on how to troubleshoot this will be highly appreciated.


Details of the environment and issue attached.

Version-Release number of selected component (if applicable):
OCP 3.4.1

How reproducible:
Always on customer end

Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 3 Jan Wozniak 2017-07-27 08:25:52 UTC

Could be an issue in DNS resolution or maybe even networking. For start, have you tried:

1) querying elastic search by the IP (100.74.246.128) associated with logging-es service directly?
2) checking the /etc/resov.conf on the logging-es containers whether DNS resolution is correctly configured?

Comment 4 Peter Portante 2017-07-27 14:28:43 UTC

Bad master URL?

Comment 5 Jeff Cantrill 2017-07-27 14:52:26 UTC

https://docs.openshift.com/container-platform/3.4/admin_guide/sdn_troubleshooting.html#debugging-kubernetes

Comment 25 Peter Portante 2017-08-14 02:28:25 UTC

(In reply to Miheer Salunke from comment #0)
> EFK stack not working. Elastic search id not reachable it seems.
> "plugin:elasticsearch Unable to connect to Elasticsearch at
> https://logging-es:9200."

What log file did you find this log message?

Comment 27 Miheer Salunke 2017-08-14 04:05:04 UTC

{"type":"log","@timestamp":"2017-07-24T20:52:09+00:00","tags":["warning","elasticsearch"],"pid":1,"message":"No living connections"}
{"type":"log","@timestamp":"2017-07-24T20:52:09+00:00","tags":["status","plugin:elasticsearch","error"],"pid":1,"name":"plugin:elasticsearch","state":"red","message":"Status changed from red to red - Unable to connect to Elasticsearch at https://logging-es:9200.","prevState":"red","prevMsg":"Service Unavailable"}

Comment 28 Jeff Cantrill 2017-10-06 14:48:58 UTC

Miheer,

Based on the attachments it doesn't appear the issue still persists.  Can you confirm?  Can we close this issue?

Comment 29 Red Hat Bugzilla 2023-09-14 04:01:37 UTC

The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days