Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1432250

Summary:	Kibana Unable to Connect to ElasticSearch
Product:	OpenShift Container Platform	Reporter:	Steven Walter <stwalter>
Component:	Logging	Assignee:	Jeff Cantrill <jcantril>
Status:	CLOSED ERRATA	QA Contact:	Xia Zhao <xiazhao>
Severity:	urgent	Docs Contact:
Priority:	unspecified
Version:	3.4.1	CC:	aos-bugs, bvincell, dlbewley, jcantril, knakayam, misalunk, ndordet, pdwyer, pportant, rkharwar, rromerom, stwalter
Target Milestone:	---
Target Release:	3.4.z
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:	Cause: The index used by the SG plugin is not seeded properly Consequence: The ES node is in a waiting state for the SG ACLs and doesnt service any requests Fix: Moved the initial seeding logic to when the server starts in its run.sh Result: The initial SG documents are seeded and the plugin services requests.	Story Points:	---
Clone Of:		Environment:
Last Closed:	2017-04-04 14:28:53 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Steven Walter 2017-03-14 22:12:23 UTC

Description of problem:
Kibana is dropping and reconnecitng with ES.  Message in the console is "Shard Failure: The following shards failed:" but no shards listed. The elasticsearch logs look healthy. The fluentd logs are noisy but no errors.

Version-Release number of selected component (if applicable):
3.4.1

How reproducible:
Not reproduced, but multiple reports

Uploading more info in private comments

Comment 14 openshift-github-bot 2017-03-17 13:03:29 UTC

Commit pushed to master at https://github.com/openshift/origin-aggregated-logging

https://github.com/openshift/origin-aggregated-logging/commit/85f2eb56e8d6ed49f820dda63a4e6e2c8dd90246
bug 1432250. fix throttle settings for json-file driver

Comment 15 Jeff Cantrill 2017-03-24 13:29:05 UTC

My apologize for not providing info regarding this build earlier.  It should already be pushed to the ops registries, but the details are:

Please validate with this image: 

12835998 buildContainer (noarch) completed successfully
koji_builds:
  https://brewweb.engineering.redhat.com/brew/buildinfo?buildID=545812
repositories:
  brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/openshift3/logging-elasticsearch:rhaos-3.4-rhel-7-docker-candidate-20170322162429
  brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/openshift3/logging-elasticsearch:3.4.1-15
  brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/openshift3/logging-elasticsearch:3.4.1
  brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/openshift3/logging-elasticsearch:latest
  brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/openshift3/logging-elasticsearch:v3.4

Comment 17 Xia Zhao 2017-03-29 09:52:08 UTC

Testing in progress:  installed OCP 3.4.1 env and deployed logging 3.4.1 stacks there, since this bz don't have clear repro steps, QE attempt to do some sanity tests on images containing bug fixes, and keep on monitoring the kibana/es stacks for longer time to see if the original bug reproduced. Will update test result later soon.

Comment 20 errata-xmlrpc 2017-04-04 14:28:53 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:0865