Bug 1655675

Summary:	Define Elasticsearch DC recreate timeout to avoid premature rollbacks
Product:	OpenShift Container Platform	Reporter:	Jeff Cantrill <jcantril>
Component:	Logging	Assignee:	Jeff Cantrill <jcantril>
Status:	CLOSED ERRATA	QA Contact:	Anping Li <anli>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	3.11.0	CC:	aos-bugs, hekumar, jgoulding, rmeggins
Target Milestone:	---	Keywords:	OpsBlocker
Target Release:	3.11.z
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Enhancement
Doc Text:	Feature: Define the recreate strategy timeout for Elasticsearch Reason: There are examples on AWS Openshift clusters where rollout of new ES pods fail because the cluster is having issues attaching storage. Defining a long recreate timeout allows the the cluster more time to attach storage to the new pod Result: Elasticsearch pods have more time to restart and experience fewer rollbacks	Story Points:	---
Clone Of:		Environment:
Last Closed:	2019-01-10 09:04:12 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Jeff Cantrill 2018-12-03 16:37:23 UTC

Description of problem:

Elasticsearch nodes are rolled back to previous versions if they are unable to achieve a ready state within the default period of 600s.  Need to set the value to 1800 to avoid rollback



Additional info:  Advised it can take upwards of 30 or 40 minutes for AWS backed clusters to attach storage

Comment 1 Rich Megginson 2018-12-03 17:31:41 UTC

Is there a policy that says "never, ever rollback unless I explicitly tell you to rollback"?

Comment 3 Anping Li 2018-12-21 09:21:40 UTC

The fix in ose-ansible:v3.11.59

Comment 5 errata-xmlrpc 2019-01-10 09:04:12 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:0024