Bug 1519622

Summary: Disable DC rollback behaviors for logging project DCs
Product: OpenShift Container Platform Reporter: Peter Portante <pportant>
Component: LoggingAssignee: Jeff Cantrill <jcantril>
Status: CLOSED ERRATA QA Contact: Anping Li <anli>
Severity: high Docs Contact:
Priority: high    
Version: 3.7.0CC: aos-bugs, rmeggins
Target Milestone: ---Keywords: OpsBlocker
Target Release: 3.7.z   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Enhancement
Doc Text:
Feature: Modify Elasticsearch deploymentconfigs to disable Openshift rollback behavior Reason: Upgrades of the logging stack that do not result in readiness probe succeeding are rolled back to the last previous successful deployment. This can result in the deployed instances being out-of-date with their configuration and old images Result: This changes will make it so the deployments will not be rolled back and operators can manually intervene without the image and configuration mismatches.
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-01-23 17:58:09 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Peter Portante 2017-12-01 03:41:12 UTC
It seems we consider disabling the auto rollback to a "working" DC version when a rollout fails.

We can do that by adding, "revisionHistoryLimit: 0" to all logging project DCs.

The problem scenario is as follows:

  1. DC version X depends on configmap "version" A
  2. An upgrade is performed modifying the configmap to "version" B, and DC to version Y
  3. Version of Y of the DC is rolled out, but fails for some reason
  4. Because of the rollout failure, the DC rolls back to version X
  5. Now DC version X fails because the corresponding configmap "version" A is no longer present

Comment 1 openshift-github-bot 2017-12-08 19:17:01 UTC
Commits pushed to master at https://github.com/openshift/openshift-ansible

https://github.com/openshift/openshift-ansible/commit/415fc54d7b44be4a1ee3cc63233c3bd8b031bdb2
bug 1519622. Disable rollback of ES DCs

https://github.com/openshift/openshift-ansible/commit/7b4c6781a680175fa985edab92c73cb3fc662073
Merge pull request #6356 from jcantrill/1519622_disable_revision_history

bug 1519622. Disable rollback of ES DCs

Comment 3 Anping Li 2017-12-19 02:43:02 UTC
This fix is not merged to v3.7. and please move the bug to a installer errata too.

Comment 4 Anping Li 2018-01-11 04:37:50 UTC
Please ignore comment 3, the fix is in openshift-ansible-3.7.22-1.git.0.a938a52.el7.noarch. The errata include both installer and package now.

Comment 5 Anping Li 2018-01-12 02:05:40 UTC
The test is blocked by 1533313

Comment 6 Anping Li 2018-01-15 04:40:14 UTC
Verified with openshift-ansible-3.7.23.

Comment 9 errata-xmlrpc 2018-01-23 17:58:09 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:0113