Bug 1962219 - NTO relies on unreliable leader-for-life implementation.
Summary: NTO relies on unreliable leader-for-life implementation.
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Node Tuning Operator
Version: 4.8
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 4.8.0
Assignee: Jiří Mencák
QA Contact: Simon
URL:
Whiteboard:
Depends On:
Blocks: 1958885
TreeView+ depends on / blocked
 
Reported: 2021-05-19 14:17 UTC by Jiří Mencák
Modified: 2021-07-27 23:09 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-07-27 23:09:14 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift cluster-node-tuning-operator pull 229 0 None open Bug 1962219: Switch to client-go leader-with-lease election. 2021-05-19 14:18:04 UTC
Red Hat Product Errata RHSA-2021:2438 0 None None None 2021-07-27 23:09:36 UTC

Description Jiří Mencák 2021-05-19 14:17:24 UTC
Description of problem:
The Node Tuning Operator (alongside with a few other SLOs) currently uses leader-for-life implementation from operator-sdk, which already caused several known upgrade failures (rhbz#1916865, rhbz#1958885) due to ConfigMap lock not being removed by the k8s garbage collection mechanism.

Version-Release number of selected component (if applicable):
All currently supported versions.

How reproducible:
Rare, but serious as it causes cluster upgrade failures.

Steps to Reproduce:
This is unfortunately hard to reproduce.

Actual results:
NTO stuck waiting for a ConfigMap lock to be removed.

Expected results:
NTO should not rely on the k8s GC mechanism for leader election.

Additional info:
Upstream fix: https://github.com/openshift/cluster-node-tuning-operator/pull/229
Related BZs:
https://bugzilla.redhat.com/show_bug.cgi?id=1916865
https://bugzilla.redhat.com/show_bug.cgi?id=1958885

Comment 2 Simon 2021-05-26 16:40:25 UTC
Cluster version: 4.8.0-0.nightly-2021-05-26-071911

Impossible to reproduce.

Regression tests 100% passed.

Comment 5 errata-xmlrpc 2021-07-27 23:09:14 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.8.2 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:2438


Note You need to log in before you can comment on or make changes to this bug.