Bug 1680115 - node-exporter daemonset can not be updated with tolerances for tainted nodes
Summary: node-exporter daemonset can not be updated with tolerances for tainted nodes
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Monitoring
Version: 3.11.0
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 4.1.0
Assignee: Frederic Branczyk
QA Contact: Junqi Zhao
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-02-22 18:03 UTC by Jose Perez Espinoza
Modified: 2020-04-17 04:44 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-06-04 10:44:26 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2019:0758 0 None None None 2019-06-04 10:44:33 UTC

Description Jose Perez Espinoza 2019-02-22 18:03:14 UTC
Description of problem:
We have added some taints to nodes with the effect "NoExecute" which evicts any pod without a matching tolerance. The node-exporter daemonset is not able to be updated with new tolerances because the cluster-monitoring-operator reverts it back to its out of the box config. The current tolerance applied to node-exporter does not prevent the pod from being evicted if any taints are applied.

Version-Release number of selected component (if applicable):
OCP 3.11.59

How reproducible:
Always, part of default configuration.

Expected results:
node-exporter daemonset should come configured with a tolerance that prevents taints on a node to evict them. The sdn daemonsets accomplish this by having the following tolerance.

tolerations:
- operator: Exists

Additional info:

Comment 1 Frederic Branczyk 2019-02-25 09:11:31 UTC
The upstream pull request has been opened to tolerate any taint: https://github.com/coreos/prometheus-operator/pull/2429. Once merged this will need to trickle down to the releases. It's unclear whether we can backport this to 3.11, but this will definitely land in 4.0.

Comment 2 Frederic Branczyk 2019-02-26 14:27:37 UTC
https://github.com/openshift/cluster-monitoring-operator/pull/269 pulls the required changes into cluster-monitoring to fix this.

Comment 3 Frederic Branczyk 2019-02-27 10:51:06 UTC
The changes have been merged. Moving to modified.

Comment 9 errata-xmlrpc 2019-06-04 10:44:26 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:0758


Note You need to log in before you can comment on or make changes to this bug.