Bug 1680115

Summary:	node-exporter daemonset can not be updated with tolerances for tainted nodes
Product:	OpenShift Container Platform	Reporter:	Jose Perez Espinoza <jperezes>
Component:	Monitoring	Assignee:	Frederic Branczyk <fbranczy>
Status:	CLOSED ERRATA	QA Contact:	Junqi Zhao <juzhao>
Severity:	high	Docs Contact:
Priority:	high
Version:	3.11.0	CC:	mloibl, palonsor, surbania
Target Milestone:	---
Target Release:	4.1.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2019-06-04 10:44:26 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Jose Perez Espinoza 2019-02-22 18:03:14 UTC

Description of problem:
We have added some taints to nodes with the effect "NoExecute" which evicts any pod without a matching tolerance. The node-exporter daemonset is not able to be updated with new tolerances because the cluster-monitoring-operator reverts it back to its out of the box config. The current tolerance applied to node-exporter does not prevent the pod from being evicted if any taints are applied.

Version-Release number of selected component (if applicable):
OCP 3.11.59

How reproducible:
Always, part of default configuration.

Expected results:
node-exporter daemonset should come configured with a tolerance that prevents taints on a node to evict them. The sdn daemonsets accomplish this by having the following tolerance.

tolerations:
- operator: Exists

Additional info:

Comment 1 Frederic Branczyk 2019-02-25 09:11:31 UTC

The upstream pull request has been opened to tolerate any taint: https://github.com/coreos/prometheus-operator/pull/2429. Once merged this will need to trickle down to the releases. It's unclear whether we can backport this to 3.11, but this will definitely land in 4.0.

Comment 2 Frederic Branczyk 2019-02-26 14:27:37 UTC

https://github.com/openshift/cluster-monitoring-operator/pull/269 pulls the required changes into cluster-monitoring to fix this.

Comment 3 Frederic Branczyk 2019-02-27 10:51:06 UTC

The changes have been merged. Moving to modified.

Comment 9 errata-xmlrpc 2019-06-04 10:44:26 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:0758