1792033 – Updating node-exporter failed: reconciling node-exporter DaemonSet failed

Bug 1792033 - Updating node-exporter failed: reconciling node-exporter DaemonSet failed

Summary: Updating node-exporter failed: reconciling node-exporter DaemonSet failed

Keywords:
Status:	CLOSED CURRENTRELEASE
Alias:	None
Product:	OpenShift Container Platform
Classification:	Red Hat
Component:	Node
Sub Component:
Version:	4.4
Hardware:	Unspecified
OS:	Unspecified
Priority:	unspecified
Severity:	unspecified
Target Milestone:	---
Target Release:	4.4.0
Assignee:	Ryan Phillips
QA Contact:	Sunil Choudhary
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2020-01-16 22:01 UTC by Kirsten Garrison
Modified:	2023-12-15 17:11 UTC (History)
CC List:	13 users (show)
Fixed In Version:
Doc Type:	No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed:	2020-02-25 19:59:29 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Description Kirsten Garrison 2020-01-16 22:01:33 UTC

MCO is seeing in CI regularly a degraded node exporter in CI runs:

Example:
                  {
                        "lastTransitionTime": "2020-01-16T18:50:16Z",
                        "message": "Failed to rollout the stack. Error: running task Updating node-exporter failed: reconciling node-exporter DaemonSet failed: updating DaemonSet object failed: waiting for DaemonSetRollout of node-exporter: daemonset node-exporter is not ready. status: (desired: 6, updated: 6, ready: 4, unavailable: 2)",
                        "reason": "UpdatingnodeExporterFailed",
                        "status": "True",
                        "type": "Degraded"
                    },

Example runs:

https://storage.googleapis.com/origin-ci-test/pr-logs/pull/openshift_machine-config-operator/1356/pull-ci-openshift-machine-config-operator-master-e2e-gcp-op/844/artifacts/e2e-gcp-op/clusteroperators.json

https://storage.googleapis.com/origin-ci-test/pr-logs/pull/openshift_machine-config-operator/1376/pull-ci-openshift-machine-config-operator-master-e2e-gcp-op/846/artifacts/e2e-gcp-op/clusteroperators.json

https://storage.googleapis.com/origin-ci-test/pr-logs/pull/openshift_machine-config-operator/1356/pull-ci-openshift-machine-config-operator-master-e2e-gcp-op/835/artifacts/e2e-gcp-op/clusteroperators.json

Comment 1 Kirsten Garrison 2020-01-16 22:04:28 UTC

Also seeing outside of MCO:

https://storage.googleapis.com/origin-ci-test/logs/release-openshift-ocp-installer-e2e-vsphere-upi-4.4/441/artifacts/e2e-vsphere-upi/clusteroperators.json

https://storage.googleapis.com/origin-ci-test/logs/release-openshift-origin-installer-e2e-gcp-shared-vpc-4.3/463/artifacts/e2e-gcp/clusteroperators.json

Comment 3 Ryan Phillips 2020-02-25 19:59:29 UTC

Fixed by https://github.com/openshift/cluster-monitoring-operator/pull/645

Initcontainers should be privileged.

Note You need to log in before you can comment on or make changes to this bug.