1880449 – Upgrade 4.5 nightly to 4.6 nightly fails on control plane

Bug 1880449 - Upgrade 4.5 nightly to 4.6 nightly fails on control plane

Summary: Upgrade 4.5 nightly to 4.6 nightly fails on control plane

Keywords:
Status:	CLOSED DUPLICATE of bug 1880591
Alias:	None
Product:	OpenShift Container Platform
Classification:	Red Hat
Component:	Networking
Sub Component:
Version:	4.6
Hardware:	Unspecified
OS:	Unspecified
Priority:	unspecified
Severity:	unspecified
Target Milestone:	---
Target Release:	---
Assignee:	Ben Bennett
QA Contact:	Anurag saxena
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2020-09-18 15:11 UTC by Paige Rubendall
Modified:	2020-09-22 12:46 UTC (History)
CC List:	5 users (show)
Fixed In Version:
Doc Type:	If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:	2020-09-22 12:46:56 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Description Paige Rubendall 2020-09-18 15:11:13 UTC

Description of problem:
Unable to upgrade from 4.5.0-0.nightly-2020-09-17-145245 to 4.6.0-0.nightly-2020-09-17-195238

Version-Release number of selected component (if applicable): 4.6


How reproducible: 100%


Steps to Reproduce:
1. Create 4.5.0-0.nightly-2020-09-17-145245
2. Upgrade to 4.6.0-0.nightly-2020-09-17-195238

Actual results:
Unable to apply 4.6.0-0.nightly-2020-09-17-195238: the control plane is reporting an internal error

Expected results:
Upgrade is successful

Additional info:
#oc get co:NAME                                       VERSION                             AVAILABLE   PROGRESSING   DEGRADED   SINCE
[2020-09-18T06:49:05.103Z] authentication                             4.6.0-0.nightly-2020-09-17-195238   True        False         False      90m
[2020-09-18T06:49:05.103Z] cloud-credential                           4.6.0-0.nightly-2020-09-17-195238   True        False         False      4h11m
[2020-09-18T06:49:05.103Z] cluster-autoscaler                         4.6.0-0.nightly-2020-09-17-195238   True        False         False      3h53m
[2020-09-18T06:49:05.103Z] config-operator                            4.6.0-0.nightly-2020-09-17-195238   True        False         False      3h53m
[2020-09-18T06:49:05.103Z] console                                    4.6.0-0.nightly-2020-09-17-195238   True        False         False      102m
[2020-09-18T06:49:05.103Z] csi-snapshot-controller                    4.6.0-0.nightly-2020-09-17-195238   True        False         False      3h46m
[2020-09-18T06:49:05.103Z] dns                                        4.5.0-0.nightly-2020-09-17-145245   True        True          False      3h59m
[2020-09-18T06:49:05.103Z] etcd                                       4.6.0-0.nightly-2020-09-17-195238   True        False         False      3h58m
[2020-09-18T06:49:05.103Z] image-registry                             4.6.0-0.nightly-2020-09-17-195238   True        False         False      3h47m
[2020-09-18T06:49:05.103Z] ingress                                    4.6.0-0.nightly-2020-09-17-195238   True        False         False      103m
[2020-09-18T06:49:05.103Z] insights                                   4.6.0-0.nightly-2020-09-17-195238   True        False         False      3h55m
[2020-09-18T06:49:05.103Z] kube-apiserver                             4.6.0-0.nightly-2020-09-17-195238   True        True          False      3h57m
[2020-09-18T06:49:05.103Z] kube-controller-manager                    4.6.0-0.nightly-2020-09-17-195238   True        False         False      3h58m
[2020-09-18T06:49:05.103Z] kube-scheduler                             4.6.0-0.nightly-2020-09-17-195238   True        False         False      3h57m
[2020-09-18T06:49:05.104Z] kube-storage-version-migrator              4.6.0-0.nightly-2020-09-17-195238   False       False         False      92m
[2020-09-18T06:49:05.104Z] machine-api                                4.6.0-0.nightly-2020-09-17-195238   True        False         False      3h51m
[2020-09-18T06:49:05.104Z] machine-approver                           4.6.0-0.nightly-2020-09-17-195238   True        False         False      3h57m
[2020-09-18T06:49:05.104Z] machine-config                             4.5.0-0.nightly-2020-09-17-145245   True        False         False      3h58m
[2020-09-18T06:49:05.104Z] marketplace                                4.6.0-0.nightly-2020-09-17-195238   True        False         False      102m
[2020-09-18T06:49:05.104Z] monitoring                                 4.6.0-0.nightly-2020-09-17-195238   True        False         False      93m
[2020-09-18T06:49:05.104Z] network                                    4.5.0-0.nightly-2020-09-17-145245   True        True          True       4h1m
[2020-09-18T06:49:05.104Z] node-tuning                                4.6.0-0.nightly-2020-09-17-195238   True        False         False      102m
[2020-09-18T06:49:05.104Z] openshift-apiserver                        4.6.0-0.nightly-2020-09-17-195238   True        False         False      93m
[2020-09-18T06:49:05.104Z] openshift-controller-manager               4.6.0-0.nightly-2020-09-17-195238   True        False         False      101m
[2020-09-18T06:49:05.104Z] openshift-samples                          4.6.0-0.nightly-2020-09-17-195238   True        False         False      102m
[2020-09-18T06:49:05.104Z] operator-lifecycle-manager                 4.6.0-0.nightly-2020-09-17-195238   True        False         False      3h59m
[2020-09-18T06:49:05.104Z] operator-lifecycle-manager-catalog         4.6.0-0.nightly-2020-09-17-195238   True        False         False      3h59m
[2020-09-18T06:49:05.104Z] operator-lifecycle-manager-packageserver   4.6.0-0.nightly-2020-09-17-195238   True        False         False      101m
[2020-09-18T06:49:05.104Z] service-ca                                 4.6.0-0.nightly-2020-09-17-195238   True        False         False      4h
[2020-09-18T06:49:05.104Z] storage                                    4.6.0-0.nightly-2020-09-17-195238   True        True          False      95m

[2020-09-18T06:49:07.422Z] Name:         network
[2020-09-18T06:49:07.422Z] Namespace:    
[2020-09-18T06:49:07.422Z] Labels:       <none>
[2020-09-18T06:49:07.422Z] Annotations:  network.operator.openshift.io/last-seen-state:
[2020-09-18T06:49:07.422Z]                 {"DaemonsetStates":[{"Namespace":"openshift-multus","Name":"multus","LastSeenStatus":{"currentNumberScheduled":6,"numberMisscheduled":0,"d...
[2020-09-18T06:49:07.422Z] API Version:  config.openshift.io/v1
[2020-09-18T06:49:07.422Z] Kind:         ClusterOperator
[2020-09-18T06:49:07.422Z] Metadata:
[2020-09-18T06:49:07.422Z]   Creation Timestamp:  2020-09-18T02:37:15Z
[2020-09-18T06:49:07.422Z]   Generation:          1
[2020-09-18T06:49:07.422Z]   Managed Fields:
[2020-09-18T06:49:07.422Z]     API Version:  config.openshift.io/v1
[2020-09-18T06:49:07.422Z]     Fields Type:  FieldsV1
[2020-09-18T06:49:07.422Z]     fieldsV1:
[2020-09-18T06:49:07.422Z]       f:spec:
[2020-09-18T06:49:07.422Z]       f:status:
[2020-09-18T06:49:07.422Z]         .:
[2020-09-18T06:49:07.422Z]         f:extension:
[2020-09-18T06:49:07.422Z]     Manager:      cluster-version-operator
[2020-09-18T06:49:07.422Z]     Operation:    Update
[2020-09-18T06:49:07.422Z]     Time:         2020-09-18T02:37:15Z
[2020-09-18T06:49:07.422Z]     API Version:  config.openshift.io/v1
[2020-09-18T06:49:07.422Z]     Fields Type:  FieldsV1
[2020-09-18T06:49:07.422Z]     fieldsV1:
[2020-09-18T06:49:07.422Z]       f:metadata:
[2020-09-18T06:49:07.422Z]         f:annotations:
[2020-09-18T06:49:07.422Z]           .:
[2020-09-18T06:49:07.422Z]           f:network.operator.openshift.io/last-seen-state:
[2020-09-18T06:49:07.422Z]       f:status:
[2020-09-18T06:49:07.422Z]         f:conditions:
[2020-09-18T06:49:07.422Z]         f:relatedObjects:
[2020-09-18T06:49:07.422Z]         f:versions:
[2020-09-18T06:49:07.422Z]     Manager:         cluster-network-operator
[2020-09-18T06:49:07.422Z]     Operation:       Update
[2020-09-18T06:49:07.422Z]     Time:            2020-09-18T06:49:05Z
[2020-09-18T06:49:07.422Z]   Resource Version:  135920
[2020-09-18T06:49:07.422Z]   Self Link:         /apis/config.openshift.io/v1/clusteroperators/network
[2020-09-18T06:49:07.422Z]   UID:               d7009a98-589d-4a0a-8c5a-11b95e96e88f
[2020-09-18T06:49:07.422Z] Spec:
[2020-09-18T06:49:07.422Z] Status:
[2020-09-18T06:49:07.422Z]   Conditions:
[2020-09-18T06:49:07.422Z]     Last Transition Time:  2020-09-18T05:24:41Z
[2020-09-18T06:49:07.422Z]     Message:               DaemonSet "openshift-multus/multus" rollout is not making progress - last change 2020-09-18T05:15:38Z
[2020-09-18T06:49:07.422Z] DaemonSet "openshift-ovn-kubernetes/ovs-node" rollout is not making progress - last change 2020-09-18T05:17:26Z
[2020-09-18T06:49:07.422Z] DaemonSet "openshift-ovn-kubernetes/ovnkube-node" rollout is not making progress - last change 2020-09-18T05:14:17Z
[2020-09-18T06:49:07.422Z]     Reason:                RolloutHung
[2020-09-18T06:49:07.422Z]     Status:                True
[2020-09-18T06:49:07.422Z]     Type:                  Degraded
[2020-09-18T06:49:07.422Z]     Last Transition Time:  2020-09-18T02:46:20Z
[2020-09-18T06:49:07.422Z]     Status:                True
[2020-09-18T06:49:07.422Z]     Type:                  Upgradeable
[2020-09-18T06:49:07.422Z]     Last Transition Time:  2020-09-18T05:14:03Z
[2020-09-18T06:49:07.422Z]     Message:               DaemonSet "openshift-multus/multus" update is rolling out (2 out of 6 updated)
[2020-09-18T06:49:07.423Z] DaemonSet "openshift-multus/network-metrics-daemon" is not available (awaiting 1 nodes)
[2020-09-18T06:49:07.423Z] DaemonSet "openshift-ovn-kubernetes/ovs-node" update is rolling out (5 out of 6 updated)
[2020-09-18T06:49:07.423Z] DaemonSet "openshift-ovn-kubernetes/ovnkube-node" update is rolling out (1 out of 6 updated)
[2020-09-18T06:49:07.423Z] DaemonSet "openshift-ovn-kubernetes/ovnkube-node-metrics" is not available (awaiting 6 nodes)

Comment 2 Scott Dodson 2020-09-21 16:29:05 UTC

This likely self resolves but ovn-kubernetes and multus are the two components marked degraded so moving to Networking.

Comment 3 Ben Bennett 2020-09-22 12:46:56 UTC


*** This bug has been marked as a duplicate of bug 1880591 ***

Note You need to log in before you can comment on or make changes to this bug.