Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1983829

Summary:	ovn-kubernetes upgrade jobs are failing disruptive tests
Product:	OpenShift Container Platform	Reporter:	Stephen Benjamin <stbenjam>
Component:	Networking	Assignee:	Nadia Pinaeva <npinaeva>
Networking sub component:	ovn-kubernetes	QA Contact:	Anurag saxena <anusaxen>
Status:	CLOSED DEFERRED	Docs Contact:
Severity:	high
Priority:	unspecified	CC:	aconstan, bbennett, wking
Version:	4.9
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:	job=periodic-ci-openshift-release-master-ci-4.9-upgrade-from-stable-4.8-e2e-azure-ovn-upgrade=all job=periodic-ci-openshift-release-master-ci-4.9-upgrade-from-stable-4.8-e2e-aws-ovn-upgrade=all [sig-api-machinery] Kubernetes APIs remain available for new connections [sig-api-machinery] OAuth APIs remain available for new connections [sig-api-machinery] OpenShift APIs remain available with reused connections [sig-network-edge] Cluster frontend ingress remain available
Last Closed:	2021-09-20 13:25:54 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Stephen Benjamin 2021-07-19 22:56:02 UTC

Description of problem:

OVN CI upgrade jobs are often failing the following tests. Azure and AWS platforms currently allow no disruptions during an upgrade. GCP should as well but it has it's own issues (BZ1983758).


[sig-network-edge] Cluster frontend ingress remain available
[sig-imageregistry] Image registry remain available
[sig-api-machinery] Kubernetes APIs remain available for new 
[sig-api-machinery] OpenShift APIs remain available for new connections
[sig-api-machinery] OAuth APIs remain available for new connections
[sig-api-machinery] Kubernetes APIs remain available with reused connections
[sig-api-machinery] OpenShift APIs remain available with reused connections
[sig-api-machinery] OAuth APIs remain available with reused connections


Azure: https://testgrid.k8s.io/redhat-openshift-ocp-release-4.9-informing#periodic-ci-openshift-release-master-ci-4.9-upgrade-from-stable-4.8-e2e-azure-ovn-upgrade

AWS: https://testgrid.k8s.io/redhat-openshift-ocp-release-4.9-informing#periodic-ci-openshift-release-master-ci-4.9-upgrade-from-stable-4.8-e2e-gcp-ovn-upgrade



Version-Release number of selected component (if applicable):
4.8 -> 4.9 upgrades

How reproducible:
Always

Steps to Reproduce:
1. Perform an upgrade, or look at a CI job

Actual results:
API and ingress frequently fail disruptive tests checking availability

Expected results:
API endpoints and ingress are not disrupted.

Additional info:

Comment 1 Nadia Pinaeva 2021-08-17 09:44:00 UTC

Now most of mentioned tests seem to be fixed by https://bugzilla.redhat.com/show_bug.cgi?id=1970985 bugfix.
But there is a new constantly failing test - "[sig-mco] Machine config pools complete upgrade" for azure and gcp. I will have a look at that new failure.

Please check current state of the tests and update a list of often failing tests you'd like to be fixed

Comment 2 Nadia Pinaeva 2021-08-20 10:31:56 UTC

Some failures (especially "[sig-mco] Machine config pools complete upgrade" one) also may be related to wrong initial release since 07-14, it should be fixed by https://github.com/openshift/release/pull/21257, then we will see if something in TestGrid changes

Comment 3 Nadia Pinaeva 2021-09-20 13:25:54 UTC

Since this bug is too broad, and we created a whole Epic to improve upgrade jobs stability https://issues.redhat.com/browse/SDN-2164, I close it as DEFERRED