Bug 2049749

Summary:	Cluster upgrades to 4.10 exhibits slowness and instability on s390x
Product:	OpenShift Container Platform	Reporter:	Lakshmi Ravichandran <lakshmi.ravichandran1>
Component:	Multi-Arch	Assignee:	Jeremy Poulin <jpoulin>
Status:	CLOSED DUPLICATE	QA Contact:	Douglas Slavens <dslavens>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	4.10	CC:	aos-bugs, danili, eparis
Target Milestone:	---
Target Release:	---
Hardware:	s390x
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2022-02-08 18:56:22 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Lakshmi Ravichandran 2022-02-02 16:23:18 UTC

Description of problem:

cluster upgrades suffer from two types of problems starting from 4.9 (4.9 to 4.10, 4.10 to 4.11):

1. cluster upgrades are observed to take a longer time (more than 75 mins) 
2. cluster instability and API timeouts of primary operators are observed which fails the upgrade tests.
    Kubernetes API timeout
    OpenShift API timeout
    OAuth API timeout
    Console becomes unavailable
    Image-registry becomes unavailable 

Version-Release number of selected component (if applicable):
4.10

How reproducible:
upgrading an existing ocp cluster to recent 4.10 version

Steps to Reproduce:
-

Actual results:
API timeouts and cluster upgrade takes long time during upgrade.
cluster becomes unstable during upgrade.

Expected results:
cluster upgrade should be smooth within the stipulated time.

Additional info:
observed in CI environment initially.
https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/periodic-ci-openshift-multiarch-master-nightly-4.10-upgrade-from-nightly-4.9-ocp-remote-libvirt-s390x/1488588046591856640

Comment 1 Dan Li 2022-02-07 21:00:49 UTC

This is similar to BZ 2049750, we are currently getting more information about the bug to make an evaluation.

Comment 2 Dan Li 2022-02-08 18:56:22 UTC

Closing this bug as it is a duplicate of BZ 2049750

*** This bug has been marked as a duplicate of bug 2049750 ***