Bug 2099206

Summary: 120 node baremetal upgrade from 4.9.29 --> 4.10.13 crashloops on machine-approver
Product: OpenShift Container Platform Reporter: OpenShift BugZilla Robot <openshift-bugzilla-robot>
Component: NetworkingAssignee: Jaime Caamaño Ruiz <jcaamano>
Networking sub component: ovn-kubernetes QA Contact: Mike Fiedler <mifiedle>
Status: CLOSED ERRATA Docs Contact:
Severity: medium    
Priority: medium CC: dwilson, fbaudin, ffernand, jcaamano, mifiedle, rsevilla, surya, yprokule
Version: 4.10   
Target Milestone: ---   
Target Release: 4.10.z   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-07-25 07:07:10 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 2089392    
Bug Blocks:    

Comment 4 Mike Fiedler 2022-07-18 13:47:38 UTC
@dwilson Any chance you can verify on 4.10?

@jcaamano Any good way to verify this on a small scale cluster?

Comment 5 Jaime Caamaño Ruiz 2022-07-18 14:56:52 UTC
I don’t think so. This is strictly a scale bug and the problem won’t show at small sizes. Unless we are whiling to white box it and look into the logs or just limit the verification for regressions.

Comment 10 Mike Fiedler 2022-07-20 23:58:21 UTC
Verified on 4.10.23

1. installed 4.9.43 on AWS
2. scaled cluster up to 120 nodes
3. ran the node-density-ci-networkpolicy kube-burner workload for 2600 pods as described in https://bugzilla.redhat.com/show_bug.cgi?id=2089392#c2
4. upgraded to 4.10.23 which has the fix for this bz

Upgrade successful in about 1 hr 45 min.

Comment 12 errata-xmlrpc 2022-07-25 07:07:10 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: OpenShift Container Platform 4.10.24 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:5664

Comment 13 Red Hat Bugzilla 2023-09-15 01:56:02 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 365 days