Bug 1949348

Summary: On GCP, load balancers report kube-apiserver fails its /readyz check 50% of the time, causing load balancer backend churn and disruptions to apiservers
Product: OpenShift Container Platform Reporter: OpenShift BugZilla Robot <openshift-bugzilla-robot>
Component: kube-apiserverAssignee: Antonio Ojea <aojeagar>
Status: CLOSED ERRATA QA Contact: Ke Wang <kewang>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: 4.7CC: akashem, aojeagar, aos-bugs, dcbw, mfojtik, mgugino, miabbott, sdodson, sttts, wking, wlewis, xxia
Target Milestone: ---   
Target Release: 4.7.z   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: tag-ci
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Cause: Google Cloud Loadbalancer healthcheckers leave stale conntrack entries on the hosts Consequence: Stale conntrack entries cause network interruptions to the apiserver traffic using the GCP loadbalancers Fix: Don't allow healthcheck traffic to loop through the host Result: No network disruption against the apiserver
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-06-29 04:19:47 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1925698    
Bug Blocks: 1930457    

Comment 9 Antonio Ojea 2021-06-03 22:46:39 UTC
*** Bug 1966595 has been marked as a duplicate of this bug. ***

Comment 13 Ke Wang 2021-06-04 09:16:15 UTC
This bug's PR is dev-approved and not yet merged, so I'm following issue DPTP-660 to do the pre-merge verifying for QE pre-merge verification goal of issue OCPQE-815 by using the bot to launch a cluster with the open PR. The verification steps see Comment #6 and Comment #7. So the bug is pre-merge verified. After the PR gets merged, the bug will be moved to VERIFIED by the bot automatically or, if not working, by me manually.

Comment 14 Siddharth Sharma 2021-06-04 18:38:47 UTC
This bug will be shipped as part of next z-stream release 4.7.15 on June 14th, as 4.7.14 was dropped due to a regression https://bugzilla.redhat.com/show_bug.cgi?id=1967614

Comment 18 Ke Wang 2021-06-15 06:40:42 UTC
The PR has been landed into 4.7.0-0.nightly-2021-06-12-151209 nightly release and the bug has been verified via pre-merge Comment #6 and Comment #7. but the bot likely did not move it to "verified". Hence manually the appropriate state.

Comment 19 OpenShift Automated Release Tooling 2021-06-17 12:29:08 UTC
OpenShift engineering has decided to not ship Red Hat OpenShift Container Platform 4.7.17 due a regression https://bugzilla.redhat.com/show_bug.cgi?id=1973006. All the fixes which were part of 4.7.17 will be now part of 4.7.18 and planned to be available in candidate channel on June 23 2021 and in fast channel on June 28th.

Comment 23 errata-xmlrpc 2021-06-29 04:19:47 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.7.18 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:2502