Bug 1906986 - Ensure failed pod adds are retried even if the pod object doesn't change
Summary: Ensure failed pod adds are retried even if the pod object doesn't change
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Networking
Version: 4.6
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 4.7.0
Assignee: Dan Williams
QA Contact: Ross Brattain
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-12-12 05:04 UTC by Dan Williams
Modified: 2021-02-24 15:43 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-02-24 15:43:00 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift ovn-kubernetes pull 384 0 None closed Bug 1906986: 12-11-20 merge 2021-02-11 19:40:11 UTC
Red Hat Product Errata RHSA-2020:5633 0 None None None 2021-02-24 15:43:26 UTC

Description Dan Williams 2020-12-12 05:04:42 UTC
If a pod add fails in ovnkube-master, and there is no subsequent update
event for the pod we never try to add it again and CNI will just
continuously fail. A prime example of this is if a pod is added for a
node, who has not yet registered with ovnkube-master by setting its L3
gateway config.

Comment 5 Ross Brattain 2021-01-06 03:59:18 UTC
I see retry notices in the logs after cluster bring up, so it looks like the retry is happening

2021-01-05T15:06:19.032219891Z I0105 15:06:19.032198       1 ovn.go:574] [3313f8eb-228c-4462-bc3a-e53a831fdc20/openshift-apiserver/apiserver-6888ff4744-cl7v5] retry pod setup
2021-01-05T15:06:19.032219891Z I0105 15:06:19.032205       1 ovn.go:580] [3313f8eb-228c-4462-bc3a-e53a831fdc20/openshift-apiserver/apiserver-6888ff4744-cl7v5] setup retry failed; will try again later
2021-01-05T15:10:19.033712862Z I0105 15:10:19.033649       1 ovn.go:574] [69d3db07-c2c1-4cfa-a919-9119fb355531/openshift-apiserver/apiserver-6888ff4744-824zf] retry pod setup
2021-01-05T15:10:19.033712862Z I0105 15:10:19.033685       1 ovn.go:580] [69d3db07-c2c1-4cfa-a919-9119fb355531/openshift-apiserver/apiserver-6888ff4744-824zf] setup retry failed; will try again later
2021-01-05T15:10:19.033712862Z I0105 15:10:19.033697       1 ovn.go:574] [cdb4f7b4-f3b8-4040-b9cd-7d65d013da5b/openshift-etcd/etcd-quorum-guard-54d9fcbb68-267pd] retry pod setup
2021-01-05T15:10:19.033712862Z I0105 15:10:19.033702       1 ovn.go:580] [cdb4f7b4-f3b8-4040-b9cd-7d65d013da5b/openshift-etcd/etcd-quorum-guard-54d9fcbb68-267pd] setup retry failed; will try again later

Verified on 4.7.0-0.nightly-2021-01-05-055003

Comment 8 errata-xmlrpc 2021-02-24 15:43:00 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.7.0 security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:5633


Note You need to log in before you can comment on or make changes to this bug.