Bug 2108244 - [Retry mechanism OVNK] Completed pods are not deleted if kapi object is already gone post update event
Summary: [Retry mechanism OVNK] Completed pods are not deleted if kapi object is alrea...
Keywords:
Status: CLOSED DUPLICATE of bug 2117310
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Networking
Version: 4.11
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: 4.12.0
Assignee: Patryk Diak
QA Contact: Anurag saxena
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-07-18 16:42 UTC by Surya Seetharaman
Modified: 2022-08-17 10:02 UTC (History)
0 users

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-08-17 10:02:33 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Surya Seetharaman 2022-07-18 16:42:01 UTC
Description of problem:

CREATE:
2022-07-11T06:03:47.665492352Z I0711 06:03:47.665439       1 obj_retry.go:1364] Creating *v1.Pod test/hello-pod took: 727ns
2022-07-11T06:03:47.665768460Z I0711 06:03:47.665747       1 obj_retry.go:1364] Creating *factory.egressIPPod test/hello-pod took: 139.716µs
2022-07-11T06:03:47.672577508Z I0711 06:03:47.672538       1 pods.go:369] [test/hello-pod] creating logical port for pod on switch ip-10-0-53-79.us-east-2.compute.internal
2022-07-11T06:03:47.672847676Z I0711 06:03:47.672767       1 kube.go:73] Setting annotations map[k8s.ovn.org/pod-networks:{"default":{"ip_addresses":["10.129.2.31/23"],"mac_address":"0a:58:0a:81:02:1f","gateway_ips":["10.129.2.1"],"ip_address":"10.129.2.31/23","gateway_ip":"10.129.2.1"}}] on pod test/hello-pod
2022-07-11T06:03:47.719710933Z I0711 06:03:47.719672       1 pods.go:358] [test/hello-pod] addLogicalPort took 47.13852ms, libovsdb time 4.248335ms, annotation time: 42.362315ms
COMPLETED:
2022-07-11T06:08:19.254219737Z I0711 06:08:19.254168       1 obj_retry.go:1276] Detected object test/hello-pod of type *factory.egressIPPod in terminal state (e.g. completed) during update event: will remove it
WE DIDN'T DELETE ANYTHING KAPI GONE!
2022-07-11T06:08:19.266628736Z W0711 06:08:19.266585       1 obj_retry.go:1401] Unable to get *v1.Pod test/hello-pod from informer cache (perhaps it was already deleted?), skipping update: pod "hello-pod" not found
2022-07-11T06:08:19.266740988Z W0711 06:08:19.266712       1 obj_retry.go:1401] Unable to get *v1.Pod test/hello-pod from informer cache (perhaps it was already deleted?), skipping update: pod "hello-pod" not found
2022-07-11T06:08:19.266740988Z W0711 06:08:19.266734       1 obj_retry.go:1401] Unable to get *factory.egressIPPod test/hello-pod from informer cache (perhaps it was already deleted?), skipping update: pod "hello-pod" not found
IGNORE DELETE EVENT
2022-07-11T06:08:19.266773661Z I0711 06:08:19.266756       1 obj_retry.go:1503] Ignoring delete event for completed resource *v1.Pod test/hello-pod
2022-07-11T06:08:19.266773661Z I0711 06:08:19.266764       1 obj_retry.go:1503] Ignoring delete event for completed resource *factory.egressIPPod test/hello-pod
RECREATE NEW POD - ENDS UP WITH SAME IP AND STALE PORT
2022-07-11T06:08:20.555542658Z I0711 06:08:20.555498       1 obj_retry.go:1364] Creating *v1.Pod test/hello-pod took: 741ns
2022-07-11T06:08:20.563196606Z I0711 06:08:20.563156       1 obj_retry.go:1364] Creating *factory.egressIPPod test/hello-pod took: 7.549346ms
2022-07-11T06:08:20.566882323Z I0711 06:08:20.566842       1 pods.go:369] [test/hello-pod] creating logical port for pod on switch ip-10-0-53-79.us-east-2.compute.internal
2022-07-11T06:08:20.567060407Z I0711 06:08:20.567038       1 kube.go:73] Setting annotations map[k8s.ovn.org/pod-networks:{"default":{"ip_addresses":["10.129.2.31/23"],"mac_address":"0a:58:0a:81:02:1f","gateway_ips":["10.129.2.1"],"ip_address":"10.129.2.31/23","gateway_ip":"10.129.2.1"}}] on pod test/hello-pod
2022-07-11T06:08:20.614135395Z I0711 06:08:20.614095       1 pods.go:358] [test/hello-pod] addLogicalPort took 47.25565ms, libovsdb time 3.232352ms, annotation time: 43.386391ms

Technically should be new bug, but let's not cause another 4 backport PRs and just fix it in this one itself.

Opening a new bug to unblock verifying https://bugzilla.redhat.com/show_bug.cgi?id=2097243#c17

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:


Note You need to log in before you can comment on or make changes to this bug.