Bug 2115481 - ovnkube direct-lists pods on a node when the node object changes
Summary: ovnkube direct-lists pods on a node when the node object changes
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Networking
Version: 4.12
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
: 4.11.z
Assignee: Dan Williams
QA Contact: Mike Fiedler
Whiteboard: perfscale-ovn
Depends On: 2115479
TreeView+ depends on / blocked
Reported: 2022-08-04 18:54 UTC by Dan Williams
Modified: 2022-09-07 20:49 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 2115479
Last Closed: 2022-09-07 20:49:26 UTC
Target Upstream Version:

Attachments (Terms of Use)

System ID Private Priority Status Summary Last Updated
Github openshift ovn-kubernetes pull 1232 0 None Merged [release-4.11] Bug 2115481: master: don't list node pods on add/update unless necessary 2022-08-23 13:13:31 UTC
Red Hat Product Errata RHSA-2022:6287 0 None None None 2022-09-07 20:49:35 UTC

Description Dan Williams 2022-08-04 18:54:40 UTC
+++ This bug was initially created as a clone of Bug #2115479 +++

2022-07-07T17:42:48.798533966Z I0707 17:42:48.798447      15 trace.go:205] Trace[17745800]: "List" url:/api/v1/pods,user-agent:ip-10-0-149-185/ovnkube@bd4f2094aeb5 (linux/amd64) kubernetes/,audit-id:6c114e16-15ac-4920-b8ed-a01cbd710909,client:,accept:application/vnd.kubernetes.protobuf,application/json,protocol:HTTP/2.0 (07-Jul-2022 17:42:36.762) (total time: 12035ms):
2022-07-07T17:42:48.889007528Z I0707 17:42:48.888869      15 trace.go:205] Trace[859805328]: "List" url:/api/v1/pods,user-agent:ip-10-0-149-185/ovnkube@bd4f2094aeb5 (linux/amd64) kubernetes/,audit-id:49ad7f2b-9c09-4d0b-9068-4e6fa606a4f8,client:,accept:application/vnd.kubernetes.protobuf,application/json,protocol:HTTP/2.0 (07-Jul-2022 17:42:36.761) (total time: 12127ms):
2022-07-07T17:42:49.272283449Z I0707 17:42:49.272192      15 trace.go:205] Trace[1792712204]: "List" url:/api/v1/pods,user-agent:ip-10-0-149-185/ovnkube@bd4f2094aeb5 (linux/amd64) kubernetes/,audit-id:e0269f80-38be-4c03-8317-6b63fd798ceb,client:,accept:application/vnd.kubernetes.protobuf,application/json,protocol:HTTP/2.0 (07-Jul-2022 17:42:36.723) (total time: 12548ms):
2022-07-07T17:42:49.596468418Z I0707 17:42:49.596368      15 trace.go:205] Trace[1248373466]: "List" url:/api/v1/pods,user-agent:ip-10-0-149-185/ovnkube@bd4f2094aeb5 (linux/amd64) kubernetes/,audit-id:9fb86c6a-4879-42de-bcce-c987b1c77931,client:,accept:application/vnd.kubernetes.protobuf,application/json,protocol:HTTP/2.0 (07-Jul-2022 17:42:36.763) (total time: 12832ms):
2022-07-07T17:42:49.771518271Z I0707 17:42:49.756590      15 trace.go:205] Trace[235168718]: "List" url:/api/v1/pods,user-agent:ip-10-0-149-185/ovnkube@bd4f2094aeb5 (linux/amd64) kubernetes/,audit-id:ea512964-5969-4612-b013-e9c63a4e72e0,client:,accept:application/vnd.kubernetes.protobuf,application/json,protocol:HTTP/2.0 (07-Jul-2022 17:42:36.751) (total time: 13005ms):

ovnkube master would list pods directly from the API (not from a shared informer cache) whenever a node add/update event happened, and worse wasn't listing from the apiserver's cache either (by setting ResourceVersion:"0" in the ListOptions).

This places a bunch of load on the apiserver/etcd that's unecessary.

Comment 4 Mike Fiedler 2022-09-01 23:34:00 UTC
Marking VERIFIED per https://bugzilla.redhat.com/show_bug.cgi?id=2108679#c11

Comment 6 errata-xmlrpc 2022-09-07 20:49:26 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.11.3 packages and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.