Bug 2091157 - [4.10z] Completed pods in Openshift cluster not releasing IP addresses and results in err: range is full unless manually deleted [NEEDINFO]
Summary: [4.10z] Completed pods in Openshift cluster not releasing IP addresses and re...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Networking
Version: 4.8
Hardware: Unspecified
OS: Linux
urgent
urgent
Target Milestone: ---
: 4.10.z
Assignee: Tim Rozet
QA Contact: zhaozhanqi
URL:
Whiteboard:
Depends On: 2026461
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-05-27 17:09 UTC by Tim Rozet
Modified: 2022-12-13 11:29 UTC (History)
19 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 2026461
Environment:
Last Closed: 2022-07-07 11:39:56 UTC
Target Upstream Version:
hyupark: needinfo? (trozet)


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift ovn-kubernetes pull 1152 0 None Merged Bug 2091157: [release-4.10] Free IPs and delete resources for completed pods 2022-07-18 21:58:23 UTC
Red Hat Product Errata RHBA-2022:5428 0 None None None 2022-07-07 11:40:06 UTC

Comment 7 zhaozhanqi 2022-06-30 06:02:17 UTC
Verified this bug on 4.10.0-0.nightly-2022-06-30-014715


ovnkube-master-7cqm9-ovnkube-master.log:I0630 04:00:05.421865       1 ovn.go:687] Detected completed pod: openshift-operator-lifecycle-manager_collect-profiles-27609360-4klmj. Will remove.
ovnkube-master-7cqm9-ovnkube-master.log:I0630 04:00:05.422036       1 pods.go:200] Releasing IPs for Completed pod: openshift-operator-lifecycle-manager/collect-profiles-27609360-4klmj, ips: 10.128.2.16
ovnkube-master-7cqm9-ovnkube-master.log:I0630 04:15:06.250055       1 ovn.go:687] Detected completed pod: openshift-operator-lifecycle-manager_collect-profiles-27609375-bpdp5. Will remove.
ovnkube-master-7cqm9-ovnkube-master.log:I0630 04:15:06.250227       1 pods.go:200] Releasing IPs for Completed pod: openshift-operator-lifecycle-manager/collect-profiles-27609375-bpdp5, ips: 10.128.2.19
ovnkube-master-7cqm9-ovnkube-master.log:I0630 04:16:37.594257       1 ovn.go:687] Detected completed pod: openshift-operator-lifecycle-manager_collect-profiles-27609375-bpdp5. Will remove.
ovnkube-master-7cqm9-ovnkube-master.log:I0630 04:16:37.594876       1 ovn.go:687] Detected completed pod: openshift-operator-lifecycle-manager_collect-profiles-27609345-zgjmv. Will remove.
ovnkube-master-7cqm9-ovnkube-master.log:I0630 04:16:37.601908       1 ovn.go:726] Ignoring delete event for completed pod openshift-operator-lifecycle-manager/collect-profiles-27609375-bpdp5
ovnkube-master-7cqm9-ovnkube-master.log:I0630 04:16:37.609332       1 ovn.go:726] Ignoring delete event for completed pod openshift-operator-lifecycle-manager/collect-profiles-27609345-zgjmv
ovnkube-master-7cqm9-ovnkube-master.log:I0630 04:16:37.797955       1 ovn.go:687] Detected completed pod: openshift-operator-lifecycle-manager_collect-profiles-27609360-4klmj. Will remove.
ovnkube-master-7cqm9-ovnkube-master.log:I0630 04:16:37.800792       1 ovn.go:726] Ignoring delete event for completed pod openshift-operator-lifecycle-manager/collect-profiles-27609360-4klmj
ovnkube-master-7cqm9-ovnkube-master.log:I0630 04:30:12.523803       1 ovn.go:687] Detected completed pod: openshift-operator-lifecycle-manager_collect-profiles-27609390-6wd55. Will remove.
ovnkube-master-7cqm9-ovnkube-master.log:I0630 04:30:12.523968       1 pods.go:200] Releasing IPs for Completed pod: openshift-operator-lifecycle-manager/collect-profiles-27609390-6wd55, ips: 10.129.2.19
ovnkube-master-7cqm9-ovnkube-master.log:I0630 04:45:08.998742       1 ovn.go:687] Detected completed pod: openshift-operator-lifecycle-manager_collect-profiles-27609405-pm2j2. Will remove.
ovnkube-master-7cqm9-ovnkube-master.log:I0630 04:45:08.998984       1 pods.go:200] Releasing IPs for Completed pod: openshift-operator-lifecycle-manager/collect-profiles-27609405-pm2j2, ips: 10.129.2.20
ovnkube-master-7cqm9-ovnkube-master.log:I0630 05:00:07.589050       1 ovn.go:687] Detected completed pod: openshift-operator-lifecycle-manager_collect-profiles-27609420-7ktpr. Will remove.
ovnkube-master-7cqm9-ovnkube-master.log:I0630 05:00:07.589236       1 pods.go:200] Releasing IPs for Completed pod: openshift-operator-lifecycle-manager/collect-profiles-27609420-7ktpr, ips: 10.128.2.31
ovnkube-master-7cqm9-ovnkube-master.log:I0630 05:15:06.280678       1 ovn.go:687] Detected completed pod: openshift-operator-lifecycle-manager_collect-profiles-27609435-dssbk. Will remove.
ovnkube-master-7cqm9-ovnkube-master.log:I0630 05:15:06.280977       1 pods.go:200] Releasing IPs for Completed pod: openshift-operator-lifecycle-manager/collect-profiles-27609435-dssbk, ips: 10.128.2.32
ovnkube-master-7cqm9-ovnkube-master.log:I0630 05:15:06.346822       1 ovn.go:687] Detected completed pod: openshift-operator-lifecycle-manager_collect-profiles-27609390-6wd55. Will remove.
ovnkube-master-7cqm9-ovnkube-master.log:I0630 05:15:06.352755       1 ovn.go:726] Ignoring delete event for completed pod openshift-operator-lifecycle-manager/collect-profiles-27609390-6wd55
ovnkube-master-7cqm9-ovnkube-master.log:I0630 05:30:11.285500       1 ovn.go:687] Detected completed pod: openshift-operator-lifecycle-manager_collect-profiles-27609450-nz6kg. Will remove.
ovnkube-master-7cqm9-ovnkube-master.log:I0630 05:30:11.285730       1 pods.go:200] Releasing IPs for Completed pod: openshift-operator-lifecycle-manager/collect-profiles-27609450-nz6kg, ips: 10.128.2.34
ovnkube-master-7cqm9-ovnkube-master.log:I0630 05:30:11.351705       1 ovn.go:687] Detected completed pod: openshift-operator-lifecycle-manager_collect-profiles-27609405-pm2j2. Will remove.
ovnkube-master-7cqm9-ovnkube-master.log:I0630 05:30:11.355063       1 ovn.go:726] Ignoring delete event for completed pod openshift-operator-lifecycle-manager/collect-profiles-27609405-pm2j2
ovnkube-master-7cqm9-ovnkube-master.log:I0630 05:45:09.269117       1 ovn.go:687] Detected completed pod: openshift-operator-lifecycle-manager_collect-profiles-27609465-p9zlg. Will remove.
ovnkube-master-7cqm9-ovnkube-master.log:I0630 05:45:09.269402       1 pods.go:200] Releasing IPs for Completed pod: openshift-operator-lifecycle-manager/collect-profiles-27609465-p9zlg, ips: 10.128.2.35
ovnkube-master-7cqm9-ovnkube-master.log:I0630 05:45:09.326167       1 ovn.go:687] Detected completed pod: openshift-operator-lifecycle-manager_collect-profiles-27609420-7ktpr. Will remove.
ovnkube-master-7cqm9-ovnkube-master.log:I0630 05:45:09.329177       1 ovn.go:726] Ignoring delete event for completed pod openshift-operator-lifecycle-manager/collect-profiles-27609420-7ktpr

Comment 9 errata-xmlrpc 2022-07-07 11:39:56 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.10.21 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:5428

Comment 11 Dave 2022-08-26 12:43:23 UTC
Hi - As requested by Martin Kennelly. My customer is running a disconnected v4.10.22 cluster (OCP on OSP) using ovn-kubernetes and we are still seeing this bug. Example pod error message:

addLogicalPort failed for <<namespaces>>/<<pod>>: failed to assign pod address for pod <<pod>> no node: worker-node, err: range is full.

The other way to resolve this is to either keep deleting the pod until it hits a node it can come up on, or removing all Completed / Error pods on the cluster.

If this could be re-looked at I would be most grateful.

Thanks - Dave


Note You need to log in before you can comment on or make changes to this bug.