Bug 2028967 - Whereabouts should reconcile stranded IP addresses
Summary: Whereabouts should reconcile stranded IP addresses
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Networking
Version: 4.10
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 4.7.z
Assignee: Douglas Smith
QA Contact: Weibin Liang
URL:
Whiteboard:
Depends On: 2028966
Blocks: 2028968
TreeView+ depends on / blocked
 
Reported: 2021-12-03 20:28 UTC by Douglas Smith
Modified: 2022-02-16 11:20 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: Enhancement
Doc Text:
Feature: Implements an IP reconciliation job for Whereabouts IPAM CNI called "ip-reconciler" which runs as a Kubernetes cronjob. Reason: On occasion events occur where the CNI DEL action will not complete for a given pod (for example, a forcefully powered off node), and in such a case stored IP address allocations may be left stranded and unable to be allocated without manual intervention. Result: Stranded IP address allocations are garbage collected automatically on a periodic basis to free unused IP addresses.
Clone Of: 2028966
: 2028968 (view as bug list)
Environment:
Last Closed: 2022-02-16 11:19:40 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift whereabouts-cni pull 79 0 None open Bug 2028967: Whereabouts should reconcile IP addresses [backport 4.7] 2022-01-28 20:13:51 UTC
Red Hat Product Errata RHSA-2022:0492 0 None None None 2022-02-16 11:20:05 UTC

Description Douglas Smith 2021-12-03 20:28:29 UTC
+++ This bug was initially created as a clone of Bug #2028966 +++

+++ This bug was initially created as a clone of Bug #2028964 +++

+++ This bug was initially created as a clone of Bug #2028963 +++

Description of problem: IP reconciliation is a feature in the latest whereabouts, and due to reports, this feature should be backported all the way to 4.6.z. The feature is in the form of a cron job which reconciles the IP addresses.


Version-Release number of selected component (if applicable): 4.6-4.10


How reproducible: Specialized. Customers often experience this when nodes are rebooted, or pods are force deleted, and therefore CNI DEL calls aren't processed in their entirety by Whereabouts


Steps to Reproduce: (We will produce a procedure which produces orphaned IP addresses)

Actual results: IP addresses will remain stranded, and never utilized again.


Expected results: IP addresses that were stranded become available for use again.


Additional info: 4.10 has the reconciliation code but still requires a bug fix from upstream.

Comment 1 Weibin Liang 2022-02-01 15:02:52 UTC
Testing passed in cluster-bot: launch openshift/whereabouts-cni#79,openshift/cluster-network-operator#1266

[weliang@weliang Test]$ oc get clusterversion
NAME      VERSION                                                  AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.7.0-0.ci.test-2022-02-01-142111-ci-ln-zp1bifk-latest   True        False         10m     Cluster version is 4.7.0-0.ci.test-2022-02-01-142111-ci-ln-zp1bifk-latest
[weliang@weliang Test]$ 

[weliang@weliang Test]$ oc create -f ippool.yml -n openshift-multus
ippool.whereabouts.cni.cncf.io/192.168.2.224-28 created
[weliang@weliang Test]$ oc get ippools 192.168.2.224-28 -o yaml -n openshift-multus
apiVersion: whereabouts.cni.cncf.io/v1alpha1
kind: IPPool
metadata:
  creationTimestamp: "2022-02-01T15:00:02Z"
  generation: 2
  name: 192.168.2.224-28
  namespace: openshift-multus
  resourceVersion: "29743"
  selfLink: /apis/whereabouts.cni.cncf.io/v1alpha1/namespaces/openshift-multus/ippools/192.168.2.224-28
  uid: d1e1ff0d-dd7e-4872-b59f-b02da12af3fc
spec:
  allocations: {}
  range: 192.168.2.224/28
[weliang@weliang Test]$ oc create job --from=cronjob/ip-reconciler -n openshift-multus testrun-ip-reconciler
job.batch/testrun-ip-reconciler created
[weliang@weliang Test]$ oc get ippools 192.168.2.224-28 -o yaml -n openshift-multus
apiVersion: whereabouts.cni.cncf.io/v1alpha1
kind: IPPool
metadata:
  creationTimestamp: "2022-02-01T15:00:02Z"
  generation: 2
  name: 192.168.2.224-28
  namespace: openshift-multus
  resourceVersion: "29743"
  selfLink: /apis/whereabouts.cni.cncf.io/v1alpha1/namespaces/openshift-multus/ippools/192.168.2.224-28
  uid: d1e1ff0d-dd7e-4872-b59f-b02da12af3fc
spec:
  allocations: {}
  range: 192.168.2.224/28
[weliang@weliang Test]$

Comment 6 errata-xmlrpc 2022-02-16 11:19:40 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.7.43 security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:0492


Note You need to log in before you can comment on or make changes to this bug.