Bug 2070392 - [OVN AWS] EgressIP was not balanced to another egress node after original node was removed egress label
Summary: [OVN AWS] EgressIP was not balanced to another egress node after original nod...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Networking
Version: 4.10
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 4.10.z
Assignee: Patryk Diak
QA Contact: huirwang
URL:
Whiteboard:
Depends On: 2078396
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-03-31 03:14 UTC by huirwang
Modified: 2023-09-15 01:53 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 2078396 (view as bug list)
Environment:
Last Closed: 2022-08-31 12:34:13 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift ovn-kubernetes pull 1203 0 None open [release-4.10] Bug 2070392: Fix egress IP reassignment on cloud 2022-07-20 09:04:33 UTC
Red Hat Product Errata RHSA-2022:6133 0 None None None 2022-08-31 12:34:43 UTC

Description huirwang 2022-03-31 03:14:11 UTC
Description of problem:
 EgressIP was not balanced to another egress node after original node was removed egress label

Version-Release number of selected component (if applicable):
4.10.0-0.nightly-2022-03-29-163038

How reproducible:
Hard to reproduce in manual test, but frequently failing in auto case.

Steps to Reproduce:
$ oc get nodes
NAME                                        STATUS   ROLES    AGE    VERSION
ip-10-0-50-148.us-east-2.compute.internal   Ready    master   109m   v1.23.5+1f952b3
ip-10-0-54-82.us-east-2.compute.internal    Ready    master   109m   v1.23.5+1f952b3
ip-10-0-55-50.us-east-2.compute.internal    Ready    worker   91m    v1.23.5+1f952b3
ip-10-0-58-148.us-east-2.compute.internal   Ready    worker   91m    v1.23.5+1f952b3
ip-10-0-65-102.us-east-2.compute.internal   Ready    master   109m   v1.23.5+1f952b3
ip-10-0-65-115.us-east-2.compute.internal   Ready    worker   91m    v1.23.5+1f952b3

1. Label one node as egress node ip-10-0-55-50.us-east-2.compute.internal
2. Create namespace test and add label name=qe
3. Create egressip object, egressip was successfully assigned.

 oc get egressip
NAME             EGRESSIPS     ASSIGNED NODE                              ASSIGNED EGRESSIPS
egressip-47028   10.0.55.117   ip-10-0-55-50.us-east-2.compute.internal   10.0.55.117

oc get egressip -o yaml
apiVersion: v1
items:
- apiVersion: k8s.ovn.org/v1
  kind: EgressIP
  metadata:
    annotations:
      kubectl.kubernetes.io/last-applied-configuration: |
        {"apiVersion":"k8s.ovn.org/v1","kind":"EgressIP","metadata":{"annotations":{},"name":"egressip-47028"},"spec":{"egressIPs":["10.0.55.117","10.0.52.72"],"namespaceSelector":{"matchLabels":{"name":"test"}}}}
    creationTimestamp: "2022-03-31T02:40:33Z"
    generation: 2
    managedFields:
    - apiVersion: k8s.ovn.org/v1
      fieldsType: FieldsV1
      fieldsV1:
        f:status:
          .: {}
          f:items: {}
      manager: ip-10-0-50-148
      operation: Update
      time: "2022-03-31T02:40:33Z"
    - apiVersion: k8s.ovn.org/v1
      fieldsType: FieldsV1
      fieldsV1:
        f:metadata:
          f:annotations:
            .: {}
            f:kubectl.kubernetes.io/last-applied-configuration: {}
        f:spec:
          .: {}
          f:egressIPs: {}
          f:namespaceSelector:
            .: {}
            f:matchLabels:
              .: {}
              f:name: {}
      manager: kubectl-client-side-apply
      operation: Update
      time: "2022-03-31T02:40:33Z"
    name: egressip-47028
    resourceVersion: "47028"
    uid: 078d97d7-e6dd-43e7-9662-4174576bcfe2
  spec:
    egressIPs:
    - 10.0.55.117
    - 10.0.52.72
    namespaceSelector:
      matchLabels:
        name: test
  status:
    items:
    - egressIP: 10.0.55.117
      node: ip-10-0-55-50.us-east-2.compute.internal
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""

Actual results:

4. Remove egress label from node ip-10-0-55-50.us-east-2.compute.internal
Add egress label to another node ip-10-0-58-148.us-east-2.compute.internal

oc get node ip-10-0-55-50.us-east-2.compute.internal --show-labels
NAME                                       STATUS   ROLES    AGE   VERSION           LABELS
ip-10-0-55-50.us-east-2.compute.internal   Ready    worker   51m   v1.23.5+1f952b3   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/instance-type=m5.xlarge,beta.kubernetes.io/os=linux,failure-domain.beta.kubernetes.io/region=us-east-2,failure-domain.beta.kubernetes.io/zone=us-east-2a,kubernetes.io/arch=amd64,kubernetes.io/hostname=ip-10-0-55-50.us-east-2.compute.internal,kubernetes.io/os=linux,node-role.kubernetes.io/worker=,node.kubernetes.io/instance-type=m5.xlarge,node.openshift.io/os_id=rhcos,topology.ebs.csi.aws.com/zone=us-east-2a,topology.kubernetes.io/region=us-east-2,topology.kubernetes.io/zone=us-east-2a

oc get node ip-10-0-58-148.us-east-2.compute.internal --show-labels
NAME                                        STATUS   ROLES    AGE   VERSION           LABELS
ip-10-0-58-148.us-east-2.compute.internal   Ready    worker   51m   v1.23.5+1f952b3   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/instance-type=m5.xlarge,beta.kubernetes.io/os=linux,failure-domain.beta.kubernetes.io/region=us-east-2,failure-domain.beta.kubernetes.io/zone=us-east-2a,k8s.ovn.org/egress-assignable=true,kubernetes.io/arch=amd64,kubernetes.io/hostname=ip-10-0-58-148.us-east-2.compute.internal,kubernetes.io/os=linux,node-role.kubernetes.io/worker=,node.kubernetes.io/instance-type=m5.xlarge,node.openshift.io/os_id=rhcos,topology.ebs.csi.aws.com/zone=us-east-2a,topology.kubernetes.io/region=us-east-2,topology.kubernetes.io/zone=us-east-2a

Actual Result:

5. Check egressip object, the assigned node was not updated
$ oc get egressip
NAME             EGRESSIPS     ASSIGNED NODE                              ASSIGNED EGRESSIPS
egressip-47028   10.0.55.117   ip-10-0-55-50.us-east-2.compute.internal   10.0.55.117

$ oc get CloudPrivateIPConfig
No resources found

Expected Result:
The assigned node should be updated correctly in egressip object.

Additional info:

Comment 9 errata-xmlrpc 2022-08-31 12:34:13 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: OpenShift Container Platform 4.10.30 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:6133

Comment 10 Red Hat Bugzilla 2023-09-15 01:53:28 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 365 days


Note You need to log in before you can comment on or make changes to this bug.