Bug 2046296

Summary: When creating multiple consecutive egressIPs on GCP not all of them get assigned to the instance
Product: OpenShift Container Platform Reporter: Patryk Diak <pdiak>
Component: NetworkingAssignee: Patryk Diak <pdiak>
Networking sub component: ovn-kubernetes QA Contact: jechen <jechen>
Status: CLOSED ERRATA Docs Contact:
Severity: high    
Priority: unspecified    
Version: 4.10   
Target Milestone: ---   
Target Release: 4.10.0   
Hardware: Unspecified   
OS: Unspecified   
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-03-12 04:41:56 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Patryk Diak 2022-01-26 14:31:25 UTC
Description of problem:
When creating or removing multiple egressIPs quickly they are not correctly removed/added on GCP instance

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
run the following script:

for i in {1..10}
    oc new-project test${i}
    cat << EOF | oc apply -f -
apiVersion: k8s.ovn.org/v1
kind: EgressIP
  name: egressip-${i}
      name: test${i}

Actual results:
Check ipCidrRange on the instance:

gcloud compute instances describe pdiak-01-26-2022-t8xmx-worker-b-7jmwh | grep ipCidrRange
No zone specified. Using zone [us-east1-b] for instance: [pdiak-01-26-2022-t8xmx-worker-b-7jmwh].
  - ipCidrRange:
  - ipCidrRange:
  - ipCidrRange:
  - ipCidrRange:
  - ipCidrRange:

egressip falsly report that all are assigned:
NAME          EGRESSIPS      ASSIGNED NODE                           ASSIGNED EGRESSIPS
egressip-1    pdiak-01-26-2022-t8xmx-worker-b-7jmwh
egressip-10   pdiak-01-26-2022-t8xmx-worker-b-7jmwh
egressip-2    pdiak-01-26-2022-t8xmx-worker-b-7jmwh
egressip-3    pdiak-01-26-2022-t8xmx-worker-b-7jmwh
egressip-4    pdiak-01-26-2022-t8xmx-worker-b-7jmwh
egressip-5    pdiak-01-26-2022-t8xmx-worker-b-7jmwh
egressip-6    pdiak-01-26-2022-t8xmx-worker-b-7jmwh
egressip-7    pdiak-01-26-2022-t8xmx-worker-b-7jmwh
egressip-8    pdiak-01-26-2022-t8xmx-worker-b-7jmwh
egressip-9    pdiak-01-26-2022-t8xmx-worker-b-7jmwh

Expected results:
all egress ips should be assigned on the gcp instance

Additional info:

Comment 4 jechen 2022-01-27 19:34:39 UTC
Verified in 4.10.0-0.nightly-2022-01-27-104747

$ oc get clusterversion
NAME      VERSION                              AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.10.0-0.nightly-2022-01-27-104747   True        False         26m     Cluster version is 4.10.0-0.nightly-2022-01-27-104747

$ oc get node
NAME                                                        STATUS   ROLES    AGE   VERSION
jechen-0127c-r76dn-master-0.c.openshift-qe.internal         Ready    master   48m   v1.23.0+d30ebbc
jechen-0127c-r76dn-master-1.c.openshift-qe.internal         Ready    master   49m   v1.23.0+d30ebbc
jechen-0127c-r76dn-master-2.c.openshift-qe.internal         Ready    master   49m   v1.23.0+d30ebbc
jechen-0127c-r76dn-worker-a-6gcp2.c.openshift-qe.internal   Ready    worker   37m   v1.23.0+d30ebbc
jechen-0127c-r76dn-worker-b-7qtqd.c.openshift-qe.internal   Ready    worker   37m   v1.23.0+d30ebbc
jechen-0127c-r76dn-worker-c-gdjfb.c.openshift-qe.internal   Ready    worker   37m   v1.23.0+d30ebbc

$ oc label node jechen-0127c-r76dn-worker-a-6gcp2.c.openshift-qe.internal "k8s.ovn.org/egress-assignable"=""
node/jechen-0127c-r76dn-worker-a-6gcp2.c.openshift-qe.internal labeled

$ oc label node jechen-0127c-r76dn-worker-b-7qtqd.c.openshift-qe.internal "k8s.ovn.org/egress-assignable"=""
node/jechen-0127c-r76dn-worker-b-7qtqd.c.openshift-qe.internal labeled

$ oc label node jechen-0127c-r76dn-worker-c-gdjfb.c.openshift-qe.internal "k8s.ovn.org/egress-assignable"=""
node/jechen-0127c-r76dn-worker-c-gdjfb.c.openshift-qe.internal labeled

# create script to add 10 egressip objects

$ oc get egressip
NAME         EGRESSIPS      ASSIGNED NODE                                               ASSIGNED EGRESSIPS
egressip1   jechen-0127c-r76dn-worker-a-6gcp2.c.openshift-qe.internal
egressip10   jechen-0127c-r76dn-worker-a-6gcp2.c.openshift-qe.internal
egressip2   jechen-0127c-r76dn-worker-a-6gcp2.c.openshift-qe.internal
egressip3   jechen-0127c-r76dn-worker-a-6gcp2.c.openshift-qe.internal
egressip4   jechen-0127c-r76dn-worker-a-6gcp2.c.openshift-qe.internal
egressip5   jechen-0127c-r76dn-worker-a-6gcp2.c.openshift-qe.internal
egressip6   jechen-0127c-r76dn-worker-a-6gcp2.c.openshift-qe.internal
egressip7   jechen-0127c-r76dn-worker-a-6gcp2.c.openshift-qe.internal
egressip8   jechen-0127c-r76dn-worker-a-6gcp2.c.openshift-qe.internal
egressip9   jechen-0127c-r76dn-worker-a-6gcp2.c.openshift-qe.internal

$ gcloud compute instances describe jechen-0127c-r76dn-worker-a-6gcp2 |grep ipCidrRange
  - ipCidrRange:
  - ipCidrRange:
  - ipCidrRange:
  - ipCidrRange:
  - ipCidrRange:
  - ipCidrRange:
  - ipCidrRange:
  - ipCidrRange:
  - ipCidrRange:
  - ipCidrRange:

All egressips are assigned on the gcp instance

Comment 7 errata-xmlrpc 2022-03-12 04:41:56 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.10.3 security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.