Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1585947

Summary: [egressip] The first element in the netnamespace egressIPs array should take effect
Product: OpenShift Container Platform Reporter: Meng Bo <bmeng>
Component: NetworkingAssignee: Dan Winship <danw>
Status: CLOSED CURRENTRELEASE QA Contact: Meng Bo <bmeng>
Severity: medium Docs Contact:
Priority: medium    
Version: 3.10.0CC: aos-bugs, bbennett, danw, hongli, xtian
Target Milestone: ---Keywords: Reopened
Target Release: 3.10.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: No Doc Update
Doc Text:
undefined
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-12-20 21:36:52 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Meng Bo 2018-06-05 07:30:57 UTC
Description of problem:
Assign multiple of egressIPs to the netnamespace, the latter one will take effect, no matter the order of the egressIP.

Version-Release number of selected component (if applicable):
v3.10.0-0.58.0

How reproducible:
always

Steps to Reproduce:
1. Setup multi-node cluster with multitenant or networkpolicy plugin
2. Assign a group IP to hostsubnet as egressIP
# oc patch hostsubnet networking-node1.cluster.local -p '{"egressIPs":["10.66.145.207","10.66.145.208","10.66.145.209"]}'
3. Add available egressIP to netnamespace
# oc patch netnamespace bmengp1 -p '{"egressIPs":["10.66.145.207"]}'
4. Check if the egressIP works
5. Add a new egressIP to netnamespace in the second order
# oc patch netnamespace bmengp1 -p '{"egressIPs":["10.66.145.207","10.66.145.208"]}'
6. Check if the egressIP works


Actual results:
4. The egressIP 10.66.145.207 works
6. The new added 10.66.145.208 takes effect

Expected results:
6. The first element in the egressIPs array should take effect

Additional info:
> netnamespace info
[root@networking-master ~]# oc get netnamespace bmengp1 -o yaml
apiVersion: network.openshift.io/v1
egressIPs:
- 10.66.145.207
- 10.66.145.208
kind: NetNamespace
metadata:
  creationTimestamp: 2018-05-31T07:19:10Z
  name: bmengp1
  resourceVersion: "1730425"
  selfLink: /apis/network.openshift.io/v1/netnamespaces/bmengp1
  uid: e990cdb9-64a2-11e8-ad0c-525400ffdbf7
netid: 7229402
netname: bmengp1


> check the iptables rules on the egress node
# iptables -nL OPENSHIFT-MASQUERADE -t nat 
Chain OPENSHIFT-MASQUERADE (1 references)
target     prot opt source               destination         
SNAT       all  --  10.128.0.0/14        0.0.0.0/0            mark match 0x6e4fda to:10.66.145.208
SNAT       all  --  10.128.0.0/14        0.0.0.0/0            mark match 0x6e4fda to:10.66.145.207
OPENSHIFT-MASQUERADE-2  all  --  10.128.0.0/14        0.0.0.0/0            /* masquerade pod-to-external traffic */



Switch the order on the netnamespace will not update the order in iptables.
Eg:
# oc patch netnamespace bmengp1 -p '{"egressIPs":["10.66.145.207","10.66.145.208"]}'
# oc patch netnamespace bmengp1 -p '{"egressIPs":["10.66.145.208","10.66.145.207"]}'

The iptables rule won't be changed by above actions.

Comment 1 Ben Bennett 2018-06-05 13:25:31 UTC
I'm not sure that we ever guaranteed what address would be used if multiple are given, other than saying that only one will be used.

The HA support will land sometime in the 3.10.z timeframe at which point we will make use of more than one if appropriately configured.

Comment 2 Dan Winship 2018-06-05 14:08:34 UTC
> I'm not sure that we ever guaranteed what address would be used if multiple
> are given, other than saying that only one will be used.

We used to say that all addresses except the first one would be ignored, so it was guaranteed that the first one would be used. https://github.com/openshift/origin/pull/19603 was supposed to cause addresses beyond the first one to be sanity-checked but *wasn't* supposed to cause them to actually get used.

Comment 3 Dan Winship 2018-06-06 19:22:31 UTC
https://github.com/openshift/origin/pull/19926

Comment 6 Meng Bo 2018-06-11 09:17:09 UTC
Tested on v3.10.0-0.64.0

Issue has been fixed.