Bug 1709254 - The EgressIps of project would not be auto assigned to the node when multiple Ips from are given in the project netnamespace.
Summary: The EgressIps of project would not be auto assigned to the node when multiple...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Networking
Version: 3.11.0
Hardware: x86_64
OS: Linux
unspecified
medium
Target Milestone: ---
: 3.11.z
Assignee: Jason Boxman
QA Contact: zhaozhanqi
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-05-13 09:39 UTC by btai
Modified: 2019-06-17 18:46 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-06-17 18:46:59 UTC
Target Upstream Version:


Attachments (Terms of Use)

Description btai 2019-05-13 09:39:32 UTC
Description of problem:
The EgressIps of project would not be auto assigned to the node when multiple Ips from are given in the project netnamespace.

When only one egressIp is given to the project then the ip would get assigned on the node as expected.

Version-Release number of selected component (if applicable):
 openshift v3.11.88

How reproducible:

Everytime.
 

Steps to Reproduce:
1.
Patch the node to have EgressCIDRS
oc patch hostsubnet h1-a-xxx.xxx.xxx -p '{"egressCIDRs": ["150.106.1.240/30"]}'
oc patch hostsubnet r1-a-xxx.xxx.xx -p '{"egressCIDRs": ["150.106.250.165/30"]}'

2.
Patch namespace to have two EgressIPS
oc patch netnamespace demo -p '{"egressIPs": ["150.106.1.241", "150.106.250.165"]}'


Actual results:
oc get hostsubnet 
 none of the nodes will have the above two EgressIP assigned in the netnamespace
NAME                      HOST                      HOST IP           SUBNET           EGRESS CIDRS           EGRESS IPS
h1-a-xxx.xxx.xxx.xxxxxx   h1-a-ocpnd1.statnett.no   150.106.1.139     10.101.12.0/23   [150.106.1.240/30]     []
r1-a-xxx.xxx.xxx.xxxxxx   r1-a-ocpnd1.statnett.no   150.106.250.135   10.101.16.0/23   [150.106.250.164/30]   []


Expected results:
oc get hostsubnet 

NAME                      HOST                      HOST IP           SUBNET           EGRESS CIDRS           EGRESS IPS
h1-a-xxx.xxx.xxxxxxxxxx   h1-a-ocpnd2.statnett.no   150.106.1.141     10.101.14.0/23   [150.106.1.240/30]     [150.106.1.241]
r1-a-xxx.xxx.xxxxxxxxxx   r1-a-ocpnd1.statnett.no   150.106.250.135   10.101.16.0/23   [150.106.250.164/30]   [150.106.250.165]

Additional info:

This works as expected if we only use one ip address in the netnamespace 
oc patch netnamespace demo -p '{"egressIPs": ["150.106.1.241"]}'

then when doing oc get hostsubnet we get 
NAME                      HOST                      HOST IP           SUBNET           EGRESS CIDRS           EGRESS IPS
h1-a-xxx.xxx.xxxxxxxxxx   h1-a-ocpnd2.statnett.no   150.106.1.141     10.101.14.0/23   [150.106.1.240/30]     [150.106.1.241]


We realised that autoassigning when multiple egressIP used was stripped out on this 
https://github.com/openshift/origin/pull/21287/commits/985a3ce0671d19a0e28466f6045902992aa99153

but it would make sense to have it autoassigned when we provide EgressIPs on netnamespace, if the IPs are from different subnet.

Comment 1 Casey Callendrello 2019-05-15 11:54:59 UTC
I think this is a usability issue, and a major wart, but it's not a bug. If you set egressCIDRs, it means you want to use fully-automatic egress IP assignment, which only works when a project has a single egress IP.

This is hinted at in the documentation, but should be called out properly.

Dan Winship, can you confirm? If so, I'll get the docs updated.

Comment 2 Dan Winship 2019-05-15 13:30:10 UTC
> but it would make sense to have it autoassigned when we provide EgressIPs on netnamespace, if the IPs are from different subnet.

There is an RFE for this somewhere, but right now we do not implement that.

(In reply to Casey Callendrello from comment #1)
> I think this is a usability issue, and a major wart, but it's not a bug. If
> you set egressCIDRs, it means you want to use fully-automatic egress IP
> assignment, which only works when a project has a single egress IP.
> 
> This is hinted at in the documentation, but should be called out properly.
> 
> Dan Winship, can you confirm? If so, I'll get the docs updated.

That's correct

Comment 3 Casey Callendrello 2019-05-15 16:43:46 UTC
Assigning this to Jason.
Jason, can you update the 3.11 and 4.1 docs around fully-automatic egress IPs to include some kind of warning that only a single egress IP can be assigned?

Comment 4 btai 2019-05-16 07:05:09 UTC
wWhile i agree that everything is perceived to be automated when using CIDR, i still think its bug or feature missing.

For example  if you have ocp nodes in two subnet. Then you will be needing to provide two Ip from each subnet in the Project.
In that case what would be expected is

if 1st Ip in the list belongs to a CIDR on aset of host then OCP assigns the IP to the matching host ( that is currently does when it sees only one IP in project)

in case when the 1st Ip of project does not match CIDR of any node in OCP, then i needs to retry the same logic with secondIP, ie try and find the node with CIDR that matches the second IP and assign the IP to the node.


So i would guess if its not a bug,, its definitely something missing.. Besides netnamespace is expecting "egressIPs": which clearly fits for the usecase.

Comment 5 Jason Boxman 2019-05-24 15:32:02 UTC
When updating the 3.x docs for this, I'm going to use this PR[0] as a reference.

[0] https://github.com/openshift/openshift-docs/pull/14933

Comment 6 Jason Boxman 2019-06-10 20:24:59 UTC
I've created a PR[0] for this that clarifies that only a single IP address can be provided to `egressIPs`.

Thanks.

[0] https://github.com/openshift/openshift-docs/pull/15311

Comment 7 zhaozhanqi 2019-06-11 02:34:44 UTC
the PR lgtm, verified this bug.

Comment 8 Jason Boxman 2019-06-13 20:07:53 UTC
@Casey

Does this apply to only 3.11 or are other 3.x versions affected?

Thanks!

Comment 9 Dan Winship 2019-06-13 20:15:08 UTC
This applies to all versions


Note You need to log in before you can comment on or make changes to this bug.