Description of problem: The EgressIps of project would not be auto assigned to the node when multiple Ips from are given in the project netnamespace. When only one egressIp is given to the project then the ip would get assigned on the node as expected. Version-Release number of selected component (if applicable): openshift v3.11.88 How reproducible: Everytime. Steps to Reproduce: 1. Patch the node to have EgressCIDRS oc patch hostsubnet h1-a-xxx.xxx.xxx -p '{"egressCIDRs": ["150.106.1.240/30"]}' oc patch hostsubnet r1-a-xxx.xxx.xx -p '{"egressCIDRs": ["150.106.250.165/30"]}' 2. Patch namespace to have two EgressIPS oc patch netnamespace demo -p '{"egressIPs": ["150.106.1.241", "150.106.250.165"]}' Actual results: oc get hostsubnet none of the nodes will have the above two EgressIP assigned in the netnamespace NAME HOST HOST IP SUBNET EGRESS CIDRS EGRESS IPS h1-a-xxx.xxx.xxx.xxxxxx h1-a-ocpnd1.statnett.no 150.106.1.139 10.101.12.0/23 [150.106.1.240/30] [] r1-a-xxx.xxx.xxx.xxxxxx r1-a-ocpnd1.statnett.no 150.106.250.135 10.101.16.0/23 [150.106.250.164/30] [] Expected results: oc get hostsubnet NAME HOST HOST IP SUBNET EGRESS CIDRS EGRESS IPS h1-a-xxx.xxx.xxxxxxxxxx h1-a-ocpnd2.statnett.no 150.106.1.141 10.101.14.0/23 [150.106.1.240/30] [150.106.1.241] r1-a-xxx.xxx.xxxxxxxxxx r1-a-ocpnd1.statnett.no 150.106.250.135 10.101.16.0/23 [150.106.250.164/30] [150.106.250.165] Additional info: This works as expected if we only use one ip address in the netnamespace oc patch netnamespace demo -p '{"egressIPs": ["150.106.1.241"]}' then when doing oc get hostsubnet we get NAME HOST HOST IP SUBNET EGRESS CIDRS EGRESS IPS h1-a-xxx.xxx.xxxxxxxxxx h1-a-ocpnd2.statnett.no 150.106.1.141 10.101.14.0/23 [150.106.1.240/30] [150.106.1.241] We realised that autoassigning when multiple egressIP used was stripped out on this https://github.com/openshift/origin/pull/21287/commits/985a3ce0671d19a0e28466f6045902992aa99153 but it would make sense to have it autoassigned when we provide EgressIPs on netnamespace, if the IPs are from different subnet.
I think this is a usability issue, and a major wart, but it's not a bug. If you set egressCIDRs, it means you want to use fully-automatic egress IP assignment, which only works when a project has a single egress IP. This is hinted at in the documentation, but should be called out properly. Dan Winship, can you confirm? If so, I'll get the docs updated.
> but it would make sense to have it autoassigned when we provide EgressIPs on netnamespace, if the IPs are from different subnet. There is an RFE for this somewhere, but right now we do not implement that. (In reply to Casey Callendrello from comment #1) > I think this is a usability issue, and a major wart, but it's not a bug. If > you set egressCIDRs, it means you want to use fully-automatic egress IP > assignment, which only works when a project has a single egress IP. > > This is hinted at in the documentation, but should be called out properly. > > Dan Winship, can you confirm? If so, I'll get the docs updated. That's correct
Assigning this to Jason. Jason, can you update the 3.11 and 4.1 docs around fully-automatic egress IPs to include some kind of warning that only a single egress IP can be assigned?
wWhile i agree that everything is perceived to be automated when using CIDR, i still think its bug or feature missing. For example if you have ocp nodes in two subnet. Then you will be needing to provide two Ip from each subnet in the Project. In that case what would be expected is if 1st Ip in the list belongs to a CIDR on aset of host then OCP assigns the IP to the matching host ( that is currently does when it sees only one IP in project) in case when the 1st Ip of project does not match CIDR of any node in OCP, then i needs to retry the same logic with secondIP, ie try and find the node with CIDR that matches the second IP and assign the IP to the node. So i would guess if its not a bug,, its definitely something missing.. Besides netnamespace is expecting "egressIPs": which clearly fits for the usecase.
When updating the 3.x docs for this, I'm going to use this PR[0] as a reference. [0] https://github.com/openshift/openshift-docs/pull/14933
I've created a PR[0] for this that clarifies that only a single IP address can be provided to `egressIPs`. Thanks. [0] https://github.com/openshift/openshift-docs/pull/15311
the PR lgtm, verified this bug.
@Casey Does this apply to only 3.11 or are other 3.x versions affected? Thanks!
This applies to all versions