Bug 1560584 - [3.9] Semi automatic namespace wide egress IP randomly shows up as node IP
Summary: [3.9] Semi automatic namespace wide egress IP randomly shows up as node IP
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Networking
Version: 3.9.0
Hardware: Unspecified
OS: Linux
high
high
Target Milestone: ---
: 3.9.0
Assignee: Dan Winship
QA Contact: Meng Bo
URL:
Whiteboard:
Depends On: 1552869
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-03-26 13:44 UTC by Dan Winship
Modified: 2018-12-29 05:46 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: The "kube-proxy" and "kubelet" parts of the OpenShift node process were being given different default values for the config options describing how to interact with iptables. Consequence: OpenShift would periodically add a bogus iptables rule that would cause *some* per-project static egress IPs to not be used for some length of time, until the bogus rule was removed again. (While the bogus rule was present, traffic from those projects would use the node IP address of the node hosting the egress IP, rather than the egress IP itself.) Fix: The inconsistent configuration was resolved, causing the bogus iptables rule to no longer be added. Result: Projects consistently use their static egress IPs.
Clone Of: 1552869
Environment:
Last Closed: 2018-12-13 19:26:59 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Origin (Github) 19099 0 None None None 2018-03-26 13:47:44 UTC
Red Hat Product Errata RHBA-2018:3748 0 None None None 2018-12-13 19:27:10 UTC

Comment 1 Dan Winship 2018-03-26 13:47:45 UTC
https://github.com/openshift/origin/pull/19099

Comment 2 Ben Bennett 2018-04-13 14:31:30 UTC
This went out in 3.9.0.

Comment 3 Meng Bo 2018-05-04 11:19:01 UTC
Tested on 3.9.27, seems it still has problem.

Copy the function from the egressip.go

func main() {
   var vnid uint32 = 11223195
   var masqueradeBit uint32 = 0
   
   if vnid == 0 {
        vnid = 0xff000000
   }
   if (vnid & masqueradeBit) != 0 {
       vnid = (vnid | 0x01000000) ^ masqueradeBit
   }
   fmt.Printf("%x\n", vnid)
}

The vnid of my project is 11223195, if the masqueradeBit set to 0, the output above should be ab409b, and if set the masqueradeBit to 14, the output above will be 1ab4095

Checking the openflow rules after the egressip configured.

cookie=0x0,table=100, priority=100,ip,reg0=0xab409b actions=set_field:ba:ee:08:a6:0b:72->eth_dst,set_field:0x1ab409a->pkt_mark,goto_table:101

The pkt_mark is set as the value when the masqueradeBit = 14

Comment 4 Meng Bo 2018-05-22 08:27:12 UTC
The comment#3 should be wrong, as the masqueradeBit value is not 0, it should be 1 since the value comes from "1 << 0".

As the result, the mark for the vnid with a even number will be the same as the vnid, and the mark for the vnid with odd number, it will add 1 bit to the beginning and minus 1 from the last bit. So the value is correct.

the function should be changed to:

func main() {
   var vnid uint32 = 11223195
   var masqueradeBit uint32 = 1 << 0
   
   if vnid == 0 {
        vnid = 0xff000000
   }
   if (vnid & masqueradeBit) != 0 {
       vnid = (vnid | 0x01000000) ^ masqueradeBit
   }
   fmt.Printf("%x\n", vnid)
}

So with the VNID = 11223195, the correct output should be 1ab409a, and if the masqueradebit is 14, the value should be 1ab408b

It is fixed actually.

Comment 7 errata-xmlrpc 2018-12-13 19:26:59 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:3748


Note You need to log in before you can comment on or make changes to this bug.