Bug 1326067

Summary: default service implementation of "iptables" has a feature regression on "load balancing"
Product: OpenShift Container Platform Reporter: Erik M Jacobs <ejacobs>
Component: NetworkingAssignee: Dan Winship <danw>
Status: CLOSED CURRENTRELEASE QA Contact: Meng Bo <bmeng>
Severity: low Docs Contact:
Priority: medium    
Version: 3.1.0CC: aloughla, aos-bugs, atragler, bbennett, danw, hongli, mleitner, pep, rkhan, sukulkar, thunt
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-06-22 21:10:43 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On:    
Bug Blocks: 1267746    

Description Erik M Jacobs 2016-04-11 17:26:38 UTC 
The userspace kube-proxy implementation provided a feature where, if an endpoint did not successfully connect, kube-proxy would "roll over" to another endpoint.

The iptables implementation of the service layer is now the default and, while it offers higher performance, it is incapable of providing this feature.

* There are documentation implications because people may not understand that this change in implementations occurred, and may not understand why things are not behaving as they once were.

* There is a documentation issue in that selecting the userspace implementation is in the installer but is undocumented.

* Understanding the implications of using one mechanism versus the other need to be documented as well. kube-proxy acts like much more of a traditional load balancer than the iptables implementation because of the retry feature.

* Understanding how all of this is related to probes (specifically liveness) as well as node timeouts/evictions/etc. is also important.
Comment 1 Ben Bennett 2016-04-12 14:23:32 UTC 
This is a documentation issue... we'll clarify the 3.2 docs to cover the choices, the pros and cons, and how to flip.
Comment 2 Dan Winship 2016-04-19 13:55:30 UTC 
Notes:
  - filed a docs bug upstream:
    https://github.com/kubernetes/kubernetes.github.io/pull/401

  - ansible var: openshift_node_proxy_mode, which can be 'iptables'
    (the default) or 'userspace'. (In theory this is a per-node
    configuration variable, but in practice you need to use the same
    value cluster-wide. So I think, in ansible terms, that means you'd
    set it in the [OSEv3:vars] section, not the [nodes] section? But
    I've never used ansible...)

  - node-config.yaml:
      proxyArguments:
        proxy-mode:
        - userspace
Comment 3 Dan Winship 2016-04-19 13:56:51 UTC 
also, upstream (non-docs) bug: https://github.com/kubernetes/kubernetes/issues/24322
Comment 4 Dan Winship 2016-04-26 14:22:26 UTC 
Filed https://github.com/openshift/openshift-docs/pull/1948