Bug 1468607
Summary: | OPENSHIFT_MASTER & KUBERNETES_MASTER env vars in deployer pods wrong | ||
---|---|---|---|
Product: | OpenShift Container Platform | Reporter: | Eduardo Minguez <eminguez> |
Component: | Master | Assignee: | Dan Mace <dmace> |
Status: | CLOSED WONTFIX | QA Contact: | Chuan Yu <chuyu> |
Severity: | low | Docs Contact: | |
Priority: | unspecified | ||
Version: | 3.5.0 | CC: | aos-bugs, dmace, eminguez, jokerman, mmccomas |
Target Milestone: | --- | ||
Target Release: | 3.7.0 | ||
Hardware: | Unspecified | ||
OS: | Unspecified | ||
Whiteboard: | |||
Fixed In Version: | Doc Type: | If docs needed, set a value | |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | 2018-04-16 14:49:44 UTC | Type: | Bug |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: |
Description
Eduardo Minguez
2017-07-07 13:58:58 UTC
I am able to reproduce. From what I can tell, the master that the deploy pods use is determined by the which controller-manager on the HA masters wins the leader election. I think the controller-manager uses the /etc/origin/master/openshift-master.kubeconfig as its client config which uses itself as the server, not the LB. This means that API request from the controller-manager that won the election do not go through the LB, leading to higher stress on the master whose controller-manager is the leader. However, it probably also leads to lower latency on those requests since the master the controller-manager is talking to is colocated. I can see it either way, but the side effect on the deploy pods is unfortunate since the pod spec that the deployer controller generates uses the server from the controller-manager's client config as the OPENSHIFT_MASTER and KUBERNETES_MASTER. sending to Master tl;dr OPENSHIFT_MASTER and KUBERNETES_MASTER env vars for builder pods are set to the master co-resident with the active controller-manager by the build controller, not the master LB. To be clear, the supported deployer code constructs API clients using the KUBERNETES_SERVICE_HOST and KUBERNETES_SERVICE_PORT variables. The OPENSHIFT_MASTER and KUBERNETES_MASTER variables are no longer used. Did somebody observe evidence of the deployer container actually communicating with the master outside the service IP, or was that just assumed based on the values of these defunct variables? Changing the KUBERNETES_MASTER and OPENSHIFT_MASTER variables should only have an effect on any custom deployer image code which happens to be using them (and any such usages should be ported to use KUBERNETES_SERVICE_HOST and KUBERNETES_SERVICE_PORT). Are we aware of any such usages? We should remove OPENSHIFT_MASTER and KUBERNETES_MASTER from the deployer pod container environment. If there are any compatibility concerns regarding custom deployer images, we could provide a transitional period during 3.7 where the values are set to a URL like: https://$KUBERNETES_SERVICE_HOST:$KUBERNETES_SERVICE_PORT Then we could remove the variables entirely in 3.8. It also seems acceptable to leave the variables as-is and mark them deprecated in release notes until we're satisfied they're not in use by custom deployer/hook code. (In reply to Dan Mace from comment #4) > To be clear, the supported deployer code constructs API clients using the > KUBERNETES_SERVICE_HOST and KUBERNETES_SERVICE_PORT variables. The > OPENSHIFT_MASTER and KUBERNETES_MASTER variables are no longer used. Did > somebody observe evidence of the deployer container actually communicating > with the master outside the service IP, or was that just assumed based on > the values of these defunct variables? I've just seen them as env vars in the deployer pod but to be honest, I don't know if they are used or not. > > Changing the KUBERNETES_MASTER and OPENSHIFT_MASTER variables should only > have an effect on any custom deployer image code which happens to be using > them (and any such usages should be ported to use KUBERNETES_SERVICE_HOST > and KUBERNETES_SERVICE_PORT). Are we aware of any such usages? > I have no idea. > We should remove OPENSHIFT_MASTER and KUBERNETES_MASTER from the deployer > pod container environment. If there are any compatibility concerns regarding > custom deployer images, we could provide a transitional period during 3.7 > where the values are set to a URL like: > > https://$KUBERNETES_SERVICE_HOST:$KUBERNETES_SERVICE_PORT > > Then we could remove the variables entirely in 3.8. I like to have a "transition period" just in case. Thanks Release note to drive deprecation of these variables: https://github.com/openshift/openshift-docs/issues/4906 According the 3.7 release notes, this has been fixed, right? KUBERNETES_MASTER and OPENSHIFT_MASTER in deployer pods were deprecated, and no further changes to their behavior will be made. |