Red Hat Bugzilla – Bug 1301654
[DOCS] Docs needed for when to set openshift_node_set_node_ip (nodeIP in the node config)
Last modified: 2016-11-08 01:29:32 EST
Currently we do not reference openshift_node_set_node_ip in the Installation docs.
The variables should be set under the [OSEv3:vars] section when installation is done in a cloud provider where internal hostnames are not configured/resolvable by all hosts.
- OpenStack installations using nova networking (or using neutron networking and internal DNS is not configured).
- Amazon Web Services VPC where the VPC is not configured for both 'DNS resolution' and 'DNS hostnames'
I found this BZ trying to find docs for openshift_node_set_node_ip, but, hmm...the description in the above comment doesn't seem to match the comments in hosts.ose.example:
# Configure nodeIP in the node config
# This is needed in cases where node traffic is desired to go over an
# interface other than the default network interface.
# Force setting of system hostname when configuring OpenShift
# This works around issues related to installations that do not have valid dns
# entries for the interfaces attached to the host.
My understanding is that the usage in the description is accurate and that the comments in the example inventory are also correct.
I've seen nodeip resolve problems where the IP of the Node from the Master's perspective is not the same as the IP of the Node from it's perspective. In normal Node registration the Node does not tell the Master it's IP the Master gets the Node IP from DNS. However, I know of at least one case where the Node reads it's IP from the Master for purposes of running liveness and readiness probe checks for Pods using "hostNetwork: true". What happened in this edge case was the Node tried to read the Pod running on itself by the IP the Master gave it. This resulted in the traffic being blocked by the Node's local firewall when instead it should have accessed the Pod using either it's internal IP or instead 127.0.0.1 (which I believe is a better solution).
Now, if there was some reason another IP was available on a Node and you had to force the probes to use that IP then you could use nodeIP for that. I suspect that is part of what the comments in the inventory file are getting at. I'm certain there are other reasons someone might want to use it but in practice we recommend not using it simply because relying on DNS will be more flexible.
Thinking more about this now I do know of one case in which heapster running on a Node was blocked without nodeIP being set because the Node didn't have the default IP in it's certificate. Again, accessing the Node via DNS would have been better in that case. Right now my opinion is that nodeIP is useful as a workaround to get out of a configuration edgecase but should not normally be used.
OK. So it sounds like
This is needed in cases where node traffic is desired to go over an interface other than the default network interface.
Will not help me create multiple networks, one for management traffic and another for the production/pod VXLAN traffic. That is my goal.
(In reply to Jeremy Eder from comment #3)
> OK. So it sounds like
> This is needed in cases where node traffic is desired to go over an
> interface other than the default network interface.
> Will not help me create multiple networks, one for management traffic and
> another for the production/pod VXLAN traffic. That is my goal.
That is correct, currently the SDN uses the same configuration as the node for which ip/interface to use.
I vaguely remember a trello card for adding support within the Node to specify multiple IPs for different uses.
Incorporated tech review feedback from Brenton and Jason. Moving to peer review:
Passed peer review, merging.