Red Hat Bugzilla – Bug 1253125
Improve on nodeselector information
Last modified: 2016-04-18 21:59:31 EDT
Suggestions for improvement:
There’s a lot of talk about the nodeselector, but nothing that describes what it is and what it does. The best info is in the REST API bit.
"Node selector" isn't anything special on its own, they're just labels (https://docs.openshift.com/enterprise/3.0/architecture/core_concepts/pods_and_services.html#labels) that have been applied to the node objects. And then because the node objects have been labeled, you can filter (i.e., "select") certain nodes using the label(s).
The docs do make it seem like they're something special though, or at the very least are nebulous. We could make it clearer in the "Scheduler" topic, and say something about them in https://docs.openshift.com/enterprise/3.0/architecture/infrastructure_components/kubernetes_infrastructure.html#node
A node selector is not just a label. You put labels on the nodes. You put a node selector on a pod (via various mechanisms) to indicate which node labels you require nodes to have in order to host that pod; this is quite separate from labels on the pod itself which are used for selecting *pods* as e.g. service endpoints.
Node selectors come from three separate sources:
1. The project in which the pod is coming from. An admin can create the project with a node selector e.g.:
# oadm new-project test --node-selector='type=user-node,zone=east'
This has the result of including an annotation in the namespace/project definition e.g.:
This can be added in after the fact by editing the namespace (but not the project! which is confusing).
2. The default node selector specified in the master config. This is used if the admin created the project without 1.
3. The pod itself. This is the only part discussed in kubernetes docs because it's the only relevant part for kubernetes. The project-level node selector is openshift-specific.
The label selector on the pod is *added* to the project-level node selector, so that candidate nodes must satisfy *all* of the labels, which allows project-creating admins to restrict project pods to a given set of nodes while still permitting the project user some leeway in setting further restrictions.
Specifically as it relates to projects, can we update the admin guide's "self provisioned projects" section (https://docs.openshift.com/enterprise/3.1/admin_guide/selfprovisioned_projects.html) to be more generic (eg: Project Management) and to itself include sections on:
* project templates
* self provisioned projects
* project quota in the template
* node selectors in the template and their effect
And possibly other topics?
(In reply to Erik M Jacobs from comment #5)
> Specifically as it relates to projects, can we update the admin guide's
> "self provisioned projects" section
> selfprovisioned_projects.html) to be more generic (eg: Project Management)
> and to itself include sections on:
> * project templates
> * self provisioned projects
> * project quota in the template
> * node selectors in the template and their effect
> And possibly other topics?
It may not yet warrant it, but if there ends up being enough content and/or other topics, it may be a good case to start a subsection, something like:
- Managing Projects
- Self-Provisioned Projects
- Defining Project Templates
- Using Node Selectors
rather, I guess.
What you're suggesting seems to be more along the lines of "Project Configuration". Looking at this BZ, I thought it would be best fit into the Scheduler topic:
From what I can tell, using the scheduler is the automatic version of using a nodeselector, right? You can point at where you want to pod to live, or get the scheduler to do it for you. I'd say info on the nodeselector would be more at home here. Then maybe a topic name change to something like "Determining Pod Placement" or something.
Am I missing something?
Unfortunately what we're talking about is not part of scheduler config, but rather is *informed by* scheduler config.
For example, take the following topology:
I have two regions (A, B) and two zones within each region (1,2).
For argument's sake, I want to place all development projects in region B. Via the *scheduler alone* there is no way to do this.
I would have to implement a nodeselector on all development projects that specifies "region=b".
Yes, node labels and scheduler config is involved, but this topic (nodeselector info) is still light.
Comment #5 and #6 lay out the case for an improvement in the overall structure of the project administration topic, and, if we go that route, I think will clear up some of the confusion on nodeselector, scheduler and etc.
Perhaps there's more clarification needed on how nodeselector *relates* to scheduler, but, more specifically, to node labels. I believe you could have node labels without any specific scheduler config and still use nodeselector (eg: the built-in scheduler configuration as opposed to a customized config).
I hope this all makes sense!
Ok. Tagged all relevant in this draft PR:
If anyone has any issues, please let me know either here or in the PR.
Commit pushed to master at https://github.com/openshift/openshift-docs
Merge pull request #1490 from bfallonf/BZ1253125
Bug 1253125 : added nodeselector information