Bug 1253125 - Improve on nodeselector information
Improve on nodeselector information
Status: CLOSED CURRENTRELEASE
Product: OpenShift Container Platform
Classification: Red Hat
Component: Documentation (Show other bugs)
3.0.0
Unspecified Unspecified
medium Severity medium
: ---
: ---
Assigned To: brice
Vikram Goyal
Vikram Goyal
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-08-13 00:46 EDT by brice
Modified: 2016-04-18 21:59 EDT (History)
8 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-04-18 21:59:31 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description brice 2015-08-13 00:46:15 EDT
Suggestions for improvement: 

There’s a lot of talk about the nodeselector, but nothing that describes what it is and what it does. The best info is in the REST API bit.

"Node selector" isn't anything special on its own, they're just labels (https://docs.openshift.com/enterprise/3.0/architecture/core_concepts/pods_and_services.html#labels) that have been applied to the node objects. And then because the node objects have been labeled, you can filter (i.e., "select") certain nodes using the label(s).

The docs do make it seem like they're something special though, or at the very least are nebulous. We could make it clearer in the "Scheduler" topic, and say something about them in https://docs.openshift.com/enterprise/3.0/architecture/infrastructure_components/kubernetes_infrastructure.html#node
Comment 4 Luke Meyer 2015-11-12 14:17:00 EST
A node selector is not just a label. You put labels on the nodes. You put a node selector on a pod (via various mechanisms) to indicate which node labels you require nodes to have in order to host that pod; this is quite separate from labels on the pod itself which are used for selecting *pods* as e.g. service endpoints.

Node selectors come from three separate sources:

1. The project in which the pod is coming from. An admin can create the project with a node selector e.g.: 

# oadm new-project test --node-selector='type=user-node,zone=east'

This has the result of including an annotation in the namespace/project definition e.g.:

metadata:
  annotations:
    openshift.io/node-selector: type=user-node,zone=east

This can be added in after the fact by editing the namespace (but not the project! which is confusing).

2. The default node selector specified in the master config. This is used if the admin created the project without 1.

3. The pod itself. This is the only part discussed in kubernetes docs because it's the only relevant part for kubernetes. The project-level node selector is openshift-specific.

The label selector on the pod is *added* to the project-level node selector, so that candidate nodes must satisfy *all* of the labels, which allows project-creating admins to restrict project pods to a given set of nodes while still permitting the project user some leeway in setting further restrictions.
Comment 5 Erik M Jacobs 2016-01-20 11:35:40 EST
Specifically as it relates to projects, can we update the admin guide's "self provisioned projects" section (https://docs.openshift.com/enterprise/3.1/admin_guide/selfprovisioned_projects.html) to be more generic (eg: Project Management) and to itself include sections on:

* project templates
* self provisioned projects
* project quota in the template
* node selectors in the template and their effect

And possibly other topics?
Comment 6 Alex Dellapenta 2016-01-20 11:46:59 EST
(In reply to Erik M Jacobs from comment #5)
> Specifically as it relates to projects, can we update the admin guide's
> "self provisioned projects" section
> (https://docs.openshift.com/enterprise/3.1/admin_guide/
> selfprovisioned_projects.html) to be more generic (eg: Project Management)
> and to itself include sections on:
> 
> * project templates
> * self provisioned projects
> * project quota in the template
> * node selectors in the template and their effect
> 
> And possibly other topics?

+1

It may not yet warrant it, but if there ends up being enough content and/or other topics, it may be a good case to start a subsection, something like:

Cluster Administration
  ...
  - Managing Projects
      - Self-Provisioned Projects
      - Defining Project Templates
      - Using Node Selectors
Comment 7 Alex Dellapenta 2016-01-20 11:49:39 EST
s/subsection/subcategory/ 

rather, I guess.
Comment 8 brice 2016-01-27 19:43:58 EST
Erik, Alex

What you're suggesting seems to be more along the lines of "Project Configuration". Looking at this BZ, I thought it would be best fit into the Scheduler topic:

https://docs.openshift.com/enterprise/3.1/admin_guide/scheduler.html

From what I can tell, using the scheduler is the automatic version of using a nodeselector, right? You can point at where you want to pod to live, or get the scheduler to do it for you. I'd say info on the nodeselector would be more at home here. Then maybe a topic name change to something like "Determining Pod Placement" or something.

Am I missing something?
Comment 9 Erik M Jacobs 2016-01-27 21:08:12 EST
Unfortunately what we're talking about is not part of scheduler config, but rather is *informed by* scheduler config.

For example, take the following topology:

I have two regions (A, B) and two zones within each region (1,2).

For argument's sake, I want to place all development projects in region B. Via the *scheduler alone* there is no way to do this.

I would have to implement a nodeselector on all development projects that specifies "region=b".

Yes, node labels and scheduler config is involved, but this topic (nodeselector info) is still light.

Comment #5 and #6 lay out the case for an improvement in the overall structure of the project administration topic, and, if we go that route, I think will clear up some of the confusion on nodeselector, scheduler and etc.

Perhaps there's more clarification needed on how nodeselector *relates* to scheduler, but, more specifically, to node labels. I believe you could have node labels without any specific scheduler config and still use nodeselector (eg: the built-in scheduler configuration as opposed to a customized config).

I hope this all makes sense!
Comment 10 brice 2016-01-28 01:25:12 EST
Ok. Tagged all relevant in this draft PR:

https://github.com/openshift/openshift-docs/pull/1490

If anyone has any issues, please let me know either here or in the PR.
Comment 11 openshift-github-bot 2016-03-02 23:35:55 EST
Commit pushed to master at https://github.com/openshift/openshift-docs

https://github.com/openshift/openshift-docs/commit/cc3d79ec3925ddf48e1cb4e472f1e9f209fd33e1
Merge pull request #1490 from bfallonf/BZ1253125

Bug 1253125 : added nodeselector information

Note You need to log in before you can comment on or make changes to this bug.