Bug 1377471 - Adding defaultNodeSelector for projectConfig restricts all projects.
Summary: Adding defaultNodeSelector for projectConfig restricts all projects.
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Node
Version: 3.2.1
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
: ---
Assignee: Avesh Agarwal
QA Contact: DeShuai Ma
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-09-19 19:04 UTC by Ryan Howe
Modified: 2019-12-16 06:49 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-10-25 16:41:33 UTC
Target Upstream Version:


Attachments (Terms of Use)

Description Ryan Howe 2016-09-19 19:04:18 UTC
Description of problem:
  
   After installing OpenShift the system admin sets projectConfig.defaultNodeSelector, all pods can then only be scheduled to a node with this selector even default and openshift projects. 

The node-selector the project or namespace object never gets set, the nodeselector is hidden.
 
  openshift.io/node-selector: "key:value"      


Version-Release number of selected component (if applicable):
openshift v3.2.1.15
kubernetes v1.2.0-36-g4a3f9c5



How reproducible:
100% 


Steps to Reproduce:
1. Install OpenShift
2. Schedule Router to nodeselector "kubernetes.io/hostname=node-2.openshift.com"
3. Set projectConfig.defaultNodeSelector in master-config to "kubernetes.io/hostname=node-1.openshift.com", restart masters
4. Deploy latest router 

Actual results:

EVENTS

15m       15m       1         router-4-deploy   Pod                 Warning   FailedScheduling   {default-scheduler }   pod (router-4-deploy) failed to fit in any node
fit failure on node (node-2.openshift.com): MatchNodeSelector
fit failure on node (master-0.openshift.com): MatchNodeSelector
fit failure on node (master-1.openshift.com): MatchNodeSelector
fit failure on node (master-2.openshift.com): MatchNodeSelector
fit failure on node (node-0.openshift.com): MatchNodeSelector
fit failure on node (node-1.openshift.com): MatchNodeSelector

NAMESPACE 

# oc get namespace default -o yaml | grep "node-selector"
--NOTHING   

Expected results:

   Pod to get scheduled 
   # oc get namespace default -o yaml | grep "node-selector"
    openshift.io/node-selector: "kubernetes.io/hostname=node-1.openshift.com" 

Additional info:

  Workaround is to add blank value for openshift.io/node-selector

apiVersion: v1
kind: Namespace
metadata:
  annotations:
    openshift.io/node-selector: ""

Comment 2 Avesh Agarwal 2016-09-30 14:21:15 UTC
Yeah, will update it with my feedback.

Comment 3 Avesh Agarwal 2016-09-30 15:45:41 UTC
I tested this on latest origin (git branch master HEAD) and I can not reproduce this bug and everything seems working as expected.

Here are my steps:

1. Started an openshift master, and the master config has:
projectConfig:
  defaultNodeSelector: "kubernetes.io/hostname=192.168.124.62"

(The node 192.168.124.62 does not exist in my setup)
 
2. Started an openshift node (192.168.124.61) and it has one of the labels as:
kubernetes.io/hostname=192.168.124.61

3. Edited the default namespace and gave an annotation as:
openshift.io/node-selector: kubernetes.io/hostname=192.168.124.61

4. Created a test pod in the default namespace. 

Result: the pod gets scheduled on 192.168.124.61 and runs happily. And it is expected because the namespace annotation (kubernetes.io/hostname=192.168.124.61) takes preference over defaultNodeselector (kubernetes.io/hostname=192.168.124.62).

5. To verify further, I deleted the test pod, and then I edited the default namepspace again and removed the annotation from the default namespace:  openshift.io/node-selector: kubernetes.io/hostname=192.168.124.61

6. Now created the test pod again in the default namespace, and the pods is in pending status as follows:
Events:

  FirstSeen	LastSeen	Count	From			SubobjectPath	Type		Reason			Message
  ---------	--------	-----	----			-------------	--------	------			-------
  11m		15s		42	{default-scheduler }			Warning		FailedScheduling	pod (dapi-test-pod) failed to fit in any node
fit failure on node (192.168.124.61): MatchNodeSelector

And this is also expected because the defaultNodeSelector takes preference as the default namepspace does not have any node selector related annotation. And since the node (192.168.124.62) does not exist, the pod stays in pending status unavailable to find a matching node.


Also I said that I have not only tested on the latest origin master branch from github.

Comment 4 Avesh Agarwal 2016-09-30 15:49:53 UTC
I meant I have only tested on the latest origin master branch from github. Will try to reproduce on 3.2.1.

Comment 5 Avesh Agarwal 2016-09-30 15:57:42 UTC
And also after step 3 in the comment above https://bugzilla.redhat.com/show_bug.cgi?id=1377471#c3 : I was able to see:

#oc get namespace default -o yaml  | grep "node-selector"
    openshift.io/node-selector: kubernetes.io/hostname=192.168.124.61

Comment 9 Avesh Agarwal 2016-10-25 16:41:33 UTC
Closing it now, please open if there is any concern.


Note You need to log in before you can comment on or make changes to this bug.