Hide Forgot
Description of problem: where we could not deploy the docker-registry anymore, as a blacklisted label was defined as node-selector on namespace and dc level. after removing the node-selector on dc level, it is working. This was working with previous version 3.2 of OpenShift. Version-Release number of selected component (if applicable): 3.3 Steps to reproduce: 1. configure the following label on masters: example-cloud.com/role=master 2. blacklist the label for normal users in master-config.yaml: admissionConfig: pluginConfig: PodNodeConstraints: configuration: apiversion: v1 kind: PodNodeConstraintsConfig nodeSelectorLabelBlacklist: - example-cloud.com/role (and in the second place where this needs to go) 3. create a new project and add as node-selector by editing the namespace: openshift.io/node-selector: example-cloud.com/role=master 4. deploy a simple application: oc new-app sonatype/nexus -> nexus is deployed on a master 5. edit dc nexus and add as node-selector: nodeSelector: example-cloud.com/role: master -> the deployment fails with the following shown in oc get event: 25m 25m 2 nexus DeploymentConfig Warning FailedRetry {deployments-controller } nexus-2: About to stop retrying nexus-2: couldn't create deployer pod for openpaas-blacklabel-test/nexus-2: pods "nexus-2-deploy" is forbidden: node selection by label(s) [example-cloud.com/role] is prohibited by policy for your role I am cluster-admin: [ansible@mon-node ~]$ oc whoami system:admin Actual results: Not working with cluster-admin user Expected results: Blacklist label should work with cluster-admin user Additional info:
even though you are a cluster admin, the deployer pod is created by the deployment controller
Was there any changes performed in 3.3 where it worked with 3.2 previously. I have only reproduced this issue with 3.3 but customer have tried with both versions
> Was there any changes performed in 3.3 where it worked with 3.2 previously. I have only reproduced this issue with 3.3 but customer have tried with both versions No changes I can see that would affect this. Was the PodNodeConstraints admission plugin enabled in the customer's 3.2 environment? I would not expect to blacklist the example-cloud.com/role label, since it seems like they would want to force normal user projects to go to a non-master node with a example-cloud.com/role=workload project node selector.
Is the intention here to use the blacklisted label as a project nodeSelector? That will force all pods (including the deployer pod) to have the blacklisted label. Note that the deployer pod runs under different SA.
So the intention here is cluster-admin should have the rights to perform all the tasks including the deployments to blacklisted labeled nodes. So that mean cluster-admin do not have control on deployer pod having different serviceaccount ? The interesting thing customer pointed is this worked in 3.2 but not on 3.3.1
nodeSelectorLabelBlacklist is intended to prevent users targeting *specific nodes* by directly specifying labels that map to an individual node, like the kubernetes.io/hostname label. It does *not* prevent the scheduler from assigning a pod to run on a node with a blacklisted label. The defaultNodeSelector and namespace openshift.io/node-selector annotation are the correct way to restrict the *set* of nodes a namespace is allowed to run pods on. Only users with permission to bind pods to particular nodes are allowed to bypass the blacklisted node-targeting labels. To see which users are allowed to do that, run this command: oadm policy who-can create pods/binding Even though a cluster admin has that permission, the controller that creates pods for a deployment does not. The only way I can see a blacklisted label would have been allowed in a DeploymentConfig is if the controllers that create pods were given bind permission. Can you run these commands on the 3.2 and 3.3 environments and see if there are differences? oadm policy who-can create pods/binding --namespace=blacklist-test oadm policy who-can create pods/binding --all-namespaces