Description of problem: This defect is split from https://bugzilla.redhat.com/show_bug.cgi?id=1548641#c21 the default node-selector is node-role.kubernetes.io/compute=true, there is a problem, see the following: 1) Create one project named 'hosa', node-selector is "" # oc get project -n hosa -o yaml apiVersion: project.openshift.io/v1 kind: Project metadata: annotations: openshift.io/description: "" openshift.io/display-name: "" openshift.io/requester: system:admin openshift.io/sa.scc.mcs: s0:c11,c10 openshift.io/sa.scc.supplemental-groups: 1000130000/10000 openshift.io/sa.scc.uid-range: 1000130000/10000 creationTimestamp: 2018-03-16T09:42:39Z *********************snipped************************************** 2) I have one hawkular-openshift-agent ds, which node-selector is not defined, this means it should start up one pod in each node, and there is not "node-role.kubernetes.io/compute=true" label on master node, the ds actually check if the master node is labeled with node-selector "node-role.kubernetes.io/compute=true", but it is not, so the hawkular-openshift-agent pod on master node would be recreated continuesly(defect is filed, https://bugzilla.redhat.com/show_bug.cgi?id=1543727), see the event Checked the running pod, its nodeSelector is node-role.kubernetes.io/compute: "true" # oc get hawkular-openshift-agent-6s6x4 -o yaml | grep nodeSelector -A 1 nodeSelector: node-role.kubernetes.io/compute: "true" Since the master node is not labeled with node-selector "node-role.kubernetes.io/compute=true", the pod on master node would be recreated continuesly # oc get event 54m 54m 1 hawkular-openshift-agent-l88s4.151b51d4b370b454 Pod Warning MatchNodeSelector kubelet, 172.16.120.93 Predicate MatchNodeSelector failed 1h 1h 1 hawkular-openshift-agent-l8bmg.151b4f89ec04badc Pod Warning MatchNodeSelector kubelet, 172.16.120.93 Predicate MatchNodeSelector failed 18m 18m 1 hawkular-openshift-agent-l8bnq.151b53d58897ac5d Pod Warning MatchNodeSelector kubelet, 172.16.120.93 Predicate MatchNodeSelector failed 18m 18m 1 hawkular-openshift-agent-l8cdn.151b53d37068bc5b Pod Warning MatchNodeSelector kubelet, 172.16.120.93 Predicate MatchNodeSelector failed 1h 1h 1 hawkular-openshift-agent-l8ct8.151b50c721482011 Pod Warning MatchNodeSelector kubelet, 172.16.120.93 Predicate MatchNodeSelector failed 49m 49m 1 hawkular-openshift-agent-l8f9g.151b521a453dd798 Pod Warning MatchNodeSelector kubelet, 172.16.120.93 Predicate MatchNodeSelector failed 1h 1h 1 hawkular-openshift-agent-l8gfg.151b5147fa1961db Pod Warning MatchNodeSelector kubelet, 172.16.120.93 Predicate MatchNodeSelector failed 31m 31m 1 hawkular-openshift-agent-l8gk9.151b531a79eff10e Pod Warning MatchNodeSelector kubelet, 172.16.120.93 Predicate MatchNodeSelector failed 18m 18m 1 hawkular-openshift-agent-l8hdz.151b53d2464b812a Pod Warning MatchNodeSelector kubelet, 172.16.120.93 Predicate MatchNodeSelector failed 1h 1h 1 hawkular-openshift-agent-l8hxs.151b4f15bcaf15af Pod Warning MatchNodeSelector kubelet, 172.16.120.93 Predicate MatchNodeSelector failed 1h 1h 1 hawkular-openshift-agent-l8m97.151b50156eaba408 Pod Warning MatchNodeSelector kubelet, 172.16.120.93 Predicate MatchNodeSelector failed 1h 1h 1 hawkular-openshift-agent-l8mwv.151b4f3e049f205a Pod Warning MatchNodeSelector kubelet, 172.16.120.93 Predicate MatchNodeSelector failed 1h 1h 1 hawkular-openshift-agent-l8ndn.151b4f65e0b4f99a Pod Warning MatchNodeSelector kubelet, 172.16.120.93 Predicate MatchNodeSelector failed 1h 1h 1 hawkular-openshift-agent-l8nsz.151b4f34c26ea5c4 Pod Warning MatchNodeSelector kubelet, 172.16.120.93 Predicate MatchNodeSelector failed 1h 1h 1 hawkular-openshift-agent-l8r9w.151b4e61aaf37da9 Pod Warning MatchNodeSelector kubelet, 172.16.120.93 Predicate MatchNodeSelector failed # oc get ds -n hosa NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE hawkular-openshift-agent 2 2 1 2 1 <none> 15h # oc get po -n hosa -o wide NAME READY STATUS RESTARTS AGE IP NODE hawkular-openshift-agent-6s6x4 1/1 Running 0 16h 10.129.0.26 172.16.120.78 hawkular-openshift-agent-j2wq4 0/1 Pending 0 0s <none> 172.16.120.93 # oc get po -n hosa -o wide NAME READY STATUS RESTARTS AGE IP NODE hawkular-openshift-agent-6s6x4 1/1 Running 0 16h 10.129.0.26 172.16.120.78 hawkular-openshift-agent-hrbhf 0/1 Pending 0 0s <none> 172.16.120.93 # oc get node --show-labels NAME STATUS ROLES AGE VERSION LABELS 172.16.120.78 Ready compute 18h v1.9.1+a0ce1bc657 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/instance-type=431ac1fb-1463-4527-b3d1-79245dd698e1,beta.kubernetes.io/os=linux,failure-domain.beta.kubernetes.io/region=regionOne,failure-domain.beta.kubernetes.io/zone=nova,kubernetes.io/hostname=172.16.120.78,node-role.kubernetes.io/compute=true,registry=enabled,role=node,router=enabled 172.16.120.93 Ready master 18h v1.9.1+a0ce1bc657 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/instance-type=431ac1fb-1463-4527-b3d1-79245dd698e1,beta.kubernetes.io/os=linux,failure-domain.beta.kubernetes.io/region=regionOne,failure-domain.beta.kubernetes.io/zone=nova,kubernetes.io/hostname=172.16.120.93,node-role.kubernetes.io/master=true,openshift-infra=apiserver,role=node Version-Release number of selected component (if applicable): openshift v3.9.9 kubernetes v1.9.1+a0ce1bc657 etcd 3.2.16 How reproducible: Always Steps to Reproduce: 1. See the description part 2. 3. Actual results: Expected results: Additional info:
correction 1) Create one project named 'hosa', node-selector is "" change to 1) Create one project named 'hosa', node-selector is not defined in project
> I have one hawkular-openshift-agent ds, which node-selector is not defined, this means it should start up one pod in each node Before you start using daemonset you need to make sure to disable project default node selector in your namespace. Documentation for 3.9.0 is being written here https://github.com/openshift/openshift-docs/pull/8125 oc patch namespace myproject -p \ '{"metadata": {"annotations": {"openshift.io/node-selector": ""}}}' *** This bug has been marked as a duplicate of bug 1501514 ***