Description of problem: 2 e2e InterPodAffinity/InterPodAntiAffinity tests are failing on OCP 3.5: [It] validates that InterPodAntiAffinity is respected if matching 2 [It] validates that InterPod Affinity and AntiAffinity is respected if matching Output for the 2 tests below. Sample failure: [91m[1m• Failure [44.074 seconds][0m [k8s.io] SchedulerPredicates [Serial] [90m/builddir/build/BUILD/atomic-openshift-git-0.86a6117/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:826[0m [91m[1mvalidates that InterPodAntiAffinity is respected if matching 2 [It][0m [90m/builddir/build/BUILD/atomic-openshift-git-0.86a6117/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:564[0m [91mExpected error: <*errors.StatusError | 0xc42199c300>: { ErrStatus: { TypeMeta: {Kind: "", APIVersion: ""}, ListMeta: {SelfLink: "", ResourceVersion: ""}, Status: "Failure", Message: "pods \"with-podantiaffinity-c4a87965-dc27-11e6-a515-0208cfd56fdf\" is forbidden: affinity.PodAntiAffinity.RequiredDuringScheduling has TopologyKey e2e.inter-pod-affinity.kubernetes.io/zone but only key kubernetes.io/hostname is allowed", Reason: "Forbidden", Details: { Name: "with-podantiaffinity-c4a87965-dc27-11e6-a515-0208cfd56fdf", Group: "", Kind: "pods", Causes: nil, RetryAfterSeconds: 0, }, Code: 403, }, } pods "with-podantiaffinity-c4a87965-dc27-11e6-a515-0208cfd56fdf" is forbidden: affinity.PodAntiAffinity.RequiredDuringScheduling has TopologyKey e2e.inter-pod-affinity.kubernetes.io/zone but only key kubernetes.io/hostname is allowed not to have occurred[0m Version-Release number of selected component (if applicable): 3.5.0.4 How reproducible: Always Steps to Reproduce: 1. Install cluster on AWS 2. Run the failing e2es listed above Additional info: [k8s.io] SchedulerPredicates [Serial] [90m/builddir/build/BUILD/atomic-openshift-git-0.86a6117/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:826[0m validates that required NodeAffinity setting is respected if matching [90m/builddir/build/BUILD/atomic-openshift-git-0.86a6117/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:373[0m [90m------------------------------[0m [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [0m[k8s.io] SchedulerPredicates [Serial][0m [1mvalidates that InterPodAntiAffinity is respected if matching 2[0m [37m/builddir/build/BUILD/atomic-openshift-git-0.86a6117/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:564[0m [BeforeEach] [Top Level] /builddir/build/BUILD/atomic-openshift-git-0.86a6117/_output/local/go/src/github.com/openshift/origin/test/extended/util/test.go:47 [BeforeEach] [k8s.io] SchedulerPredicates [Serial] /builddir/build/BUILD/atomic-openshift-git-0.86a6117/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:141 [1mSTEP[0m: Creating a kubernetes client Jan 16 15:09:51.890: INFO: >>> kubeConfig: /etc/origin/master/admin.kubeconfig [1mSTEP[0m: Building a namespace api object Jan 16 15:09:51.970: INFO: About to run a Kube e2e test, ensuring namespace is privileged [1mSTEP[0m: Waiting for a default service account to be provisioned in namespace [BeforeEach] [k8s.io] SchedulerPredicates [Serial] /builddir/build/BUILD/atomic-openshift-git-0.86a6117/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:100 Jan 16 15:09:52.039: INFO: Waiting up to 1m0s for all (but 0) nodes to be ready Jan 16 15:09:52.046: INFO: Waiting for terminating namespaces to be deleted... Jan 16 15:09:52.051: INFO: Waiting up to 5m0s for all pods (need at least 0) in namespace 'kube-system' to be running and ready Jan 16 15:09:52.054: INFO: Waiting for pods to enter Success, but no pods in "kube-system" match label map[name:e2e-image-puller] Jan 16 15:09:52.057: INFO: 0 / 0 pods in namespace 'kube-system' are running and ready (0 seconds elapsed) Jan 16 15:09:52.057: INFO: expected 0 pod replicas in namespace 'kube-system', 0 are Running and Ready. Jan 16 15:09:52.057: INFO: Logging pods the kubelet thinks is on node ip-172-31-34-189.us-west-2.compute.internal before test Jan 16 15:09:52.062: INFO: Logging pods the kubelet thinks is on node ip-172-31-35-171.us-west-2.compute.internal before test Jan 16 15:09:52.067: INFO: registry-console-1-zt4kz from default started at 2017-01-16 08:15:30 -0500 EST (1 container statuses recorded) Jan 16 15:09:52.067: INFO: Container registry-console ready: true, restart count 0 Jan 16 15:09:52.068: INFO: Logging pods the kubelet thinks is on node ip-172-31-40-79.us-west-2.compute.internal before test Jan 16 15:09:52.073: INFO: docker-registry-5-1y3cm from default started at 2017-01-16 11:22:35 -0500 EST (1 container statuses recorded) Jan 16 15:09:52.073: INFO: Container registry ready: true, restart count 0 Jan 16 15:09:52.073: INFO: router-1-98ocr from default started at 2017-01-16 08:15:25 -0500 EST (1 container statuses recorded) Jan 16 15:09:52.073: INFO: Container router ready: true, restart count 0 [It] validates that InterPodAntiAffinity is respected if matching 2 /builddir/build/BUILD/atomic-openshift-git-0.86a6117/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:564 [1mSTEP[0m: Launching two pods on two distinct nodes to get two node names [1mSTEP[0m: Running RC which reserves host port [1mSTEP[0m: creating replication controller host-port in namespace e2e-tests-sched-pred-xglum I0116 15:09:52.078941 116021 runners.go:103] Created replication controller with name: host-port, namespace: e2e-tests-sched-pred-xglum, replica count: 2 I0116 15:09:52.078980 116021 reflector.go:196] Starting reflector *api.Pod (0s) from github.com/openshift/origin/vendor/k8s.io/kubernetes/test/utils/pod_store.go:52 I0116 15:09:52.079014 116021 reflector.go:234] Listing and watching *api.Pod from github.com/openshift/origin/vendor/k8s.io/kubernetes/test/utils/pod_store.go:52 I0116 15:10:02.079197 116021 runners.go:103] host-port Pods: 2 out of 2 created, 2 running, 0 pending, 0 waiting, 0 inactive, 0 terminating, 0 unknown, 0 runningButNotReady [1mSTEP[0m: Applying a random label to both nodes. [1mSTEP[0m: verifying the node has the label e2e.inter-pod-affinity.kubernetes.io/zone china-e2etest [1mSTEP[0m: verifying the node has the label e2e.inter-pod-affinity.kubernetes.io/zone china-e2etest [1mSTEP[0m: Trying to launch another pod on the first node with the service label. [1mSTEP[0m: Trying to launch another pod, now with podAntiAffinity with same Labels. Jan 16 15:10:04.263: INFO: Unexpected error occurred: pods "with-podantiaffinity-c4a87965-dc27-11e6-a515-0208cfd56fdf" is forbidden: affinity.PodAntiAffinity.RequiredDuringScheduling has TopologyKey e2e.inter-pod-affinity.kubernetes.io/zone but only key kubernetes.io/hostname is allowed [1mSTEP[0m: removing the label e2e.inter-pod-affinity.kubernetes.io/zone off the node ip-172-31-35-171.us-west-2.compute.internal [1mSTEP[0m: verifying the node doesn't have the label e2e.inter-pod-affinity.kubernetes.io/zone [1mSTEP[0m: removing the label e2e.inter-pod-affinity.kubernetes.io/zone off the node ip-172-31-40-79.us-west-2.compute.internal [1mSTEP[0m: verifying the node doesn't have the label e2e.inter-pod-affinity.kubernetes.io/zone [1mSTEP[0m: deleting replication controller host-port in namespace e2e-tests-sched-pred-xglum I0116 15:10:04.293563 116021 reflector.go:196] Starting reflector *api.Pod (0s) from github.com/openshift/origin/vendor/k8s.io/kubernetes/test/utils/pod_store.go:52 I0116 15:10:04.293617 116021 reflector.go:234] Listing and watching *api.Pod from github.com/openshift/origin/vendor/k8s.io/kubernetes/test/utils/pod_store.go:52 Jan 16 15:10:05.344: INFO: Deleting RC host-port took: 50.656747ms Jan 16 15:10:05.344: INFO: Terminating RC host-port pods took: 30.556µs [AfterEach] [k8s.io] SchedulerPredicates [Serial] /builddir/build/BUILD/atomic-openshift-git-0.86a6117/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:142 [1mSTEP[0m: Collecting events from namespace "e2e-tests-sched-pred-xglum". [1mSTEP[0m: Found 18 events. Jan 16 15:10:15.348: INFO: At 2017-01-16 15:09:52 -0500 EST - event for host-port: {replication-controller } SuccessfulCreate: Created pod: host-port-xqnkt Jan 16 15:10:15.348: INFO: At 2017-01-16 15:09:52 -0500 EST - event for host-port: {replication-controller } SuccessfulCreate: Created pod: host-port-zoi01 Jan 16 15:10:15.348: INFO: At 2017-01-16 15:09:52 -0500 EST - event for host-port-xqnkt: {default-scheduler } Scheduled: Successfully assigned host-port-xqnkt to ip-172-31-40-79.us-west-2.compute.internal Jan 16 15:10:15.348: INFO: At 2017-01-16 15:09:52 -0500 EST - event for host-port-zoi01: {default-scheduler } Scheduled: Successfully assigned host-port-zoi01 to ip-172-31-35-171.us-west-2.compute.internal Jan 16 15:10:15.348: INFO: At 2017-01-16 15:09:53 -0500 EST - event for host-port-xqnkt: {kubelet ip-172-31-40-79.us-west-2.compute.internal} Pulled: Container image "gcr.io/google_containers/pause-amd64:3.0" already present on machine Jan 16 15:10:15.348: INFO: At 2017-01-16 15:09:53 -0500 EST - event for host-port-xqnkt: {kubelet ip-172-31-40-79.us-west-2.compute.internal} Created: Created container with docker id 87ced1bc7a6e; Security:[seccomp=unconfined] Jan 16 15:10:15.348: INFO: At 2017-01-16 15:09:53 -0500 EST - event for host-port-zoi01: {kubelet ip-172-31-35-171.us-west-2.compute.internal} Pulled: Container image "gcr.io/google_containers/pause-amd64:3.0" already present on machine Jan 16 15:10:15.348: INFO: At 2017-01-16 15:09:53 -0500 EST - event for host-port-zoi01: {kubelet ip-172-31-35-171.us-west-2.compute.internal} Created: Created container with docker id 44927f8b7363; Security:[seccomp=unconfined] Jan 16 15:10:15.348: INFO: At 2017-01-16 15:09:53 -0500 EST - event for host-port-zoi01: {kubelet ip-172-31-35-171.us-west-2.compute.internal} Started: Started container with docker id 44927f8b7363 Jan 16 15:10:15.348: INFO: At 2017-01-16 15:09:54 -0500 EST - event for host-port-xqnkt: {kubelet ip-172-31-40-79.us-west-2.compute.internal} Started: Started container with docker id 87ced1bc7a6e Jan 16 15:10:15.348: INFO: At 2017-01-16 15:10:02 -0500 EST - event for with-label-c3601ca6-dc27-11e6-a515-0208cfd56fdf: {default-scheduler } Scheduled: Successfully assigned with-label-c3601ca6-dc27-11e6-a515-0208cfd56fdf to ip-172-31-35-171.us-west-2.compute.internal Jan 16 15:10:15.348: INFO: At 2017-01-16 15:10:03 -0500 EST - event for with-label-c3601ca6-dc27-11e6-a515-0208cfd56fdf: {kubelet ip-172-31-35-171.us-west-2.compute.internal} Created: Created container with docker id e2bc17a54db9; Security:[seccomp=unconfined] Jan 16 15:10:15.348: INFO: At 2017-01-16 15:10:03 -0500 EST - event for with-label-c3601ca6-dc27-11e6-a515-0208cfd56fdf: {kubelet ip-172-31-35-171.us-west-2.compute.internal} Pulled: Container image "gcr.io/google_containers/pause-amd64:3.0" already present on machine Jan 16 15:10:15.348: INFO: At 2017-01-16 15:10:04 -0500 EST - event for with-label-c3601ca6-dc27-11e6-a515-0208cfd56fdf: {kubelet ip-172-31-35-171.us-west-2.compute.internal} Started: Started container with docker id e2bc17a54db9 Jan 16 15:10:15.348: INFO: At 2017-01-16 15:10:05 -0500 EST - event for host-port: {replication-controller } SuccessfulDelete: Deleted pod: host-port-xqnkt Jan 16 15:10:15.348: INFO: At 2017-01-16 15:10:05 -0500 EST - event for host-port: {replication-controller } SuccessfulDelete: Deleted pod: host-port-zoi01 Jan 16 15:10:15.348: INFO: At 2017-01-16 15:10:05 -0500 EST - event for host-port-xqnkt: {kubelet ip-172-31-40-79.us-west-2.compute.internal} Killing: Killing container with docker id 87ced1bc7a6e: Need to kill pod. Jan 16 15:10:15.348: INFO: At 2017-01-16 15:10:05 -0500 EST - event for host-port-zoi01: {kubelet ip-172-31-35-171.us-west-2.compute.internal} Killing: Killing container with docker id 44927f8b7363: Need to kill pod. Jan 16 15:10:15.358: INFO: POD NODE PHASE GRACE CONDITIONS Jan 16 15:10:15.358: INFO: docker-registry-5-1y3cm ip-172-31-40-79.us-west-2.compute.internal Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2017-01-16 11:22:35 -0500 EST } {Ready True 0001-01-01 00:00:00 +0000 UTC 2017-01-16 11:22:38 -0500 EST } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2017-01-16 11:22:35 -0500 EST }] Jan 16 15:10:15.358: INFO: registry-console-1-zt4kz ip-172-31-35-171.us-west-2.compute.internal Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2017-01-16 08:15:30 -0500 EST } {Ready True 0001-01-01 00:00:00 +0000 UTC 2017-01-16 08:16:10 -0500 EST } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2017-01-16 08:15:30 -0500 EST }] Jan 16 15:10:15.358: INFO: router-1-98ocr ip-172-31-40-79.us-west-2.compute.internal Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2017-01-16 08:15:25 -0500 EST } {Ready True 0001-01-01 00:00:00 +0000 UTC 2017-01-16 08:15:45 -0500 EST } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2017-01-16 08:15:25 -0500 EST }] Jan 16 15:10:15.358: INFO: with-label-c3601ca6-dc27-11e6-a515-0208cfd56fdf ip-172-31-35-171.us-west-2.compute.internal Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2017-01-16 15:10:02 -0500 EST } {Ready True 0001-01-01 00:00:00 +0000 UTC 2017-01-16 15:10:04 -0500 EST } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2017-01-16 15:10:02 -0500 EST }] Jan 16 15:10:15.358: INFO: Jan 16 15:10:15.362: INFO: Logging node info for node ip-172-31-34-189.us-west-2.compute.internal Jan 16 15:10:15.365: INFO: Node Info: &TypeMeta{Kind:,APIVersion:,} Jan 16 15:10:15.365: INFO: Logging kubelet events for node ip-172-31-34-189.us-west-2.compute.internal Jan 16 15:10:15.367: INFO: Logging pods the kubelet thinks is on node ip-172-31-34-189.us-west-2.compute.internal W0116 15:10:15.375846 116021 metrics_grabber.go:73] Master node is not registered. Grabbing metrics from Scheduler and ControllerManager is disabled. Jan 16 15:10:15.473: INFO: Latency metrics for node ip-172-31-34-189.us-west-2.compute.internal Jan 16 15:10:15.473: INFO: Logging node info for node ip-172-31-35-171.us-west-2.compute.internal Jan 16 15:10:15.476: INFO: Node Info: &TypeMeta{Kind:,APIVersion:,} Jan 16 15:10:15.476: INFO: Logging kubelet events for node ip-172-31-35-171.us-west-2.compute.internal Jan 16 15:10:15.478: INFO: Logging pods the kubelet thinks is on node ip-172-31-35-171.us-west-2.compute.internal Jan 16 15:10:15.483: INFO: registry-console-1-zt4kz started at 2017-01-16 08:15:30 -0500 EST (0+1 container statuses recorded) Jan 16 15:10:15.483: INFO: Container registry-console ready: true, restart count 0 Jan 16 15:10:15.483: INFO: with-label-c3601ca6-dc27-11e6-a515-0208cfd56fdf started at 2017-01-16 15:10:02 -0500 EST (0+1 container statuses recorded) Jan 16 15:10:15.483: INFO: Container pfpod ready: true, restart count 0 W0116 15:10:15.487237 116021 metrics_grabber.go:73] Master node is not registered. Grabbing metrics from Scheduler and ControllerManager is disabled. Jan 16 15:10:15.636: INFO: Latency metrics for node ip-172-31-35-171.us-west-2.compute.internal Jan 16 15:10:15.636: INFO: Logging node info for node ip-172-31-40-79.us-west-2.compute.internal Jan 16 15:10:15.639: INFO: Node Info: &TypeMeta{Kind:,APIVersion:,} Jan 16 15:10:15.639: INFO: Logging kubelet events for node ip-172-31-40-79.us-west-2.compute.internal Jan 16 15:10:15.641: INFO: Logging pods the kubelet thinks is on node ip-172-31-40-79.us-west-2.compute.internal Jan 16 15:10:15.647: INFO: router-1-98ocr started at 2017-01-16 08:15:25 -0500 EST (0+1 container statuses recorded) Jan 16 15:10:15.647: INFO: Container router ready: true, restart count 0 Jan 16 15:10:15.647: INFO: docker-registry-5-1y3cm started at 2017-01-16 11:22:35 -0500 EST (0+1 container statuses recorded) Jan 16 15:10:15.647: INFO: Container registry ready: true, restart count 0 W0116 15:10:15.650623 116021 metrics_grabber.go:73] Master node is not registered. Grabbing metrics from Scheduler and ControllerManager is disabled. Jan 16 15:10:15.798: INFO: Latency metrics for node ip-172-31-40-79.us-west-2.compute.internal Jan 16 15:10:15.798: INFO: Logging node info for node ip-172-31-58-91.us-west-2.compute.internal Jan 16 15:10:15.800: INFO: Node Info: &TypeMeta{Kind:,APIVersion:,} Jan 16 15:10:15.800: INFO: Logging kubelet events for node ip-172-31-58-91.us-west-2.compute.internal Jan 16 15:10:15.802: INFO: Logging pods the kubelet thinks is on node ip-172-31-58-91.us-west-2.compute.internal W0116 15:10:15.812928 116021 metrics_grabber.go:73] Master node is not registered. Grabbing metrics from Scheduler and ControllerManager is disabled. Jan 16 15:10:15.890: INFO: Latency metrics for node ip-172-31-58-91.us-west-2.compute.internal [1mSTEP[0m: Dumping a list of prepulled images on each node Jan 16 15:10:15.892: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [1mSTEP[0m: Destroying namespace "e2e-tests-sched-pred-xglum" for this suite. Jan 16 15:10:35.942: INFO: namespace: e2e-tests-sched-pred-xglum, resource: bindings, ignored listing per whitelist [AfterEach] [k8s.io] SchedulerPredicates [Serial] /builddir/build/BUILD/atomic-openshift-git-0.86a6117/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:67 I0116 15:10:35.964419 116021 request.go:768] Error in request: resource name may not be empty [91m[1m• Failure [44.074 seconds][0m [k8s.io] SchedulerPredicates [Serial] [90m/builddir/build/BUILD/atomic-openshift-git-0.86a6117/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:826[0m [91m[1mvalidates that InterPodAntiAffinity is respected if matching 2 [It][0m [90m/builddir/build/BUILD/atomic-openshift-git-0.86a6117/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:564[0m [91mExpected error: <*errors.StatusError | 0xc42199c300>: { ErrStatus: { TypeMeta: {Kind: "", APIVersion: ""}, ListMeta: {SelfLink: "", ResourceVersion: ""}, Status: "Failure", Message: "pods \"with-podantiaffinity-c4a87965-dc27-11e6-a515-0208cfd56fdf\" is forbidden: affinity.PodAntiAffinity.RequiredDuringScheduling has TopologyKey e2e.inter-pod-affinity.kubernetes.io/zone but only key kubernetes.io/hostname is allowed", Reason: "Forbidden", Details: { Name: "with-podantiaffinity-c4a87965-dc27-11e6-a515-0208cfd56fdf", Group: "", Kind: "pods", Causes: nil, RetryAfterSeconds: 0, }, Code: 403, }, } pods "with-podantiaffinity-c4a87965-dc27-11e6-a515-0208cfd56fdf" is forbidden: affinity.PodAntiAffinity.RequiredDuringScheduling has TopologyKey e2e.inter-pod-affinity.kubernetes.io/zone but only key kubernetes.io/hostname is allowed not to have occurred[0m ==================================================== [k8s.io] SchedulerPredicates [Serial] [90m/builddir/build/BUILD/atomic-openshift-git-0.86a6117/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:826[0m validates that embedding the JSON PodAffinity and PodAntiAffinity setting as a string in the annotation value work [90m/builddir/build/BUILD/atomic-openshift-git-0.86a6117/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:664[0m [90m------------------------------[0m [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [0m[k8s.io] SchedulerPredicates [Serial][0m [1mvalidates that InterPod Affinity and AntiAffinity is respected if matching[0m [37m/builddir/build/BUILD/atomic-openshift-git-0.86a6117/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:640[0m [BeforeEach] [Top Level] /builddir/build/BUILD/atomic-openshift-git-0.86a6117/_output/local/go/src/github.com/openshift/origin/test/extended/util/test.go:47 [BeforeEach] [k8s.io] SchedulerPredicates [Serial] /builddir/build/BUILD/atomic-openshift-git-0.86a6117/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:141 [1mSTEP[0m: Creating a kubernetes client Jan 16 15:15:26.816: INFO: >>> kubeConfig: /etc/origin/master/admin.kubeconfig [1mSTEP[0m: Building a namespace api object Jan 16 15:15:26.901: INFO: About to run a Kube e2e test, ensuring namespace is privileged [1mSTEP[0m: Waiting for a default service account to be provisioned in namespace [BeforeEach] [k8s.io] SchedulerPredicates [Serial] /builddir/build/BUILD/atomic-openshift-git-0.86a6117/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:100 Jan 16 15:15:26.968: INFO: Waiting up to 1m0s for all (but 0) nodes to be ready Jan 16 15:15:26.975: INFO: Waiting for terminating namespaces to be deleted... Jan 16 15:15:26.980: INFO: Waiting up to 5m0s for all pods (need at least 0) in namespace 'kube-system' to be running and ready Jan 16 15:15:26.982: INFO: Waiting for pods to enter Success, but no pods in "kube-system" match label map[name:e2e-image-puller] Jan 16 15:15:26.986: INFO: 0 / 0 pods in namespace 'kube-system' are running and ready (0 seconds elapsed) Jan 16 15:15:26.986: INFO: expected 0 pod replicas in namespace 'kube-system', 0 are Running and Ready. Jan 16 15:15:26.986: INFO: Logging pods the kubelet thinks is on node ip-172-31-34-189.us-west-2.compute.internal before test Jan 16 15:15:26.991: INFO: Logging pods the kubelet thinks is on node ip-172-31-35-171.us-west-2.compute.internal before test Jan 16 15:15:26.996: INFO: registry-console-1-zt4kz from default started at 2017-01-16 08:15:30 -0500 EST (1 container statuses recorded) Jan 16 15:15:26.996: INFO: Container registry-console ready: true, restart count 0 Jan 16 15:15:26.996: INFO: Logging pods the kubelet thinks is on node ip-172-31-40-79.us-west-2.compute.internal before test Jan 16 15:15:27.002: INFO: router-1-98ocr from default started at 2017-01-16 08:15:25 -0500 EST (1 container statuses recorded) Jan 16 15:15:27.002: INFO: Container router ready: true, restart count 0 Jan 16 15:15:27.002: INFO: docker-registry-5-1y3cm from default started at 2017-01-16 11:22:35 -0500 EST (1 container statuses recorded) Jan 16 15:15:27.002: INFO: Container registry ready: true, restart count 0 [It] validates that InterPod Affinity and AntiAffinity is respected if matching /builddir/build/BUILD/atomic-openshift-git-0.86a6117/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:640 [1mSTEP[0m: Trying to launch a pod with a label to get a node which can launch it. [1mSTEP[0m: Trying to apply a random label on the found node. [1mSTEP[0m: verifying the node has the label e2e.inter-pod-affinity.kubernetes.io/zone e2e-testing [1mSTEP[0m: Trying to launch the pod, now with Pod affinity and anti affinity. Jan 16 15:15:29.965: INFO: Unexpected error occurred: pods "with-podantiaffinity-86ca2603-dc28-11e6-a515-0208cfd56fdf" is forbidden: affinity.PodAntiAffinity.RequiredDuringScheduling has TopologyKey e2e.inter-pod-affinity.kubernetes.io/zone but only key kubernetes.io/hostname is allowed [1mSTEP[0m: removing the label e2e.inter-pod-affinity.kubernetes.io/zone off the node ip-172-31-34-189.us-west-2.compute.internal [1mSTEP[0m: verifying the node doesn't have the label e2e.inter-pod-affinity.kubernetes.io/zone [AfterEach] [k8s.io] SchedulerPredicates [Serial] /builddir/build/BUILD/atomic-openshift-git-0.86a6117/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:142 [1mSTEP[0m: Collecting events from namespace "e2e-tests-sched-pred-sj8ls". [1mSTEP[0m: Found 4 events. Jan 16 15:15:29.981: INFO: At 2017-01-16 15:15:27 -0500 EST - event for with-label-85073525-dc28-11e6-a515-0208cfd56fdf: {default-scheduler } Scheduled: Successfully assigned with-label-85073525-dc28-11e6-a515-0208cfd56fdf to ip-172-31-34-189.us-west-2.compute.internal Jan 16 15:15:29.981: INFO: At 2017-01-16 15:15:28 -0500 EST - event for with-label-85073525-dc28-11e6-a515-0208cfd56fdf: {kubelet ip-172-31-34-189.us-west-2.compute.internal} Pulled: Container image "gcr.io/google_containers/pause-amd64:3.0" already present on machine Jan 16 15:15:29.981: INFO: At 2017-01-16 15:15:28 -0500 EST - event for with-label-85073525-dc28-11e6-a515-0208cfd56fdf: {kubelet ip-172-31-34-189.us-west-2.compute.internal} Created: Created container with docker id 1124ca57f561; Security:[seccomp=unconfined] Jan 16 15:15:29.981: INFO: At 2017-01-16 15:15:29 -0500 EST - event for with-label-85073525-dc28-11e6-a515-0208cfd56fdf: {kubelet ip-172-31-34-189.us-west-2.compute.internal} Started: Started container with docker id 1124ca57f561 Jan 16 15:15:29.991: INFO: POD NODE PHASE GRACE CONDITIONS Jan 16 15:15:29.991: INFO: docker-registry-5-1y3cm ip-172-31-40-79.us-west-2.compute.internal Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2017-01-16 11:22:35 -0500 EST } {Ready True 0001-01-01 00:00:00 +0000 UTC 2017-01-16 11:22:38 -0500 EST } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2017-01-16 11:22:35 -0500 EST }] Jan 16 15:15:29.991: INFO: registry-console-1-zt4kz ip-172-31-35-171.us-west-2.compute.internal Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2017-01-16 08:15:30 -0500 EST } {Ready True 0001-01-01 00:00:00 +0000 UTC 2017-01-16 08:16:10 -0500 EST } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2017-01-16 08:15:30 -0500 EST }] Jan 16 15:15:29.991: INFO: router-1-98ocr ip-172-31-40-79.us-west-2.compute.internal Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2017-01-16 08:15:25 -0500 EST } {Ready True 0001-01-01 00:00:00 +0000 UTC 2017-01-16 08:15:45 -0500 EST } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2017-01-16 08:15:25 -0500 EST }] Jan 16 15:15:29.991: INFO: with-label-85073525-dc28-11e6-a515-0208cfd56fdf ip-172-31-34-189.us-west-2.compute.internal Running [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2017-01-16 15:15:27 -0500 EST } {Ready True 0001-01-01 00:00:00 +0000 UTC 2017-01-16 15:15:29 -0500 EST } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2017-01-16 15:15:27 -0500 EST }] Jan 16 15:15:29.991: INFO: Jan 16 15:15:29.994: INFO: Logging node info for node ip-172-31-34-189.us-west-2.compute.internal Jan 16 15:15:29.997: INFO: Node Info: &TypeMeta{Kind:,APIVersion:,} Jan 16 15:15:29.997: INFO: Logging kubelet events for node ip-172-31-34-189.us-west-2.compute.internal Jan 16 15:15:29.999: INFO: Logging pods the kubelet thinks is on node ip-172-31-34-189.us-west-2.compute.internal Jan 16 15:15:30.005: INFO: with-label-85073525-dc28-11e6-a515-0208cfd56fdf started at 2017-01-16 15:15:27 -0500 EST (0+1 container statuses recorded) Jan 16 15:15:30.005: INFO: Container pfpod ready: true, restart count 0 W0116 15:15:30.008575 116021 metrics_grabber.go:73] Master node is not registered. Grabbing metrics from Scheduler and ControllerManager is disabled. Jan 16 15:15:30.161: INFO: Latency metrics for node ip-172-31-34-189.us-west-2.compute.internal Jan 16 15:15:30.161: INFO: {Operation: Method:pod_start_latency_microseconds Quantile:0.99 Latency:11.488931s} Jan 16 15:15:30.161: INFO: {Operation: Method:pod_start_latency_microseconds Quantile:0.9 Latency:11.309982s} Jan 16 15:15:30.161: INFO: Logging node info for node ip-172-31-35-171.us-west-2.compute.internal Jan 16 15:15:30.164: INFO: Node Info: &TypeMeta{Kind:,APIVersion:,} Jan 16 15:15:30.164: INFO: Logging kubelet events for node ip-172-31-35-171.us-west-2.compute.internal Jan 16 15:15:30.166: INFO: Logging pods the kubelet thinks is on node ip-172-31-35-171.us-west-2.compute.internal Jan 16 15:15:30.172: INFO: registry-console-1-zt4kz started at 2017-01-16 08:15:30 -0500 EST (0+1 container statuses recorded) Jan 16 15:15:30.172: INFO: Container registry-console ready: true, restart count 0 W0116 15:15:30.175457 116021 metrics_grabber.go:73] Master node is not registered. Grabbing metrics from Scheduler and ControllerManager is disabled. Jan 16 15:15:30.313: INFO: Latency metrics for node ip-172-31-35-171.us-west-2.compute.internal Jan 16 15:15:30.313: INFO: {Operation: Method:pod_start_latency_microseconds Quantile:0.99 Latency:15.173499s} Jan 16 15:15:30.313: INFO: {Operation: Method:pod_start_latency_microseconds Quantile:0.9 Latency:11.105399s} Jan 16 15:15:30.313: INFO: Logging node info for node ip-172-31-40-79.us-west-2.compute.internal Jan 16 15:15:30.316: INFO: Node Info: &TypeMeta{Kind:,APIVersion:,} Jan 16 15:15:30.316: INFO: Logging kubelet events for node ip-172-31-40-79.us-west-2.compute.internal Jan 16 15:15:30.318: INFO: Logging pods the kubelet thinks is on node ip-172-31-40-79.us-west-2.compute.internal Jan 16 15:15:30.324: INFO: router-1-98ocr started at 2017-01-16 08:15:25 -0500 EST (0+1 container statuses recorded) Jan 16 15:15:30.324: INFO: Container router ready: true, restart count 0 Jan 16 15:15:30.324: INFO: docker-registry-5-1y3cm started at 2017-01-16 11:22:35 -0500 EST (0+1 container statuses recorded) Jan 16 15:15:30.324: INFO: Container registry ready: true, restart count 0 W0116 15:15:30.328204 116021 metrics_grabber.go:73] Master node is not registered. Grabbing metrics from Scheduler and ControllerManager is disabled. Jan 16 15:15:30.427: INFO: Latency metrics for node ip-172-31-40-79.us-west-2.compute.internal Jan 16 15:15:30.427: INFO: {Operation: Method:pod_start_latency_microseconds Quantile:0.99 Latency:10.545574s} Jan 16 15:15:30.427: INFO: Logging node info for node ip-172-31-58-91.us-west-2.compute.internal Jan 16 15:15:30.430: INFO: Node Info: &TypeMeta{Kind:,APIVersion:,} Jan 16 15:15:30.430: INFO: Logging kubelet events for node ip-172-31-58-91.us-west-2.compute.internal Jan 16 15:15:30.432: INFO: Logging pods the kubelet thinks is on node ip-172-31-58-91.us-west-2.compute.internal W0116 15:15:30.441733 116021 metrics_grabber.go:73] Master node is not registered. Grabbing metrics from Scheduler and ControllerManager is disabled. Jan 16 15:15:30.566: INFO: Latency metrics for node ip-172-31-58-91.us-west-2.compute.internal [1mSTEP[0m: Dumping a list of prepulled images on each node Jan 16 15:15:30.569: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [1mSTEP[0m: Destroying namespace "e2e-tests-sched-pred-sj8ls" for this suite. Jan 16 15:15:50.620: INFO: namespace: e2e-tests-sched-pred-sj8ls, resource: bindings, ignored listing per whitelist [AfterEach] [k8s.io] SchedulerPredicates [Serial] /builddir/build/BUILD/atomic-openshift-git-0.86a6117/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:67 I0116 15:15:50.642091 116021 request.go:768] Error in request: resource name may not be empty [91m[1m• Failure [23.825 seconds][0m [k8s.io] SchedulerPredicates [Serial] [90m/builddir/build/BUILD/atomic-openshift-git-0.86a6117/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:826[0m [91m[1mvalidates that InterPod Affinity and AntiAffinity is respected if matching [It][0m [90m/builddir/build/BUILD/atomic-openshift-git-0.86a6117/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:640[0m [91mExpected error: <*errors.StatusError | 0xc421bd0280>: { ErrStatus: { TypeMeta: {Kind: "", APIVersion: ""}, ListMeta: {SelfLink: "", ResourceVersion: ""}, Status: "Failure", Message: "pods \"with-podantiaffinity-86ca2603-dc28-11e6-a515-0208cfd56fdf\" is forbidden: affinity.PodAntiAffinity.RequiredDuringScheduling has TopologyKey e2e.inter-pod-affinity.kubernetes.io/zone but only key kubernetes.io/hostname is allowed", Reason: "Forbidden", Details: { Name: "with-podantiaffinity-86ca2603-dc28-11e6-a515-0208cfd56fdf", Group: "", Kind: "pods", Causes: nil, RetryAfterSeconds: 0, }, Code: 403, }, } pods "with-podantiaffinity-86ca2603-dc28-11e6-a515-0208cfd56fdf" is forbidden: affinity.PodAntiAffinity.RequiredDuringScheduling has TopologyKey e2e.inter-pod-affinity.kubernetes.io/zone but only key kubernetes.io/hostname is allowed not to have occurred[0m
I tested latest kube and these 2 tests are passing, I will test on ocp now. [root@fedora24 kubernetes]# KUBERNETES_PROVIDER=local go run hack/e2e.go -v --test --test_args="--ginkgo.focus="InterPod*Affinity"" 2017/01/18 17:33:15 e2e.go:946: Running: ./cluster/kubectl.sh version --match-server-version=false Client Version: version.Info{Major:"1", Minor:"6+", GitVersion:"v1.6.0-alpha.0.3482+dd2cca470fff7e-dirty", GitCommit:"dd2cca470fff7edc9deaa8de40e29cdcdc9f611c", GitTreeState:"dirty", BuildDate:"2017-01-18T22:00:44Z", GoVersion:"go1.7.4", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"6+", GitVersion:"v1.6.0-alpha.0.3482+dd2cca470fff7e-dirty", GitCommit:"dd2cca470fff7edc9deaa8de40e29cdcdc9f611c", GitTreeState:"dirty", BuildDate:"2017-01-18T22:00:44Z", GoVersion:"go1.7.4", Compiler:"gc", Platform:"linux/amd64"} 2017/01/18 17:33:15 e2e.go:948: Step './cluster/kubectl.sh version --match-server-version=false' finished in 70.654786ms 2017/01/18 17:33:15 e2e.go:946: Running: ./hack/e2e-internal/e2e-status.sh Local doesn't need special preparations for e2e tests Client Version: version.Info{Major:"1", Minor:"6+", GitVersion:"v1.6.0-alpha.0.3482+dd2cca470fff7e-dirty", GitCommit:"dd2cca470fff7edc9deaa8de40e29cdcdc9f611c", GitTreeState:"dirty", BuildDate:"2017-01-18T22:00:44Z", GoVersion:"go1.7.4", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"6+", GitVersion:"v1.6.0-alpha.0.3482+dd2cca470fff7e-dirty", GitCommit:"dd2cca470fff7edc9deaa8de40e29cdcdc9f611c", GitTreeState:"dirty", BuildDate:"2017-01-18T22:00:44Z", GoVersion:"go1.7.4", Compiler:"gc", Platform:"linux/amd64"} 2017/01/18 17:33:15 e2e.go:948: Step './hack/e2e-internal/e2e-status.sh' finished in 67.445437ms 2017/01/18 17:33:15 e2e.go:946: Running: ./hack/ginkgo-e2e.sh --ginkgo.focus=InterPod*Affinity Setting up for KUBERNETES_PROVIDER="local". Local doesn't need special preparations for e2e tests Jan 18 17:33:16.113: INFO: Overriding default scale value of zero to 1 Jan 18 17:33:16.113: INFO: Overriding default milliseconds value of zero to 5000 I0118 17:33:16.151390 10555 e2e.go:250] Starting e2e run "1a97a6b8-ddce-11e6-bc46-525400b5933e" on Ginkgo node 1 Running Suite: Kubernetes e2e suite =================================== Random Seed: 1484778795 - Will randomize all specs Will run 2 of 494 specs Jan 18 17:33:16.174: INFO: >>> kubeConfig: /root/.kube/config Jan 18 17:33:16.177: INFO: Waiting up to 4h0m0s for all (but 0) nodes to be schedulable Jan 18 17:33:16.178: INFO: Waiting up to 10m0s for all pods (need at least 0) in namespace 'kube-system' to be running and ready Jan 18 17:33:16.180: INFO: Waiting for pods to enter Success, but no pods in "kube-system" match label map[name:e2e-image-puller] Jan 18 17:33:16.182: INFO: 0 / 0 pods in namespace 'kube-system' are running and ready (0 seconds elapsed) Jan 18 17:33:16.182: INFO: expected 0 pod replicas in namespace 'kube-system', 0 are Running and Ready. Jan 18 17:33:16.183: INFO: Waiting for pods to enter Success, but no pods in "kube-system" match label map[name:e2e-image-puller] Jan 18 17:33:16.183: INFO: Dumping network health container logs from all nodes SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [k8s.io] SchedulerPredicates [Serial] validates that InterPodAffinity is respected if matching /root/upstream-code/gocode/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:498 [BeforeEach] [k8s.io] SchedulerPredicates [Serial] /root/upstream-code/gocode/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:141 STEP: Creating a kubernetes client Jan 18 17:33:16.186: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [k8s.io] SchedulerPredicates [Serial] /root/upstream-code/gocode/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:102 Jan 18 17:33:16.209: INFO: Waiting up to 1m0s for all nodes to be ready Jan 18 17:34:16.215: INFO: Waiting for terminating namespaces to be deleted... Jan 18 17:34:16.217: INFO: Waiting up to 5m0s for all pods (need at least 0) in namespace 'kube-system' to be running and ready Jan 18 17:34:16.219: INFO: Waiting for pods to enter Success, but no pods in "kube-system" match label map[name:e2e-image-puller] Jan 18 17:34:16.220: INFO: 0 / 0 pods in namespace 'kube-system' are running and ready (0 seconds elapsed) Jan 18 17:34:16.220: INFO: expected 0 pod replicas in namespace 'kube-system', 0 are Running and Ready. Jan 18 17:34:16.220: INFO: Logging pods the kubelet thinks is on node 127.0.0.1 before test [It] validates that InterPodAffinity is respected if matching /root/upstream-code/gocode/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:498 STEP: Trying to launch a pod with a label to get a node which can launch it. STEP: Trying to apply a random label on the found node. STEP: verifying the node has the label e2e.inter-pod-affinity.kubernetes.io/zone china-e2etest STEP: Trying to launch the pod, now with podAffinity. STEP: removing the label e2e.inter-pod-affinity.kubernetes.io/zone off the node 127.0.0.1 STEP: verifying the node doesn't have the label e2e.inter-pod-affinity.kubernetes.io/zone [AfterEach] [k8s.io] SchedulerPredicates [Serial] /root/upstream-code/gocode/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:142 Jan 18 17:34:28.498: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "e2e-tests-sched-pred-gmz8c" for this suite. Jan 18 17:34:53.524: INFO: namespace: e2e-tests-sched-pred-gmz8c, resource: bindings, ignored listing per whitelist [AfterEach] [k8s.io] SchedulerPredicates [Serial] /root/upstream-code/gocode/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:69 • [SLOW TEST:97.351 seconds] [k8s.io] SchedulerPredicates [Serial] /root/upstream-code/gocode/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:827 validates that InterPodAffinity is respected if matching /root/upstream-code/gocode/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:498 ------------------------------ SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS ------------------------------ [k8s.io] SchedulerPredicates [Serial] validates that InterPodAffinity is respected if matching with multiple Affinities /root/upstream-code/gocode/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:615 [BeforeEach] [k8s.io] SchedulerPredicates [Serial] /root/upstream-code/gocode/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:141 STEP: Creating a kubernetes client Jan 18 17:34:53.538: INFO: >>> kubeConfig: /root/.kube/config STEP: Building a namespace api object STEP: Waiting for a default service account to be provisioned in namespace [BeforeEach] [k8s.io] SchedulerPredicates [Serial] /root/upstream-code/gocode/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:102 Jan 18 17:34:53.571: INFO: Waiting up to 1m0s for all nodes to be ready Jan 18 17:35:53.576: INFO: Waiting for terminating namespaces to be deleted... Jan 18 17:35:53.577: INFO: Waiting up to 5m0s for all pods (need at least 0) in namespace 'kube-system' to be running and ready Jan 18 17:35:53.579: INFO: Waiting for pods to enter Success, but no pods in "kube-system" match label map[name:e2e-image-puller] Jan 18 17:35:53.580: INFO: 0 / 0 pods in namespace 'kube-system' are running and ready (0 seconds elapsed) Jan 18 17:35:53.580: INFO: expected 0 pod replicas in namespace 'kube-system', 0 are Running and Ready. Jan 18 17:35:53.580: INFO: Logging pods the kubelet thinks is on node 127.0.0.1 before test Jan 18 17:35:53.583: INFO: with-label-3e75d219-ddce-11e6-bc46-525400b5933e from e2e-tests-sched-pred-gmz8c started at 2017-01-18 17:34:16 -0500 EST (1 container statuses recorded) Jan 18 17:35:53.583: INFO: Container pfpod ready: false, restart count 0 Jan 18 17:35:53.583: INFO: with-podaffinity-435463d6-ddce-11e6-bc46-525400b5933e from e2e-tests-sched-pred-gmz8c started at 2017-01-18 17:34:24 -0500 EST (1 container statuses recorded) Jan 18 17:35:53.583: INFO: Container pfpod ready: false, restart count 0 [It] validates that InterPodAffinity is respected if matching with multiple Affinities /root/upstream-code/gocode/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:615 STEP: Trying to launch a pod with a label to get a node which can launch it. STEP: Trying to apply a random label on the found node. STEP: verifying the node has the label e2e.inter-pod-affinity.kubernetes.io/zone kubernetes-e2e STEP: Trying to launch the pod, now with multiple pod affinities with diff LabelOperators. STEP: removing the label e2e.inter-pod-affinity.kubernetes.io/zone off the node 127.0.0.1 STEP: verifying the node doesn't have the label e2e.inter-pod-affinity.kubernetes.io/zone [AfterEach] [k8s.io] SchedulerPredicates [Serial] /root/upstream-code/gocode/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:142 Jan 18 17:35:59.858: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready STEP: Destroying namespace "e2e-tests-sched-pred-v62rs" for this suite. Jan 18 17:36:24.874: INFO: namespace: e2e-tests-sched-pred-v62rs, resource: bindings, ignored listing per whitelist [AfterEach] [k8s.io] SchedulerPredicates [Serial] /root/upstream-code/gocode/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:69 • [SLOW TEST:91.349 seconds] [k8s.io] SchedulerPredicates [Serial] /root/upstream-code/gocode/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:827 validates that InterPodAffinity is respected if matching with multiple Affinities /root/upstream-code/gocode/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:615 ------------------------------ SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS Ran 2 of 494 Specs in 188.717 seconds SUCCESS! -- 2 Passed | 0 Failed | 0 Pending | 492 Skipped PASS Ginkgo ran 1 suite in 3m8.921199721s Test Suite Passed 2017/01/18 17:36:24 e2e.go:948: Step './hack/ginkgo-e2e.sh --ginkgo.focus=InterPod*Affinity' finished in 3m8.957169789s
I have sent a PR to fix following e2e: validates that InterPodAntiAffinity is respected if matching 2. This e2e test was failing due to a different reason than the one mentioned here in upstream kube. https://github.com/kubernetes/kubernetes/pull/40534
Mike, Could you check if the admission plugin LimitPodHardAntiAffinityTopology is enabled when running these tests? These tests seem to be designed with zones and pod anti affinity, and in direct conflict of the LimitPodHardAntiAffinityTopology plugin. I could change the tests, but then it wouldn't test what is needed to test zones and pod anti affinity together.
I get this at startup. LimitPodHardAntiAffinityTopology plugin is not in the list of plugins NOT enabled. I don't see a corresponding list of those enabled. messages:Jan 27 10:25:23 ip-172-31-12-197 atomic-openshift-master: I0127 10:25:23.299335 2516 admission.go:105] Admission plugin ProjectRequestLimit is not enabled. It will not be started. messages:Jan 27 10:25:23 ip-172-31-12-197 atomic-openshift-master: I0127 10:25:23.299366 2516 admission.go:105] Admission plugin openshift.io/RestrictSubjectBindings is not enabled. It will not be started. messages:Jan 27 10:25:23 ip-172-31-12-197 atomic-openshift-master: I0127 10:25:23.299376 2516 admission.go:105] Admission plugin PodNodeConstraints is not enabled. It will not be started. messages:Jan 27 10:25:23 ip-172-31-12-197 atomic-openshift-master: I0127 10:25:23.299417 2516 admission.go:105] Admission plugin RunOnceDuration is not enabled. It will not be started. messages:Jan 27 10:25:23 ip-172-31-12-197 atomic-openshift-master: I0127 10:25:23.299427 2516 admission.go:105] Admission plugin PodNodeConstraints is not enabled. It will not be started. messages:Jan 27 10:25:23 ip-172-31-12-197 atomic-openshift-master: I0127 10:25:23.299439 2516 admission.go:105] Admission plugin ClusterResourceOverride is not enabled. It will not be started. messages:Jan 27 10:25:23 ip-172-31-12-197 atomic-openshift-master: I0127 10:25:23.299474 2516 admission.go:105] Admission plugin openshift.io/ImagePolicy is not enabled. It will not be started. messages:Jan 27 10:25:23 ip-172-31-12-197 atomic-openshift-master: I0127 10:25:23.299483 2516 admission.go:105] Admission plugin ImagePolicyWebhook is not enabled. It will not be started. messages:Jan 27 10:25:23 ip-172-31-12-197 atomic-openshift-master: I0127 10:25:23.299567 2516 admission.go:105] Admission plugin AlwaysPullImages is not enabled. It will not be started.
I think it would require disabling the plugin which is enabled by default to run these e2e tests: https://github.com/openshift/origin/blob/master/pkg/cmd/server/start/admission.go#L68 I have created origin issue: https://github.com/openshift/origin/issues/12712
PR to fix this issue: https://github.com/openshift/origin/pull/12782
This has been merged into ocp and is in OCP v3.5.0.17 or newer.
Verified on 3.5.0.17
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2017:0884