Description of problem: https://prow.svc.ci.openshift.org/view/gcs/origin-ci-test/logs/canary-openshift-ocp-installer-e2e-azure-serial-4.2/86: [sig-scheduling] SchedulerPreemption [Serial] validates pod anti-affinity works in preemption [Suite:openshift/conformance/serial] [Suite:k8s] We can see in log(blob:null/a83e4bb6-e21d-4fa3-8e61-c56dfd67d938): [AfterEach] [sig-scheduling] SchedulerPreemption [Serial] /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/scheduling/preemption.go:64 Sep 2 18:27:22.499: INFO: Running AfterSuite actions on all nodes Sep 2 18:27:22.499: INFO: Running AfterSuite actions on node 1 /opt/rh/go-toolset-1.12/root/usr/lib/go-toolset-1.12-golang/src/runtime/panic.go:522 +0x1b5 github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/scheduling.glob..func4.5() /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/scheduling/preemption.go:415 +0x1792 github.com/openshift/origin/vendor/github.com/onsi/ginkgo/internal/leafnodes.(*runner).runSync(0xc00095ede0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...) /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/cmd/openshift-tests/openshift-tests.go:233 +0x113 github.com/openshift/origin/vendor/github.com/spf13/cobra.(*Command).execute(0xc003317680, 0xc0033b1600, 0x1, 0x1, 0xc003317680, 0xc0033b1600) /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/github.com/spf13/cobra/command.go:756 +0x465 github.com/openshift/origin/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xc003316f00, 0x62dd218, 0xa5716d0, 0xa5716d0) /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/github.com/spf13/cobra/command.go:846 +0x2ec github.com/openshift/origin/vendor/github.com/spf13/cobra.(*Command).Execute(...) /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/github.com/spf13/cobra/command.go:794 main.main.func1(0xc003316f00, 0x0, 0x0) /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/cmd/openshift-tests/openshift-tests.go:71 +0x93 main.main() /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/cmd/openshift-tests/openshift-tests.go:72 +0x327 fail [runtime/panic.go:82]: Test Panicked: runtime error: invalid memory address or nil pointer dereference Version-Release number of selected component (if applicable): 4.2.0-0.nightly-2019-09-02-172410 How reproducible: sometimes Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info:
Mike, it looks like your fix from https://github.com/openshift/origin/pull/23645 is causing that pods are not created, see logs: Sep 2 18:26:45.291: INFO: Current cpu usage and memory usage is 1410, 3176136704 STEP: verifying the node has the label node ci-op-s3mbwj7y-3a8ca-hshst-worker-centralus1-fqkwq Sep 2 18:26:45.445: INFO: Created pod: pod0-sched-preemption-medium-priority Sep 2 18:26:45.445: INFO: Current cpu usage and memory usage is 1560, 3582984192 Sep 2 18:26:45.445: INFO: Node is heavily utilized, let's not create a pod there Sep 2 18:26:45.445: INFO: Current cpu usage and memory usage is 1710, 3759144960 Sep 2 18:26:45.445: INFO: Node is heavily utilized, let's not create a pod there but further down the code you're iterating over the pods array which is always created with exactly 4 elements and that might fail with the panic above: /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/scheduling/preemption.go:415 +0x1792 github.com/openshift/origin/vendor/github.com/onsi/ginkgo/internal/leafnodes.(*runner).runSync(0xc00095ede0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...) /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/cmd/openshift-tests/openshift-tests.go:233 +0x113 github.com/openshift/origin/vendor/github.com/spf13/cobra.(*Command).execute(0xc003317680, 0xc0033b1600, 0x1, 0x1, 0xc003317680, 0xc0033b1600) /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/github.com/spf13/cobra/command.go:756 +0x465 github.com/openshift/origin/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xc003316f00, 0x62dd218, 0xa5716d0, 0xa5716d0) /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/github.com/spf13/cobra/command.go:846 +0x2ec github.com/openshift/origin/vendor/github.com/spf13/cobra.(*Command).Execute(...) /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/github.com/spf13/cobra/command.go:794 main.main.func1(0xc003316f00, 0x0, 0x0) /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/cmd/openshift-tests/openshift-tests.go:71 +0x93 main.main() /go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/cmd/openshift-tests/openshift-tests.go:72 +0x327 Fix this ASAP.
Checked https://testgrid.k8s.io/redhat-openshift-release-informing#redhat-canary-openshift-ocp-installer-e2e-azure-serial-4.2&sort-by-flakiness&show-stale-tests= , the testing "validates pod anti-affinity works in preemption" is shown as not executed since 7 Sept, and shown as failed since 3 Sept.
That test is no longer executed because it was removed in https://github.com/openshift/origin/pull/23728 which also addresses https://bugzilla.redhat.com/show_bug.cgi?id=1743102. Due to that at this point I think this BZ is effectively a duplicate of that one so I am closing it.
*** This bug has been marked as a duplicate of bug 1743102 ***