github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/controller/podautoscaler TestEventNotCreated 3m0s https://openshift-gce-devel.appspot.com/build/origin-ci-test/pr-logs/pull/22447/pull-ci-openshift-origin-master-unit/4610#githubcomopenshiftoriginvendork8siokubernetespkgcontrollerpodautoscaler-testeventnotcreated goroutine 1200 [chan receive, 2 minutes]: github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/controller/podautoscaler.(*HorizontalController).Run(0xc420138f00, 0xc420048d20) /go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/controller/podautoscaler/horizontal.go:166 +0x29f created by github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/controller/podautoscaler.(*testCase).runTestWithController /go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/controller/podautoscaler/horizontal_test.go:701 +0xbf Set to high because it is flaking not infrequently.
Also https://openshift-gce-devel.appspot.com/build/origin-ci-test/pr-logs/pull/22488/pull-ci-openshift-origin-master-unit/4694#githubcomopenshiftoriginvendork8siokubernetespkgcontrollerpodautoscaler-testlegacysuperfluousmetrics
Joel from the Pod team is taking care of pod autoscaler things. Re-assigning.
Removing as blocker but still need to stay on this. 15 occurrences in the last 48h. https://search.svc.ci.openshift.org/?search=TestEventNotCreated&maxAge=48h&context=2&type=all
This is absolutely still a blocker. 50% of origin merges / PR jobs were failing on this. I am disabling the test here: https://github.com/openshift/origin/pull/22527 Please ensure you follow up and fix.
I think I have figured out the cause of the deadlock in the test. I've got a WIP PR that should address the deadlock and fix another potential flake. I'm going to test it over and over to see if I can get it to hit the flake again with my change. If it looks good, I'll work on getting it merged upstream and in Origin. https://github.com/openshift/origin/pull/22531
No flakes on my tests that ran overnight. Hopefully that's a good sign.
Hi Sunil, Here's a reproducer for the flake. From the origin source directory run this: GOMAXPROCS=1 go test ./vendor/k8s.io/kubernetes/pkg/controller/podautoscaler/... -timeout 60s -count 1 Before the fix, it should timeout after 60 seconds with a deadlock backtrace. After the fix, it should give an "ok" result. Unfortunately, there is another flake that we're fixing in https://github.com/openshift/origin/pull/22591 that needs to merge too, or the reproducer will fail on that one too. Once both fixes are in, then you should get an "ok" result.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:0758