Bug 1695326 - Unit test flake post rebase, TestLegacyScaleUpUnreadyLessScale
Summary: Unit test flake post rebase, TestLegacyScaleUpUnreadyLessScale
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Node
Version: 4.1.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: 4.1.0
Assignee: Joel Smith
QA Contact: Jianwei Hou
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-04-02 20:13 UTC by Clayton Coleman
Modified: 2019-06-04 10:47 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-06-04 10:47:00 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2019:0758 None None None 2019-06-04 10:47:10 UTC

Description Clayton Coleman 2019-04-02 20:13:24 UTC
https://openshift-gce-devel.appspot.com/build/origin-ci-test/pr-logs/pull/22455/pull-ci-openshift-origin-master-unit/4554#githubcomopenshiftoriginvendork8siokubernetespkgcontrollerpodautoscaler-testlegacyscaleupunreadylessscale

== RUN   TestLegacyScaleUpUnreadyLessScale
E0402 19:32:53.847496    2111 runtime.go:69] Observed a panic: &runtime.TypeAssertionError{interfaceString:"", concreteString:"testing.PatchActionImpl", assertedString:"testing.CreateAction", missingMethod:"GetObject"} (interface conversion: testing.PatchActionImpl is not testing.CreateAction: missing method GetObject)
/go/src/github.com/openshift/origin/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:76
/go/src/github.com/openshift/origin/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:65
/go/src/github.com/openshift/origin/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:51
/usr/local/go/src/runtime/asm_amd64.s:573
/usr/local/go/src/runtime/panic.go:502
/usr/local/go/src/runtime/iface.go:85
/usr/local/go/src/runtime/iface.go:562
/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/controller/podautoscaler/legacy_horizontal_test.go:475
/go/src/github.com/openshift/origin/vendor/k8s.io/client-go/testing/fixture.go:504
/go/src/github.com/openshift/origin/vendor/k8s.io/client-go/testing/fake.go:140
/go/src/github.com/openshift/origin/vendor/k8s.io/client-go/kubernetes/typed/core/v1/fake/fake_event_expansion.go:64
/go/src/github.com/openshift/origin/vendor/k8s.io/client-go/kubernetes/typed/core/v1/event_expansion.go:163
/go/src/github.com/openshift/origin/vendor/k8s.io/client-go/tools/record/event.go:178
/go/src/github.com/openshift/origin/vendor/k8s.io/client-go/tools/record/event.go:142
/go/src/github.com/openshift/origin/vendor/k8s.io/client-go/tools/record/event.go:124
/go/src/github.com/openshift/origin/vendor/k8s.io/client-go/tools/record/event.go:238
/usr/local/go/src/runtime/asm_amd64.s:2361
--- PASS: TestLegacyScaleUpUnreadyLessScale (0.13s)
panic: interface conversion: testing.PatchActionImpl is not testing.CreateAction: missing method GetObject [recovered]
	panic: interface conversion: testing.PatchActionImpl is not testing.CreateAction: missing method GetObject

goroutine 1859 [running]:
github.com/openshift/origin/vendor/k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0)
	/go/src/github.com/openshift/origin/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:58 +0x16b
panic(0x1a2a1c0, 0xc4200ae840)
	/usr/local/go/src/runtime/panic.go:502 +0x24a
github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/controller/podautoscaler.(*legacyTestCase).runTest.func1(0x1d51b80, 0xc4203cac80, 0x0, 0x0, 0x0, 0x0, 0x0)
	/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/controller/podautoscaler/legacy_horizontal_test.go:475 +0x1da
github.com/openshift/origin/vendor/k8s.io/client-go/testing.(*SimpleReactor).React(0xc420e26d80, 0x1d51b80, 0xc4203cac80, 0x1, 0x2, 0xc4204f6240, 0x1, 0x2)
	/go/src/github.com/openshift/origin/vendor/k8s.io/client-go/testing/fixture.go:504 +0x65
github.com/openshift/origin/vendor/k8s.io/client-go/testing.(*Fake).Invokes(0xc4205526e0, 0x1d51b80, 0xc4203ca500, 0x1d31240, 0xc4204e8280, 0x0, 0x0, 0x0, 0x0)
	/go/src/github.com/openshift/origin/vendor/k8s.io/client-go/testing/fake.go:140 +0x276
github.com/openshift/origin/vendor/k8s.io/client-go/kubernetes/typed/core/v1/fake.(*FakeEvents).PatchWithEventNamespace(0xc42085aae0, 0xc4204e8280, 0xc420c3e000, 0x91, 0xd2, 0x0, 0xc4205fe7b0, 0x30)
	/go/src/github.com/openshift/origin/vendor/k8s.io/client-go/kubern

Failing almost every time

Comment 1 Frederic Branczyk 2019-04-08 14:21:02 UTC
Joel from the Pod team is taking care of pod autoscaler things. Re-assigning.

Comment 2 Seth Jennings 2019-04-10 14:50:10 UTC
This has stopped happening.

Most recent occurrence was 5d ago.  Seems there was a 2d window 5d ago where this was happening.

https://search.svc.ci.openshift.org/?search=TestLegacyScaleUpUnreadyLessScale&maxAge=336h&context=2&type=all

Comment 3 Joel Smith 2019-04-17 14:54:14 UTC
I'm not sure if this flake is fixed by the deadlock in the same file, but it's certainly possible. Since the flake isn't happening now, I'm going to move forward under the assumption that the fix will address this flake. If it doesn't, we can always re-open this bug.

Here's the PR: https://github.com/openshift/origin/pull/22591

And here's how to trigger the deadlock flake. From the Origin source directory, run:

GOMAXPROCS=1 go test ./vendor/k8s.io/kubernetes/pkg/controller/podautoscaler/... -timeout 60s -count 1

Comment 5 Sunil Choudhary 2019-04-30 14:45:24 UTC
$ GOMAXPROCS=1 go test ./vendor/k8s.io/kubernetes/pkg/controller/podautoscaler/.. -timeout 60s -count 1
ok  	origin/vendor/k8s.io/kubernetes/pkg/controller	0.028s

And no longer see the panic in https://search.svc.ci.openshift.org/?search=TestLegacyScaleUpUnreadyLessScale&maxAge=336h&context=2&type=all

Comment 7 errata-xmlrpc 2019-06-04 10:47:00 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:0758


Note You need to log in before you can comment on or make changes to this bug.