Bug 1384639

Summary:	panic and stack trace in master API logs
Product:	OpenShift Container Platform	Reporter:	Luke Meyer <lmeyer>
Component:	Image Registry	Assignee:	Maciej Szulik <maszulik>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Wei Sun <wsun>
Severity:	medium	Docs Contact:
Priority:	unspecified
Version:	3.2.0	CC:	agladkov, aos-bugs, decarr, jokerman, mmccomas
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2016-10-31 19:43:26 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:

Description Luke Meyer 2016-10-13 18:16:17 UTC

I found this while looking through customer logs, thought it would be good to find out if it's a known problem, already solved in later versions, or not yet known. This was in the log from atomic-openshift-master-api

E0808 10:04:48.280408   80260 runtime.go:58] Recovered from panic: "invalid memory address or nil pointer dereference" (runtime error: invalid memory address or nil pointer dereference)
/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_thirdpartyhacks/src/k8s.io/kubernetes/pkg/util/runtime/runtime.go:52
/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_thirdpartyhacks/src/k8s.io/kubernetes/pkg/util/runtime/runtime.go:40
/usr/lib/golang/src/runtime/asm_amd64.s:402
/usr/lib/golang/src/runtime/panic.go:387
/usr/lib/golang/src/runtime/panic.go:42
/usr/lib/golang/src/runtime/sigpanic_unix.go:26
/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_build/src/github.com/openshift/origin/pkg/image/registry/imagestreamimport/rest.go:181
/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_thirdpartyhacks/src/k8s.io/kubernetes/pkg/apiserver/resthandler.go:437
/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_thirdpartyhacks/src/k8s.io/kubernetes/pkg/apiserver/resthandler.go:400
/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_thirdpartyhacks/src/k8s.io/kubernetes/pkg/apiserver/resthandler.go:916
/usr/lib/golang/src/runtime/asm_amd64.s:2232
E0808 10:04:48.280677   80260 apiserver.go:207] recover from panic situation: - runtime error: invalid memory address or nil pointer dereference
/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_thirdpartyhacks/src/github.com/emicklei/go-restful/container.go:183
/usr/lib/golang/src/runtime/asm_amd64.s:401
/usr/lib/golang/src/runtime/panic.go:387
/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_thirdpartyhacks/src/k8s.io/kubernetes/pkg/apiserver/resthandler.go:932
/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_thirdpartyhacks/src/k8s.io/kubernetes/pkg/apiserver/resthandler.go:405
/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_thirdpartyhacks/src/k8s.io/kubernetes/pkg/apiserver/metrics/metrics.go:101
/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_thirdpartyhacks/src/github.com/emicklei/go-restful/container.go:249
/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_thirdpartyhacks/src/github.com/emicklei/go-restful/container.go:95
/usr/lib/golang/src/net/http/server.go:1265
/usr/lib/golang/src/net/http/server.go:1541
/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_thirdpartyhacks/src/github.com/emicklei/go-restful/container.go:264
<autogenerated>:32
/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_build/src/github.com/openshift/origin/pkg/cmd/server/origin/handlers.go:95
/usr/lib/golang/src/net/http/server.go:1265
/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_build/src/github.com/openshift/origin/pkg/cmd/server/origin/auth.go:764
/usr/lib/golang/src/net/http/server.go:1265
/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_build/src/github.com/openshift/origin/pkg/cmd/server/origin/handlers.go:177
/usr/lib/golang/src/net/http/server.go:1265
/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_build/src/github.com/openshift/origin/pkg/cmd/server/origin/handlers.go:143
/usr/lib/golang/src/net/http/server.go:1265
/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_build/src/github.com/openshift/origin/pkg/cmd/server/origin/handlers.go:62
/usr/lib/golang/src/net/http/server.go:1265
/usr/lib/golang/src/net/http/server.go:1541
/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_thirdpartyhacks/src/github.com/emicklei/go-restful/container.go:264
<autogenerated>:32
/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_thirdpartyhacks/src/k8s.io/kubernetes/pkg/apiserver/handlers.go:364
/usr/lib/golang/src/net/http/server.go:1265
/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_build/src/github.com/openshift/origin/pkg/cmd/server/origin/handlers.go:291
/usr/lib/golang/src/net/http/server.go:1265
/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_thirdpartyhacks/src/k8s.io/kubernetes/pkg/api/requestcontext.go:101
/usr/lib/golang/src/net/http/server.go:1265
/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_thirdpartyhacks/src/k8s.io/kubernetes/pkg/apiserver/handlers.go:118
/usr/lib/golang/src/net/http/server.go:1265
/usr/lib/golang/src/net/http/server.go:1703
/usr/lib/golang/src/net/http/server.go:1204
/usr/lib/golang/src/runtime/asm_amd64.s:2232

I don't have any further context for what caused this, but it repeats again in exactly 3 minutes and then several times after.

Comment 1 Ben Parees 2016-10-19 22:27:47 UTC

this was assigned to the image component...what image is it in reference to?

Comment 2 Ben Parees 2016-10-20 14:25:24 UTC

moving to registry component since this appears to be a panic in the registry image import code.

Comment 3 Luke Meyer 2016-10-20 16:34:45 UTC

Sorry about that, must have picked the component next to the one I wanted.

Comment 4 Maciej Szulik 2016-10-31 16:34:34 UTC

This check [1] was added starting from 1.3/3.3. That should address the nil panic in :

/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_build/src/github.com/openshift/origin/pkg/image/registry/imagestreamimport/rest.go:181

I don't see any other problem here.

[1] https://github.com/openshift/origin/blob/11d3928180e4063720053ad2a8aaf93f6e17b1b6/pkg/image/registry/imagestreamimport/rest.go#L181-L183

Comment 5 Luke Meyer 2016-10-31 16:49:11 UTC

The exact master version here was 3.2.1.4

Comment 6 Luke Meyer 2016-10-31 19:43:26 UTC

I'll close this then, as it's fixed in newer version and doesn't seem to be causing much headache in the 3.2 version so I don't see a backport being required.