Bug 1384639 - panic and stack trace in master API logs
Summary: panic and stack trace in master API logs
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Image Registry
Version: 3.2.0
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
: ---
Assignee: Maciej Szulik
QA Contact: Wei Sun
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-10-13 18:16 UTC by Luke Meyer
Modified: 2016-10-31 19:43 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-10-31 19:43:26 UTC
Target Upstream Version:


Attachments (Terms of Use)

Description Luke Meyer 2016-10-13 18:16:17 UTC
I found this while looking through customer logs, thought it would be good to find out if it's a known problem, already solved in later versions, or not yet known. This was in the log from atomic-openshift-master-api

E0808 10:04:48.280408   80260 runtime.go:58] Recovered from panic: "invalid memory address or nil pointer dereference" (runtime error: invalid memory address or nil pointer dereference)
/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_thirdpartyhacks/src/k8s.io/kubernetes/pkg/util/runtime/runtime.go:52
/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_thirdpartyhacks/src/k8s.io/kubernetes/pkg/util/runtime/runtime.go:40
/usr/lib/golang/src/runtime/asm_amd64.s:402
/usr/lib/golang/src/runtime/panic.go:387
/usr/lib/golang/src/runtime/panic.go:42
/usr/lib/golang/src/runtime/sigpanic_unix.go:26
/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_build/src/github.com/openshift/origin/pkg/image/registry/imagestreamimport/rest.go:181
/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_thirdpartyhacks/src/k8s.io/kubernetes/pkg/apiserver/resthandler.go:437
/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_thirdpartyhacks/src/k8s.io/kubernetes/pkg/apiserver/resthandler.go:400
/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_thirdpartyhacks/src/k8s.io/kubernetes/pkg/apiserver/resthandler.go:916
/usr/lib/golang/src/runtime/asm_amd64.s:2232
E0808 10:04:48.280677   80260 apiserver.go:207] recover from panic situation: - runtime error: invalid memory address or nil pointer dereference
/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_thirdpartyhacks/src/github.com/emicklei/go-restful/container.go:183
/usr/lib/golang/src/runtime/asm_amd64.s:401
/usr/lib/golang/src/runtime/panic.go:387
/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_thirdpartyhacks/src/k8s.io/kubernetes/pkg/apiserver/resthandler.go:932
/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_thirdpartyhacks/src/k8s.io/kubernetes/pkg/apiserver/resthandler.go:405
/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_thirdpartyhacks/src/k8s.io/kubernetes/pkg/apiserver/metrics/metrics.go:101
/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_thirdpartyhacks/src/github.com/emicklei/go-restful/container.go:249
/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_thirdpartyhacks/src/github.com/emicklei/go-restful/container.go:95
/usr/lib/golang/src/net/http/server.go:1265
/usr/lib/golang/src/net/http/server.go:1541
/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_thirdpartyhacks/src/github.com/emicklei/go-restful/container.go:264
<autogenerated>:32
/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_build/src/github.com/openshift/origin/pkg/cmd/server/origin/handlers.go:95
/usr/lib/golang/src/net/http/server.go:1265
/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_build/src/github.com/openshift/origin/pkg/cmd/server/origin/auth.go:764
/usr/lib/golang/src/net/http/server.go:1265
/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_build/src/github.com/openshift/origin/pkg/cmd/server/origin/handlers.go:177
/usr/lib/golang/src/net/http/server.go:1265
/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_build/src/github.com/openshift/origin/pkg/cmd/server/origin/handlers.go:143
/usr/lib/golang/src/net/http/server.go:1265
/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_build/src/github.com/openshift/origin/pkg/cmd/server/origin/handlers.go:62
/usr/lib/golang/src/net/http/server.go:1265
/usr/lib/golang/src/net/http/server.go:1541
/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_thirdpartyhacks/src/github.com/emicklei/go-restful/container.go:264
<autogenerated>:32
/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_thirdpartyhacks/src/k8s.io/kubernetes/pkg/apiserver/handlers.go:364
/usr/lib/golang/src/net/http/server.go:1265
/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_build/src/github.com/openshift/origin/pkg/cmd/server/origin/handlers.go:291
/usr/lib/golang/src/net/http/server.go:1265
/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_thirdpartyhacks/src/k8s.io/kubernetes/pkg/api/requestcontext.go:101
/usr/lib/golang/src/net/http/server.go:1265
/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_thirdpartyhacks/src/k8s.io/kubernetes/pkg/apiserver/handlers.go:118
/usr/lib/golang/src/net/http/server.go:1265
/usr/lib/golang/src/net/http/server.go:1703
/usr/lib/golang/src/net/http/server.go:1204
/usr/lib/golang/src/runtime/asm_amd64.s:2232

I don't have any further context for what caused this, but it repeats again in exactly 3 minutes and then several times after.

Comment 1 Ben Parees 2016-10-19 22:27:47 UTC
this was assigned to the image component...what image is it in reference to?

Comment 2 Ben Parees 2016-10-20 14:25:24 UTC
moving to registry component since this appears to be a panic in the registry image import code.

Comment 3 Luke Meyer 2016-10-20 16:34:45 UTC
Sorry about that, must have picked the component next to the one I wanted.

Comment 4 Maciej Szulik 2016-10-31 16:34:34 UTC
This check [1] was added starting from 1.3/3.3. That should address the nil panic in :

/builddir/build/BUILD/atomic-openshift-git-0.9fe156c/_build/src/github.com/openshift/origin/pkg/image/registry/imagestreamimport/rest.go:181

I don't see any other problem here.

[1] https://github.com/openshift/origin/blob/11d3928180e4063720053ad2a8aaf93f6e17b1b6/pkg/image/registry/imagestreamimport/rest.go#L181-L183

Comment 5 Luke Meyer 2016-10-31 16:49:11 UTC
The exact master version here was 3.2.1.4

Comment 6 Luke Meyer 2016-10-31 19:43:26 UTC
I'll close this then, as it's fixed in newer version and doesn't seem to be causing much headache in the 3.2 version so I don't see a backport being required.


Note You need to log in before you can comment on or make changes to this bug.