Bug 1854115
| Summary: | Installer panics when creating cluster using macOS 10.14.6 | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Dan Mace <dmace> |
| Component: | Installer | Assignee: | Abhinav Dahiya <adahiya> |
| Installer sub component: | openshift-installer | QA Contact: | To Hung Sze <tsze> |
| Status: | CLOSED ERRATA | Docs Contact: | |
| Severity: | urgent | ||
| Priority: | high | CC: | adahiya, bleanhar, tsze, yanyang |
| Version: | 4.6 | ||
| Target Milestone: | --- | ||
| Target Release: | 4.6.0 | ||
| Hardware: | Unspecified | ||
| OS: | Mac OS | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2020-10-27 16:12:20 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Last published working build: 4.6.0-0.nightly-2020-06-25-221502 First published broken build: 4.6.0-0.nightly-2020-06-30-020342 Also reproduced on Mac OS X 10.13.3 with registry.svc.ci.openshift.org/ocp/release:4.6.0-0.nightly-2020-07-07-233934 ./openshift-install version runtime: netpoll: break fd ready for -2 fatal error: runtime: netpoll: break fd ready for something unexpected runtime stack: runtime.throw(0xd914804, 0x39) /opt/rh/go-toolset-1.14/root/usr/lib/go-toolset-1.14-golang/src/runtime/panic.go:1116 +0x72 runtime.netpoll(0x129111c59, 0xc000091801) /opt/rh/go-toolset-1.14/root/usr/lib/go-toolset-1.14-golang/src/runtime/netpoll_kqueue.go:140 +0x30a runtime.findrunnable(0xc00008f000, 0x0) /opt/rh/go-toolset-1.14/root/usr/lib/go-toolset-1.14-golang/src/runtime/proc.go:2329 +0x72b runtime.schedule() /opt/rh/go-toolset-1.14/root/usr/lib/go-toolset-1.14-golang/src/runtime/proc.go:2526 +0x2fc runtime.park_m(0xc000500180) /opt/rh/go-toolset-1.14/root/usr/lib/go-toolset-1.14-golang/src/runtime/proc.go:2696 +0x9d runtime.mcall(0x80000) /opt/rh/go-toolset-1.14/root/usr/lib/go-toolset-1.14-golang/src/runtime/asm_amd64.s:318 +0x5b goroutine 1 [runnable, locked to thread]: reflect.Value.Pointer(0xce625e0, 0x0, 0x16, 0x2b) /opt/rh/go-toolset-1.14/root/usr/lib/go-toolset-1.14-golang/src/reflect/value.go:1429 +0x1af github.com/golang/protobuf/proto.RegisterType(0xee37000, 0x0, 0xd893e0a, 0x2b) /go/src/github.com/openshift/installer/vendor/github.com/golang/protobuf/proto/properties.go:486 +0x324 github.com/hashicorp/terraform-plugin-sdk/tfplugin5.init.0() /go/src/github.com/openshift/installer/vendor/github.com/hashicorp/terraform-plugin-sdk/tfplugin5/tfplugin5.pb.go:2779 +0x744 goroutine 18 [chan receive]: k8s.io/klog.(*loggingT).flushDaemon(0x16565fa0) /go/src/github.com/openshift/installer/vendor/k8s.io/klog/klog.go:1010 +0x8b created by k8s.io/klog.init.0 /go/src/github.com/openshift/installer/vendor/k8s.io/klog/klog.go:411 +0xd6 goroutine 5 [select]: go.opencensus.io/stats/view.(*worker).start(0xc0002a0190) /go/src/github.com/openshift/installer/vendor/go.opencensus.io/stats/view/worker.go:154 +0x100 created by go.opencensus.io/stats/view.init.0 /go/src/github.com/openshift/installer/vendor/go.opencensus.io/stats/view/worker.go:32 +0x57 Let's try to add the patch https://github.com/hashicorp/terraform/pull/24562/files to installer's terraform fork and see if this fixes this. I built the installer with https://github.com/hashicorp/terraform/commit/cd65b28da051174a13ac76e54b7bb95d3051255c applied and verified the crash is fixed on macOS 10.14.6. I tried openshift-install-mac-4.6.0-0.nightly-2020-07-16-005008 and still see the issue. Which build will have the fix? @adahiya Used openshift-install-mac-4.6.0-0.nightly-2020-07-16-162619 and am able to successfully install and destroy a cluster on Mac 10.13.3 Also verified working with openshift-client-mac-4.6.0-0.nightly-2020-07-17-043219. Thanks! Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (OpenShift Container Platform 4.6 GA Images), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:4196 |
The latest nightly 4.6 build I pulled today[1] is causing a panic when running `create cluster` on macOS 10.14.6: runtime: netpoll: break fd ready for -2 fatal error: runtime: netpoll: break fd ready for something unexpected runtime stack: runtime.throw(0xd701e49, 0x39) /opt/rh/go-toolset-1.14/root/usr/lib/go-toolset-1.14-golang/src/runtime/panic.go:1116 +0x72 runtime.netpoll(0x128e073de, 0x123648477b901) /opt/rh/go-toolset-1.14/root/usr/lib/go-toolset-1.14-golang/src/runtime/netpoll_kqueue.go:140 +0x30a runtime.findrunnable(0xc00008e800, 0x0) /opt/rh/go-toolset-1.14/root/usr/lib/go-toolset-1.14-golang/src/runtime/proc.go:2329 +0x72b runtime.schedule() /opt/rh/go-toolset-1.14/root/usr/lib/go-toolset-1.14-golang/src/runtime/proc.go:2526 +0x2fc runtime.park_m(0xc000340180) /opt/rh/go-toolset-1.14/root/usr/lib/go-toolset-1.14-golang/src/runtime/proc.go:2696 +0x9d runtime.mcall(0x80000) /opt/rh/go-toolset-1.14/root/usr/lib/go-toolset-1.14-golang/src/runtime/asm_amd64.s:318 +0x5b goroutine 1 [runnable, locked to thread]: reflect.Value.Pointer(0xcd7dcc0, 0x0, 0x16, 0x12) /opt/rh/go-toolset-1.14/root/usr/lib/go-toolset-1.14-golang/src/reflect/value.go:1429 +0x1af github.com/golang/protobuf/proto.RegisterType(0xebdf320, 0x0, 0xd55f527, 0x12) /go/src/github.com/openshift/installer/vendor/github.com/golang/protobuf/proto/properties.go:486 +0x324 github.com/hashicorp/terraform-plugin-sdk/tfplugin5.init.0() /go/src/github.com/openshift/installer/vendor/github.com/hashicorp/terraform-plugin-sdk/tfplugin5/tfplugin5.pb.go:2651 +0x2bb goroutine 18 [chan receive]: k8s.io/klog.(*loggingT).flushDaemon(0x161e40c0) /go/src/github.com/openshift/installer/vendor/k8s.io/klog/klog.go:1010 +0x8b created by k8s.io/klog.init.0 /go/src/github.com/openshift/installer/vendor/k8s.io/klog/klog.go:411 +0xd6 goroutine 7 [select]: go.opencensus.io/stats/view.(*worker).start(0xc00029c0a0) /go/src/github.com/openshift/installer/vendor/go.opencensus.io/stats/view/worker.go:154 +0x100 created by go.opencensus.io/stats/view.init.0 /go/src/github.com/openshift/installer/vendor/go.opencensus.io/stats/view/worker.go:32 +0x57 Some related issues I found with a quick search: https://github.com/golang/go/issues/35566 https://github.com/golang/go/issues/37726 https://go-review.googlesource.com/c/go/+/207078/ https://github.com/hashicorp/packer/issues/9032 https://github.com/hashicorp/terraform/pull/24562/commits/cd65b28da051174a13ac76e54b7bb95d3051255c https://github.com/hashicorp/terraform/issues/24287 [1] https://mirror.openshift.com/pub/openshift-v4/clients/ocp-dev-preview/latest-4.6/openshift-install-mac-4.6.0-0.nightly-2020-07-06-084958.tar.gz