Bug 2031056
| Summary: | [vsphere] mco could not be started with error "invalid memory address or nil pointer dereference" | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | jima |
| Component: | Machine Config Operator | Assignee: | MCO Team <team-mco> |
| Machine Config Operator sub component: | platform-vsphere | QA Contact: | Rio Liu <rioliu> |
| Status: | CLOSED DUPLICATE | Docs Contact: | |
| Severity: | unspecified | ||
| Priority: | unspecified | CC: | aos-bugs, mkrejci |
| Version: | 4.10 | ||
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2021-12-10 12:14:40 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
*** This bug has been marked as a duplicate of bug 2031049 *** |
Description of problem: Install upi-on-vsphere cluster against 4.10.0-0.nightly-2021-12-10-033652, where boot image is rhcos-410.84.202112040202-0. Installation failed and worker nodes could not get ignition file. Detailed check: $ oc get pod -n openshift-machine-config-operator NAME READY STATUS RESTARTS AGE machine-config-operator-5b98d5894f-bh7jt 0/1 CrashLoopBackOff 18 (4m16s ago) 107m $ oc logs machine-config-operator-5b98d5894f-bh7jt -n openshift-machine-config-operator I1210 09:29:24.385964 1 start.go:43] Version: 4.10.0-0.nightly-2021-12-10-033652 (Raw: v4.10.0-202112100221.p0.g3cc5461.assembly.stream-dirty, Hash: 3cc546103bc9042d899c34ada12e429b86aa6547) I1210 09:29:24.388047 1 leaderelection.go:248] attempting to acquire leader lease openshift-machine-config-operator/machine-config... I1210 09:31:20.044690 1 leaderelection.go:258] successfully acquired lease openshift-machine-config-operator/machine-config I1210 09:31:20.464300 1 operator.go:267] Starting MachineConfigOperator E1210 09:31:20.496277 1 runtime.go:78] Observed a panic: "invalid memory address or nil pointer dereference" (runtime error: invalid memory address or nil pointer dereference) goroutine 276 [running]: k8s.io/apimachinery/pkg/util/runtime.logPanic({0x16a1f00, 0x28267f0}) /go/src/github.com/openshift/machine-config-operator/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:74 +0x85 k8s.io/apimachinery/pkg/util/runtime.HandleCrash({0x0, 0x0, 0xc00054ea40}) /go/src/github.com/openshift/machine-config-operator/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:48 +0x75 panic({0x16a1f00, 0x28267f0}) /usr/lib/golang/src/runtime/panic.go:1038 +0x215 github.com/openshift/machine-config-operator/pkg/operator.getIgnitionHost(0xc0004fc3e8) /go/src/github.com/openshift/machine-config-operator/pkg/operator/sync.go:356 +0x22a github.com/openshift/machine-config-operator/pkg/operator.(*Operator).syncRenderConfig(0xc0004fc000, 0x4b7477) /go/src/github.com/openshift/machine-config-operator/pkg/operator/sync.go:320 +0xdbb github.com/openshift/machine-config-operator/pkg/operator.(*Operator).syncAll(0xc0004fc000, {0xc0009d5d00, 0x6, 0x0}) /go/src/github.com/openshift/machine-config-operator/pkg/operator/sync.go:113 +0x3b8 github.com/openshift/machine-config-operator/pkg/operator.(*Operator).sync(0xc0004fc000, {0xc00050b6e0, 0x30}) /go/src/github.com/openshift/machine-config-operator/pkg/operator/operator.go:359 +0x453 github.com/openshift/machine-config-operator/pkg/operator.(*Operator).processNextWorkItem(0xc0004fc000) /go/src/github.com/openshift/machine-config-operator/pkg/operator/operator.go:315 +0xe5 github.com/openshift/machine-config-operator/pkg/operator.(*Operator).worker(...) /go/src/github.com/openshift/machine-config-operator/pkg/operator/operator.go:304 k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0x7f1ef2935c50) /go/src/github.com/openshift/machine-config-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:155 +0x67 k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0x0, {0x1af8fe0, 0xc00031e030}, 0x1, 0xc00041a2a0) /go/src/github.com/openshift/machine-config-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:156 +0xb6 k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc00041b1a0, 0x3b9aca00, 0x0, 0x0, 0x43cfe5) /go/src/github.com/openshift/machine-config-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133 +0x89 k8s.io/apimachinery/pkg/util/wait.Until(0x87adea, 0xc0002c5a20, 0x0) /go/src/github.com/openshift/machine-config-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:90 +0x25 created by github.com/openshift/machine-config-operator/pkg/operator.(*Operator).Run /go/src/github.com/openshift/machine-config-operator/pkg/operator/operator.go:273 +0x6c9 panic: runtime error: invalid memory address or nil pointer dereference [recovered] panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x14d8e6a] goroutine 276 [running]: k8s.io/apimachinery/pkg/util/runtime.HandleCrash({0x0, 0x0, 0xc00054ea40}) /go/src/github.com/openshift/machine-config-operator/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:55 +0xd8 panic({0x16a1f00, 0x28267f0}) /usr/lib/golang/src/runtime/panic.go:1038 +0x215 github.com/openshift/machine-config-operator/pkg/operator.getIgnitionHost(0xc0004fc3e8) /go/src/github.com/openshift/machine-config-operator/pkg/operator/sync.go:356 +0x22a github.com/openshift/machine-config-operator/pkg/operator.(*Operator).syncRenderConfig(0xc0004fc000, 0x4b7477) /go/src/github.com/openshift/machine-config-operator/pkg/operator/sync.go:320 +0xdbb github.com/openshift/machine-config-operator/pkg/operator.(*Operator).syncAll(0xc0004fc000, {0xc0009d5d00, 0x6, 0x0}) /go/src/github.com/openshift/machine-config-operator/pkg/operator/sync.go:113 +0x3b8 github.com/openshift/machine-config-operator/pkg/operator.(*Operator).sync(0xc0004fc000, {0xc00050b6e0, 0x30}) /go/src/github.com/openshift/machine-config-operator/pkg/operator/operator.go:359 +0x453 github.com/openshift/machine-config-operator/pkg/operator.(*Operator).processNextWorkItem(0xc0004fc000) /go/src/github.com/openshift/machine-config-operator/pkg/operator/operator.go:315 +0xe5 github.com/openshift/machine-config-operator/pkg/operator.(*Operator).worker(...) /go/src/github.com/openshift/machine-config-operator/pkg/operator/operator.go:304 k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0x7f1ef2935c50) /go/src/github.com/openshift/machine-config-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:155 +0x67 k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0x0, {0x1af8fe0, 0xc00031e030}, 0x1, 0xc00041a2a0) /go/src/github.com/openshift/machine-config-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:156 +0xb6 k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc00041b1a0, 0x3b9aca00, 0x0, 0x0, 0x43cfe5) /go/src/github.com/openshift/machine-config-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133 +0x89 k8s.io/apimachinery/pkg/util/wait.Until(0x87adea, 0xc0002c5a20, 0x0) /go/src/github.com/openshift/machine-config-operator/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:90 +0x25 created by github.com/openshift/machine-config-operator/pkg/operator.(*Operator).Run /go/src/github.com/openshift/machine-config-operator/pkg/operator/operator.go:273 +0x6c9 Version-Release number of MCO (Machine Config Operator) (if applicable): 4.10.0-0.nightly-2021-12-10-033652 Platform: vsphere Are you certain that the root cause of the issue being reported is the MCO (Machine Config Operator)? Not sure How reproducible: Always when installing upi-on-vsphere with boot image rhcos-410.84.202112040202-0, ipi-on-vsphere against same nightly build is successful. Did you catch this issue by running a Jenkins job? If yes, please list: 1. Jenkins job: https://mastern-jenkins-csb-openshift-qe.apps.ocp-c1.prod.psi.redhat.com/job/ocp-common/job/Flexy-install/58863 2. Profile: common installation on upi-on-vsphere Steps to Reproduce: 1. install upi-on-vsphere with boot image rhcos-410.84.202112040202-0 2. 3. Actual results: Installation failed and worker could not not get ignition file. Expected results: Installation is successful. Additional info: Also get same issue in CI: https://search.ci.openshift.org/?search=invalid+memory+address+or+nil+pointer+dereference&maxAge=48h&context=1&type=junit&name=&excludeName=&maxMatches=5&maxBytes=20971520&groupBy=job