Hit this in CI https://prow.svc.ci.openshift.org/view/gcs/origin-ci-test/pr-logs/pull/23461/pull-ci-openshift-origin-master-e2e-aws/11505 https://storage.googleapis.com/origin-ci-test/pr-logs/pull/23461/pull-ci-openshift-origin-master-e2e-aws/11505/artifacts/e2e-aws/pods/openshift-machine-api_machine-api-operator-5669c4cdff-ssnf9_machine-api-operator_previous.log I0723 20:36:37.879244 1 start.go:58] Version: 0.1.0-463-g0c0a0666-dirty I0723 20:36:37.882340 1 leaderelection.go:217] attempting to acquire leader lease openshift-machine-api/machine-api-operator... I0723 20:38:33.536335 1 leaderelection.go:227] successfully acquired lease openshift-machine-api/machine-api-operator I0723 20:38:33.543384 1 operator.go:111] Starting Machine API Operator I0723 20:38:33.642964 1 start.go:104] Synced up machine api informer caches I0723 20:38:33.643594 1 operator.go:120] Synced up caches I0723 20:38:33.654707 1 status.go:58] Syncing status: re-syncing I0723 20:38:34.682851 1 sync.go:40] Synced up all components I0723 20:38:34.686491 1 status.go:89] Syncing status: available panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0x118beef] goroutine 124 [running]: github.com/openshift/machine-api-operator/pkg/metrics.MachineCollector.collectMachineMetrics(0x174bf40, 0xc0002b0190, 0x174bf80, 0xc0002b01b0, 0xc000042074, 0x15, 0xc00007e780) /go/src/github.com/openshift/machine-api-operator/pkg/metrics/metrics.go:90 +0x33f github.com/openshift/machine-api-operator/pkg/metrics.(*MachineCollector).Collect(0xc0004282d0, 0xc00007e780) /go/src/github.com/openshift/machine-api-operator/pkg/metrics/metrics.go:59 +0x86 github.com/openshift/machine-api-operator/vendor/github.com/prometheus/client_golang/prometheus.(*Registry).Gather.func1() /go/src/github.com/openshift/machine-api-operator/vendor/github.com/prometheus/client_golang/prometheus/registry.go:434 +0x19d created by github.com/openshift/machine-api-operator/vendor/github.com/prometheus/client_golang/prometheus.(*Registry).Gather /go/src/github.com/openshift/machine-api-operator/vendor/github.com/prometheus/client_golang/prometheus/registry.go:526 +0xdf8
https://github.com/openshift/machine-api-operator/pull/361/files
This code issue was uncovered by a machine not becoming a node consistently for any reason, so it's worth investigating that
Verifed. create a failed machine with 4.2.0-0.nightly-2019-07-21-222447 $ oc logs -f machine-api-operator-7ffdb69466-m447h panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0x1107923] goroutine 13867 [running]: github.com/openshift/machine-api-operator/pkg/metrics.MachineCollector.collectMachineMetrics(0x1644560, 0xc0003a61e0, 0x16445a0, 0xc0003a6200, 0xc000040044, 0x15, 0xc000107da0) /go/src/github.com/openshift/machine-api-operator/pkg/metrics/metrics.go:90 +0x2b3 github.com/openshift/machine-api-operator/pkg/metrics.(*MachineCollector).Collect(0xc00039e420, 0xc000107da0) /go/src/github.com/openshift/machine-api-operator/pkg/metrics/metrics.go:59 +0x4a github.com/openshift/machine-api-operator/vendor/github.com/prometheus/client_golang/prometheus.(*Registry).Gather.func1() /go/src/github.com/openshift/machine-api-operator/vendor/github.com/prometheus/client_golang/prometheus/registry.go:434 +0x193 created by github.com/openshift/machine-api-operator/vendor/github.com/prometheus/client_golang/prometheus.(*Registry).Gather /go/src/github.com/openshift/machine-api-operator/vendor/github.com/prometheus/client_golang/prometheus/registry.go:526 +0xe23 create a failed machine with 4.2.0-0.nightly-2019-07-21-222447, without this error.
*** Bug 1733474 has been marked as a duplicate of this bug. ***
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:2922