Bug 2021622
| Summary: | Migrated RHV VM causes forklift-controller crash loop due to incorrect detail level in inventory query | ||
|---|---|---|---|
| Product: | Migration Toolkit for Virtualization | Reporter: | Ilanit Stein <istein> |
| Component: | Controller | Assignee: | Sam Lucidi <slucidi> |
| Status: | CLOSED ERRATA | QA Contact: | kpunwatk |
| Severity: | high | Docs Contact: | Avital Pinnick <apinnick> |
| Priority: | high | ||
| Version: | 2.2.0 | CC: | fbladilo, fdupont, slucidi |
| Target Milestone: | --- | Keywords: | Regression |
| Target Release: | 2.2.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2021-12-09 19:21:12 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Caused by not setting the right detail level while querying the inventory for Kubevirt VMs on the destination cluster. Fixed by https://github.com/konveyor/forklift-controller/pull/388 Please verify with mtv-operator-bundle-container-2.2.0-87 / iib:134964, or later. Verified on MTV-2.2.0-87 / iib:134964, CNV-4.9.1-23. Migrated successfully VMs from RHV and also the controller doesn't went into crash loop . Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (MTV 2.2.0 Images), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2021:5066 |
Description of problem: Migrate a VM from RHV cause the controller to be in a crash loop with the error messages [1] Deleting the plan stops the crash loop, but as soon as the plan is recreated and started it fails again. Creating the plan is not enough to trigger the error. probably it's the migration reconcile that fails. Version-Release number of selected component (if applicable): MTV-2.2.0-80 [1] {"level":"info","ts":1636481464.6427453,"logger":"plan|f94ct","msg":"Reconcile ended.","plan":"openshift-mtv/test","reQ":0} E1109 18:11:04.642874 1 runtime.go:78] Observed a panic: "invalid memory address or nil pointer dereference" (runtime error: invalid memory address or nil pointer dereference) goroutine 594 [running]: k8s.io/apimachinery/pkg/util/runtime.logPanic(0x2826f00, 0x45a9d80) /remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery.3/pkg/util/runtime/runtime.go:74 +0xa6 k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0) /remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery.3/pkg/util/runtime/runtime.go:48 +0x86 panic(0x2826f00, 0x45a9d80) /usr/lib/golang/src/runtime/panic.go:965 +0x1b9 github.com/konveyor/forklift-controller/pkg/controller/plan/adapter/ovirt.(*Builder).macConflicts(0xc002a43bc0, 0xc000680800, 0xc000680800, 0xc00047fb30, 0x24, 0xc00044f334, 0xc) /remote-source/app/pkg/controller/plan/adapter/ovirt/builder.go:124 +0x3de github.com/konveyor/forklift-controller/pkg/controller/plan/adapter/ovirt.(*Builder).VirtualMachine(0xc002a43bc0, 0xc00047fb30, 0x24, 0xc00044f334, 0xc, 0x0, 0x0, 0xc00026e6b8, 0xc002d32200, 0x1, ...) /remote-source/app/pkg/controller/plan/adapter/ovirt/builder.go:248 +0xd7 github.com/konveyor/forklift-controller/pkg/controller/plan.(*KubeVirt).virtualMachine(0xc0008cb7d0, 0xc000718c40, 0xc0005ed680, 0x200000003, 0xc0005ed680) /remote-source/app/pkg/controller/plan/kubevirt.go:733 +0x4d3 github.com/konveyor/forklift-controller/pkg/controller/plan.(*KubeVirt).EnsureVM(0xc0042337d0, 0xc000718c40, 0xa, 0xc000e826c0) /remote-source/app/pkg/controller/plan/kubevirt.go:214 +0x50 github.com/konveyor/forklift-controller/pkg/controller/plan.(*Migration).execute(0xc0042337b8, 0xc000718c40, 0x1, 0x1) /remote-source/app/pkg/controller/plan/migration.go:582 +0x28d0 github.com/konveyor/forklift-controller/pkg/controller/plan.(*Migration).Run(0xc0042337b8, 0xb2d05e00, 0x0, 0x0) /remote-source/app/pkg/controller/plan/migration.go:168 +0x138 github.com/konveyor/forklift-controller/pkg/controller/plan.(*Reconciler).execute(0xc000dfe390, 0xc000101800, 0x0, 0x0, 0x0) /remote-source/app/pkg/controller/plan/controller.go:405 +0x854 github.com/konveyor/forklift-controller/pkg/controller/plan.Reconciler.Reconcile(0x30bfca0, 0xc000845540, 0x30e0de8, 0xc00045db60, 0xc0007a1980, 0xc000f01520, 0xd, 0xc000f01504, 0x4, 0x0, ...) /remote-source/app/pkg/controller/plan/controller.go:263 +0x6e5 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc000a16990, 0x295ac80, 0xc000846920, 0x0) /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime.4/pkg/internal/controller/controller.go:244 +0x2a9 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc000a16990, 0x203000) /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime.4/pkg/internal/controller/controller.go:218 +0xb0 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker(...) /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime.4/pkg/internal/controller/controller.go:197 k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc00559c000) /remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery.3/pkg/util/wait/wait.go:155 +0x5f k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc00559c000, 0x307e660, 0xc0000fe030, 0x1, 0xc00073f860) /remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery.3/pkg/util/wait/wait.go:156 +0x9b k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc00559c000, 0x3b9aca00, 0x0, 0x1, 0xc00073f860) /remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery.3/pkg/util/wait/wait.go:133 +0x98 k8s.io/apimachinery/pkg/util/wait.Until(0xc00559c000, 0x3b9aca00, 0xc00073f860) /remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery.3/pkg/util/wait/wait.go:90 +0x4d created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1 /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime.4/pkg/internal/controller/controller.go:179 +0x3d6 panic: runtime error: invalid memory address or nil pointer dereference [recovered] panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x188 pc=0x2353bfe] goroutine 594 [running]: k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0) /remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery.3/pkg/util/runtime/runtime.go:55 +0x109 panic(0x2826f00, 0x45a9d80) /usr/lib/golang/src/runtime/panic.go:965 +0x1b9 github.com/konveyor/forklift-controller/pkg/controller/plan/adapter/ovirt.(*Builder).macConflicts(0xc002a43bc0, 0xc000680800, 0xc000680800, 0xc00047fb30, 0x24, 0xc00044f334, 0xc) /remote-source/app/pkg/controller/plan/adapter/ovirt/builder.go:124 +0x3de github.com/konveyor/forklift-controller/pkg/controller/plan/adapter/ovirt.(*Builder).VirtualMachine(0xc002a43bc0, 0xc00047fb30, 0x24, 0xc00044f334, 0xc, 0x0, 0x0, 0xc00026e6b8, 0xc002d32200, 0x1, ...) /remote-source/app/pkg/controller/plan/adapter/ovirt/builder.go:248 +0xd7 github.com/konveyor/forklift-controller/pkg/controller/plan.(*KubeVirt).virtualMachine(0xc0008cb7d0, 0xc000718c40, 0xc0005ed680, 0x200000003, 0xc0005ed680) /remote-source/app/pkg/controller/plan/kubevirt.go:733 +0x4d3 github.com/konveyor/forklift-controller/pkg/controller/plan.(*KubeVirt).EnsureVM(0xc0042337d0, 0xc000718c40, 0xa, 0xc000e826c0) /remote-source/app/pkg/controller/plan/kubevirt.go:214 +0x50 github.com/konveyor/forklift-controller/pkg/controller/plan.(*Migration).execute(0xc0042337b8, 0xc000718c40, 0x1, 0x1) /remote-source/app/pkg/controller/plan/migration.go:582 +0x28d0 github.com/konveyor/forklift-controller/pkg/controller/plan.(*Migration).Run(0xc0042337b8, 0xb2d05e00, 0x0, 0x0) /remote-source/app/pkg/controller/plan/migration.go:168 +0x138 github.com/konveyor/forklift-controller/pkg/controller/plan.(*Reconciler).execute(0xc000dfe390, 0xc000101800, 0x0, 0x0, 0x0) /remote-source/app/pkg/controller/plan/controller.go:405 +0x854 github.com/konveyor/forklift-controller/pkg/controller/plan.Reconciler.Reconcile(0x30bfca0, 0xc000845540, 0x30e0de8, 0xc00045db60, 0xc0007a1980, 0xc000f01520, 0xd, 0xc000f01504, 0x4, 0x0, ...) /remote-source/app/pkg/controller/plan/controller.go:263 +0x6e5 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc000a16990, 0x295ac80, 0xc000846920, 0x0) /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime.4/pkg/internal/controller/controller.go:244 +0x2a9 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc000a16990, 0x203000) /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime.4/pkg/internal/controller/controller.go:218 +0xb0 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker(...) /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime.4/pkg/internal/controller/controller.go:197 k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc00559c000) /remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery.3/pkg/util/wait/wait.go:155 +0x5f k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc00559c000, 0x307e660, 0xc0000fe030, 0x1, 0xc00073f860) /remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery.3/pkg/util/wait/wait.go:156 +0x9b k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc00559c000, 0x3b9aca00, 0x0, 0x1, 0xc00073f860) /remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery.3/pkg/util/wait/wait.go:133 +0x98 k8s.io/apimachinery/pkg/util/wait.Until(0xc00559c000, 0x3b9aca00, 0xc00073f860) /remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery.3/pkg/util/wait/wait.go:90 +0x4d created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1 /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime.4/pkg/internal/controller/controller.go:179 +0x3d6