Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 2033289

Summary: [VMware non Admin user] invalid memory address or nil pointer dereference in forklift-controller
Product: Migration Toolkit for Virtualization Reporter: Gellert Kis <gekis>
Component: ControllerAssignee: Sam Lucidi <slucidi>
Status: CLOSED ERRATA QA Contact: Maayan Hadasi <mguetta>
Severity: high Docs Contact: Avital Pinnick <apinnick>
Priority: unspecified    
Version: 2.2.0CC: fbladilo
Target Milestone: ---   
Target Release: 2.3.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-04-04 18:03:42 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
mtv_ui_screenshot none

Description Gellert Kis 2021-12-16 12:56:08 UTC
Description of problem:

Error could be related to environment of connection , to disks , or a performance load issue relate to big environment.

Maybe related an older issue when cluster had an unusual settings. or inventory related. 

We're advised to open a bug report so we can take a look also with performance team . 

during warm migration , forklift-controller  panic with invalid memory address reference:

# oc logs forklift-controller -f -c main

{"level":"info","ts":1639561300.7314377,"logger":"plan|tzs49","msg":"Conditions.","plan":"openshift-mtv/acronisvm1-plan01","all":{"conditions":[{"type":"Ready","status":"True","category":"Required","message"    :"The migration plan is ready.","lastTransitionTime":"2021-12-15T09:40:03Z"},{"type":"Executing","status":"True","category":"Advisory","message":"The plan is EXECUTING.","lastTransitionTime":"2021-12-15T09:4    0:39Z","durable":true}]}}
134 {"level":"info","ts":1639561300.7315397,"logger":"plan|tzs49","msg":"Reconcile ended.","plan":"openshift-mtv/acronisvm1-plan01","reQ":0}
135 E1215 09:41:40.731906       1 runtime.go:78] Observed a panic: "invalid memory address or nil pointer dereference" (runtime error: invalid memory address or nil pointer dereference)
136 goroutine 431 [running]:
137 k8s.io/apimachinery/pkg/util/runtime.logPanic(0x2828540, 0x45abd80)
138     /remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery.3/pkg/util/runtime/runtime.go:74 +0xa6
139 k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0)
140     /remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery.3/pkg/util/runtime/runtime.go:48 +0x86
141 panic(0x2828540, 0x45abd80)
142     /usr/lib/golang/src/runtime/panic.go:965 +0x1b9
143 github.com/vmware/govmomi/object.(*Task).WaitForResult(0x0, 0x30cb028, 0xc000196008, 0xc0038a8fb0, 0x1, 0x1, 0x9, 0x30cb028, 0xc000196008)
144     /remote-source/deps/gomod/pkg/mod/github.com/vmware/govmomi.1/object/task.go:56 +0x61
145 github.com/konveyor/forklift-controller/pkg/controller/plan/adapter/vsphere.(*Client).CreateSnapshot(0xc00574d5e0, 0xc003841720, 0x9, 0xc003841730, 0xa, 0x0, 0x0, 0xc000e10d00, 0x2cc8a5c, 0x9, ...)
146     /remote-source/app/pkg/controller/plan/adapter/vsphere/client.go:50 +0x190
147 github.com/konveyor/forklift-controller/pkg/controller/plan.(*Migration).execute(0xc0038a97b8, 0xc003610fc0, 0x1, 0x1)
148     /remote-source/app/pkg/controller/plan/migration.go:664 +0x5dbb
149 github.com/konveyor/forklift-controller/pkg/controller/plan.(*Migration).Run(0xc0038a97b8, 0xb2d05e00, 0x0, 0x0)
150     /remote-source/app/pkg/controller/plan/migration.go:170 +0x138
151 github.com/konveyor/forklift-controller/pkg/controller/plan.(*Reconciler).execute(0xc0039496e0, 0xc003672400, 0x0, 0x0, 0x0)
152     /remote-source/app/pkg/controller/plan/controller.go:405 +0x854
153 github.com/konveyor/forklift-controller/pkg/controller/plan.Reconciler.Reconcile(0x30c16e0, 0xc000dc5880, 0x30e2858, 0xc0007bf860, 0xc000a60db0, 0xc001194a00, 0xd, 0xc000b8c570, 0x11, 0x0, ...)
154     /remote-source/app/pkg/controller/plan/controller.go:263 +0x6e5
155 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc000f18ab0, 0x295c340, 0xc000b182a0, 0x0)
156     /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime.4/pkg/internal/controller/controller.go:244 +0x2a9
157 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc000f18ab0, 0x203000)
158     /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime.4/pkg/internal/controller/controller.go:218 +0xb0
159 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker(...)
160     /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime.4/pkg/internal/controller/controller.go:197
161 k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc000e68000)
162     /remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery.3/pkg/util/wait/wait.go:155 +0x5f
163 k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc000e68000, 0x3080060, 0xc000932030, 0xc00074b001, 0xc000a3e1e0)
164     /remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery.3/pkg/util/wait/wait.go:156 +0x9b
165 k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc000e68000, 0x3b9aca00, 0x0, 0x2dfa701, 0xc000a3e1e0)
166     /remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery.3/pkg/util/wait/wait.go:133 +0x98
167 k8s.io/apimachinery/pkg/util/wait.Until(0xc000e68000, 0x3b9aca00, 0xc000a3e1e0)
168     /remote-source/deps/gomod/pkg/mod/k8s.io/apimachinery.3/pkg/util/wait/wait.go:90 +0x4d
169 created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1
170     /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime.4/pkg/internal/controller/controller.go:179 +0x3d6
171 panic: runtime error: invalid memory address or nil pointer dereference [recovered]
172     panic: runtime error: invalid memory address or nil pointer dereference
173 [signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0x199d321]
174 


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1..warm migration of all vm
2.
3.

Actual results:
invalid memory address or nil pointer dereference in forklift-controller

Expected results:
forklift-controller not fails

Additional info:

Comment 1 Ilanit Stein 2021-12-19 15:27:50 UTC
After adding to the VMware non-Admin user the "Manage snapshots" permissions,
the warm migration ended successfully, without the error mentioned in the bug description.

Comment 3 Maayan Hadasi 2022-02-13 11:22:18 UTC
Created attachment 1860862 [details]
mtv_ui_screenshot

Verified as fixed.

VMware warm migration fails as expected in case the VMware user has no 'Snapshot management' privilege.
Error message: "ServerFaultCode: Permission to perform this operation was denied." (see screenshot)

Versions:
mtv-operator-bundle-container-2.3.0-26 / iib:174827
CNV 4.10.0-617 / iib:165547

Comment 6 errata-xmlrpc 2022-04-04 18:03:42 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (MTV 2.3.0 images), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2022:1183