Bug 2041530

Description of problem:
HostPathProvisioner CSI custom resource can't be deleted if it's a combination of a basic storage pool, and a pvcTemplate. This causes the hostpath-provisioner-operator to CrashLoopBackOff.

We are NOT facing this issue in HPP CR with just a basic storage pool, or in HPP CR with just a pvcTemplate.

Version-Release number of selected component (if applicable):
4.10

How reproducible:
Always

Steps to Reproduce:
1. Get a cnv 4.10 cluster
2. Deploy an HPP CR with both basic storage pool and pvcTemplate
3. Delete HPP CR

Actual results:
HPP CR is not deleted, hostpath-provisioner-operator - CrashLoopBackOff.

Expected results:
HPP CR deleted.

Additional info:

HPP CR YAML:

$ cat hpp-cr-basic-pvc-block.yaml
apiVersion: hostpathprovisioner.kubevirt.io/v1beta1
kind: HostPathProvisioner
metadata:
  name: hostpath-provisioner
spec:
  imagePullPolicy: IfNotPresent
  storagePools:
      - name: local-basic
        path: "/var/local-basic"
      - name: local-pvc-template
        pvcTemplate:
          volumeMode: Block
          storageClassName: ocs-storagecluster-ceph-rbd
          accessModes:
          - ReadWriteOnce
          resources:
            requests:
              storage: 40Gi
        path: "/var/hpvolumes"
  workload:
   nodeSelector:
     kubernetes.io/os: linux



$ oc delete HostPathProvisioner hostpath-provisioner
hostpathprovisioner.hostpathprovisioner.kubevirt.io "hostpath-provisioner" deleted
^C
$
$ oc get pods -A | grep hostpath
openshift-cnv                                      hostpath-provisioner-csi-2z2jq                                    4/4     Running            0               8m33s
openshift-cnv                                      hostpath-provisioner-csi-cmlxz                                    4/4     Running            0               8m33s
openshift-cnv                                      hostpath-provisioner-csi-tggh6                                    4/4     Running            0               8m33s
openshift-cnv                                      hostpath-provisioner-operator-79c988456c-5mnqf                    0/1     CrashLoopBackOff   5 (64s ago)     21m


$ oc logs -n openshift-cnv hostpath-provisioner-operator-79c988456c-5mnqf -p
{"level":"info","ts":1642427183.4980357,"logger":"cmd","msg":"Go Version: go1.16.6"}
{"level":"info","ts":1642427183.498063,"logger":"cmd","msg":"Go OS/Arch: linux/amd64"}
{"level":"info","ts":1642427183.4980674,"logger":"cmd","msg":"Version of operator-sdk: v0.16.0"}
I0117 13:46:24.549470       1 request.go:668] Waited for 1.032877849s due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/apis/rbac.authorization.k8s.io/v1?timeout=32s
{"level":"info","ts":1642427187.3572679,"logger":"controller-runtime.metrics","msg":"metrics server is starting to listen","addr":":8080"}
{"level":"info","ts":1642427187.3657196,"logger":"cmd","msg":"Registering Components."}
{"level":"info","ts":1642427187.3664596,"logger":"controller-runtime.builder","msg":"skip registering a mutating webhook, admission.Defaulter interface is not implemented","GVK":"hostpathprovisioner.kubevirt.io/v1beta1, Kind=HostPathProvisioner"}
{"level":"info","ts":1642427187.366523,"logger":"controller-runtime.builder","msg":"Registering a validating webhook","GVK":"hostpathprovisioner.kubevirt.io/v1beta1, Kind=HostPathProvisioner","path":"/validate-hostpathprovisioner-kubevirt-io-v1beta1-hostpathprovisioner"}
{"level":"info","ts":1642427187.3666127,"logger":"controller-runtime.webhook","msg":"registering webhook","path":"/validate-hostpathprovisioner-kubevirt-io-v1beta1-hostpathprovisioner"}
{"level":"info","ts":1642427187.3666918,"logger":"cmd","msg":"Starting the Cmd."}
I0117 13:46:27.366945       1 leaderelection.go:243] attempting to acquire leader lease openshift-cnv/hostpath-provisioner-operator-lock...
{"level":"info","ts":1642427187.366945,"msg":"starting metrics server","path":"/metrics"}
{"level":"info","ts":1642427187.3670144,"logger":"controller-runtime.webhook.webhooks","msg":"starting webhook server"}
{"level":"info","ts":1642427187.3749099,"logger":"controller-runtime.certwatcher","msg":"Updated current TLS certificate"}
{"level":"info","ts":1642427187.375009,"logger":"controller-runtime.webhook","msg":"serving webhook server","host":"","port":9443}
{"level":"info","ts":1642427187.3757684,"logger":"controller-runtime.certwatcher","msg":"Starting certificate watcher"}
I0117 13:46:44.468614       1 leaderelection.go:253] successfully acquired lease openshift-cnv/hostpath-provisioner-operator-lock
{"level":"info","ts":1642427204.4702606,"logger":"controller.hostpathprovisioner-controller","msg":"Starting EventSource","source":"kind source: /, Kind="}
{"level":"info","ts":1642427204.4703252,"logger":"controller.hostpathprovisioner-controller","msg":"Starting EventSource","source":"kind source: /, Kind="}
{"level":"info","ts":1642427204.4703326,"logger":"controller.hostpathprovisioner-controller","msg":"Starting EventSource","source":"kind source: /, Kind="}
{"level":"info","ts":1642427204.470341,"logger":"controller.hostpathprovisioner-controller","msg":"Starting EventSource","source":"kind source: /, Kind="}
{"level":"info","ts":1642427204.4703543,"logger":"controller.hostpathprovisioner-controller","msg":"Starting EventSource","source":"kind source: /, Kind="}
{"level":"info","ts":1642427204.4703605,"logger":"controller.hostpathprovisioner-controller","msg":"Starting EventSource","source":"kind source: /, Kind="}
{"level":"info","ts":1642427204.4703777,"logger":"controller.hostpathprovisioner-controller","msg":"Starting EventSource","source":"kind source: /, Kind="}
{"level":"info","ts":1642427204.4703848,"logger":"controller.hostpathprovisioner-controller","msg":"Starting EventSource","source":"kind source: /, Kind="}
{"level":"info","ts":1642427204.4703944,"logger":"controller.hostpathprovisioner-controller","msg":"Starting EventSource","source":"kind source: /, Kind="}
{"level":"info","ts":1642427204.4704046,"logger":"controller.hostpathprovisioner-controller","msg":"Starting EventSource","source":"kind source: /, Kind="}
{"level":"info","ts":1642427204.4704115,"logger":"controller.hostpathprovisioner-controller","msg":"Starting EventSource","source":"kind source: /, Kind="}
{"level":"info","ts":1642427204.4704182,"logger":"controller.hostpathprovisioner-controller","msg":"Starting EventSource","source":"kind source: /, Kind="}
{"level":"info","ts":1642427204.4704297,"logger":"controller.hostpathprovisioner-controller","msg":"Starting EventSource","source":"kind source: /, Kind="}
{"level":"info","ts":1642427204.4704354,"logger":"controller.hostpathprovisioner-controller","msg":"Starting EventSource","source":"kind source: /, Kind="}
{"level":"info","ts":1642427204.4704466,"logger":"controller.hostpathprovisioner-controller","msg":"Starting EventSource","source":"kind source: /, Kind="}
{"level":"info","ts":1642427204.470458,"logger":"controller.hostpathprovisioner-controller","msg":"Starting Controller"}
{"level":"info","ts":1642427204.5743701,"logger":"controller.hostpathprovisioner-controller","msg":"Starting workers","worker count":1}
{"level":"info","ts":1642427204.574589,"logger":"controller_hostpathprovisioner","msg":"Detected CSI CR, reconciling CSI only","Request.Namespace":"","Request.Name":"hostpath-provisioner"}
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x48 pc=0x155adb2]
 
goroutine 672 [running]:
kubevirt.io/hostpath-provisioner-operator/pkg/controller/hostpathprovisioner.(*ReconcileHostPathProvisioner).storagePoolDeploymentByNode(0xc00030c480, 0x1abed68, 0xc001355d80, 0xc00072f600, 0xc00123f6b0, 0xc000046030, 0xd, 0xc0006f2000, 0xc0006f2000)
        /remote-source/app/pkg/controller/hostpathprovisioner/storagepool.go:330 +0x252
kubevirt.io/hostpath-provisioner-operator/pkg/controller/hostpathprovisioner.(*ReconcileHostPathProvisioner).cleanDeployments(0xc00030c480, 0x1abed68, 0xc001355d80, 0xc00072f600, 0xc000046030, 0xd, 0x14, 0x1ad8350)
        /remote-source/app/pkg/controller/hostpathprovisioner/storagepool.go:116 +0x1ee
kubevirt.io/hostpath-provisioner-operator/pkg/controller/hostpathprovisioner.(*ReconcileHostPathProvisioner).Reconcile(0xc00030c480, 0x1ab61f8, 0xc00055b1a0, 0x0, 0x0, 0xc0005b10b0, 0x14, 0xc00055b1a0, 0xc00055b110, 0xc00123fdb0, ...)
        /remote-source/app/pkg/controller/hostpathprovisioner/controller.go:308 +0x40b
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile(0xc0001cab40, 0x1ab61f8, 0xc00055b110, 0x0, 0x0, 0xc0005b10b0, 0x14, 0xc00055b100, 0x0, 0x0, ...)
        /remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:114 +0x247
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc0001cab40, 0x1ab6150, 0xc0007bc000, 0x1728440, 0xc0002c8180)
        /remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:311 +0x305
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc0001cab40, 0x1ab6150, 0xc0007bc000, 0xc00093a700)
        /remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:266 +0x205
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2(0xc000673c60, 0xc0001cab40, 0x1ab6150, 0xc0007bc000)
        /remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:227 +0x6b
created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2
        /remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:223 +0x425