Bug 2057301 - CDI controller fails with Reconciler error "StorageClass.storage.k8s.io \"storage class name\" not found"
Summary: CDI controller fails with Reconciler error "StorageClass.storage.k8s.io \"st...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Container Native Virtualization (CNV)
Classification: Red Hat
Component: Storage
Version: 4.9.3
Hardware: All
OS: Linux
high
high
Target Milestone: ---
: 4.9.5
Assignee: Alex Kalenyuk
QA Contact: Yan Du
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-02-23 07:23 UTC by nijin ashok
Modified: 2025-08-08 12:35 UTC (History)
6 users (show)

Fixed In Version: virt-cdi-controller v4.9.5-1, CNV v4.9.5-1
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-06-28 16:30:39 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github kubevirt containerized-data-importer pull 2230 0 None Merged [release-v1.38] Delete profile as a follow up to storage class getting deleted 2022-04-18 15:17:33 UTC
Red Hat Knowledge Base (Solution) 6756531 0 None None None 2022-02-24 11:00:32 UTC
Red Hat Product Errata RHEA-2022:5389 0 None None None 2022-06-28 16:30:54 UTC

Description nijin ashok 2022-02-23 07:23:25 UTC
Description of problem:

When a storage class is deleted, it doesn't automatically delete the storage profile.

~~~
# oc delete sc ocs-external-storagecluster-cephfs
storageclass.storage.k8s.io "ocs-external-storagecluster-cephfs" deleted

oc get storageprofile
NAME                                   AGE
managed-nfs-storage                    19d
ocs-external-storagecluster-ceph-rbd   19d
ocs-external-storagecluster-cephfs     9m51s  <<<< storageprofile still exists
openshift-storage.noobaa.io            19d
~~~

If we restart the cdi-deployment pod at this stage, it fails with the error below:

~~~
{"level":"error","ts":1645589400.0923438,"logger":"controller-runtime.manager.controller.storageprofile-controller","msg":"Reconciler error","name":"ocs-external-storagecluster-cephfs","namespace":"","error":"StorageClass.storage.k8s.io \"ocs-external-storagecluster-cephfs\" not found","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/remote-source/app/vendor/github.com/go-logr/zapr/zapr.go:132\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:302\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:253\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:216\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1\n\t/remote-source/app/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/remote-source/app/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:155\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/remote-source/app/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:156\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/remote-source/app/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext\n\t/remote-source/app/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.UntilWithContext\n\t/remote-source/app/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:99"}
~~~

If an environment with the above conf is upgraded, the HCO will be unavailable with the error "CDINotAvailable".

We have to manually delete the storage profile and have to restart the cdi-deployment pod to solve the issue.

Version-Release number of selected component (if applicable):

OpenShift Virtualization   4.9.3

How reproducible:

100%

Steps to Reproduce:

1. Delete a storage class.

~~~
oc delete sc storage-class-name
~~~

2. Check the list of storage profiles. We have the storage profile pointing to the unavailable storage class.

~~~
oc get storageprofile|grep storage-class-name
~~~

3. Delete the cdi-deployment pod.

~~~
oc delete pod cdi-deployment-xxxx
~~~

4. Watch the logs of the restarted pod.

~~~
oc logs -f cdi-deployment-xxxx
~~~

Actual results:

CDI controller fails with Reconciler error  "StorageClass.storage.k8s.io \"storage class name\" not found"

Expected results:

I think when a storage class is deleted, it should also delete the storage profile since those are created by CDI.

Additional info:

Comment 2 Bartosz Rybacki 2022-03-31 09:24:10 UTC
This was already corrected with PR 2027: https://github.com/kubevirt/containerized-data-importer/pull/2027, and is included in 4.10

Is there a need to backport it to 4.9?

Comment 4 Yan Du 2022-04-06 12:30:02 UTC
Alex, could you please help to backport it to 4.9.5?

Comment 5 Yan Du 2022-04-20 13:56:54 UTC
Test on CNV v4.9.4-67, issue has been fixed.
Need to retest on 4.9.5 build

Comment 6 Yan Du 2022-05-06 08:11:59 UTC
Test on CNV-v4.9.5-4, bug has been fixed.

Comment 12 errata-xmlrpc 2022-06-28 16:30:39 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Virtualization 4.9.5 Images), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2022:5389


Note You need to log in before you can comment on or make changes to this bug.