Bug 2057301
| Summary: | CDI controller fails with Reconciler error "StorageClass.storage.k8s.io \"storage class name\" not found" | ||
|---|---|---|---|
| Product: | Container Native Virtualization (CNV) | Reporter: | nijin ashok <nashok> |
| Component: | Storage | Assignee: | Alex Kalenyuk <akalenyu> |
| Status: | CLOSED ERRATA | QA Contact: | Yan Du <yadu> |
| Severity: | high | Docs Contact: | |
| Priority: | high | ||
| Version: | 4.9.3 | CC: | akalenyu, alitke, cnv-qe-bugs, fdeutsch, mrashish, yadu |
| Target Milestone: | --- | ||
| Target Release: | 4.9.5 | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | virt-cdi-controller v4.9.5-1, CNV v4.9.5-1 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2022-06-28 16:30:39 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
This was already corrected with PR 2027: https://github.com/kubevirt/containerized-data-importer/pull/2027, and is included in 4.10 Is there a need to backport it to 4.9? Alex, could you please help to backport it to 4.9.5? Test on CNV v4.9.4-67, issue has been fixed. Need to retest on 4.9.5 build Test on CNV-v4.9.5-4, bug has been fixed. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (OpenShift Virtualization 4.9.5 Images), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2022:5389 |
Description of problem: When a storage class is deleted, it doesn't automatically delete the storage profile. ~~~ # oc delete sc ocs-external-storagecluster-cephfs storageclass.storage.k8s.io "ocs-external-storagecluster-cephfs" deleted oc get storageprofile NAME AGE managed-nfs-storage 19d ocs-external-storagecluster-ceph-rbd 19d ocs-external-storagecluster-cephfs 9m51s <<<< storageprofile still exists openshift-storage.noobaa.io 19d ~~~ If we restart the cdi-deployment pod at this stage, it fails with the error below: ~~~ {"level":"error","ts":1645589400.0923438,"logger":"controller-runtime.manager.controller.storageprofile-controller","msg":"Reconciler error","name":"ocs-external-storagecluster-cephfs","namespace":"","error":"StorageClass.storage.k8s.io \"ocs-external-storagecluster-cephfs\" not found","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\t/remote-source/app/vendor/github.com/go-logr/zapr/zapr.go:132\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:302\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:253\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2\n\t/remote-source/app/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:216\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1\n\t/remote-source/app/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\t/remote-source/app/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:155\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\t/remote-source/app/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:156\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/remote-source/app/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext\n\t/remote-source/app/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:185\nk8s.io/apimachinery/pkg/util/wait.UntilWithContext\n\t/remote-source/app/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:99"} ~~~ If an environment with the above conf is upgraded, the HCO will be unavailable with the error "CDINotAvailable". We have to manually delete the storage profile and have to restart the cdi-deployment pod to solve the issue. Version-Release number of selected component (if applicable): OpenShift Virtualization 4.9.3 How reproducible: 100% Steps to Reproduce: 1. Delete a storage class. ~~~ oc delete sc storage-class-name ~~~ 2. Check the list of storage profiles. We have the storage profile pointing to the unavailable storage class. ~~~ oc get storageprofile|grep storage-class-name ~~~ 3. Delete the cdi-deployment pod. ~~~ oc delete pod cdi-deployment-xxxx ~~~ 4. Watch the logs of the restarted pod. ~~~ oc logs -f cdi-deployment-xxxx ~~~ Actual results: CDI controller fails with Reconciler error "StorageClass.storage.k8s.io \"storage class name\" not found" Expected results: I think when a storage class is deleted, it should also delete the storage profile since those are created by CDI. Additional info: