Description of problem (please be detailed as possible and provide log snippests): Deleting a storageconsumer which is missing heartbeat from client does not automatically deletes the associated storagerequest, cephblockpoolradosnamespace and cephfilesystemsubvolumegroup. Name of the storageclient from hosted cluster. % oc get storageclient NAME PHASE CONSUMER ocs-storagecluster Connected ac2ed050-d5ce-49cd-be39-343a6836eff0 % oc get storageclient ocs-storagecluster -n openshift-storage-client -o=jsonpath={.status.id} ac2ed050-d5ce-49cd-be39-343a6836eff0 On provider side, the storageconsumer name is storageconsumer-276f92ad-8b8f-4aeb-beb5-5a9f6e0b8304 % oc get storageconsumer storageconsumer-276f92ad-8b8f-4aeb-beb5-5a9f6e0b8304 -o yaml | grep ac2ed050-d5ce-49cd-be39-343a6836eff0 uid: ac2ed050-d5ce-49cd-be39-343a6836eff0 Cephfilesystemsubvolumegroups is cephfilesystemsubvolumegroup-822d83fa5a6f6ca4cb3dc8502a32bbae % oc get cephfilesystemsubvolumegroups cephfilesystemsubvolumegroup-822d83fa5a6f6ca4cb3dc8502a32bbae -o yaml | grep storageconsumer-276f92ad-8b8f-4aeb-beb5-5a9f6e0b8304 ocs.openshift.io/storageconsumer-name: storageconsumer-276f92ad-8b8f-4aeb-beb5-5a9f6e0b8304 Cephblockpoolradosnamespace is cephradosnamespace-d1e7af4edd1801d99d70245229722eae % oc get cephblockpoolradosnamespace cephradosnamespace-d1e7af4edd1801d99d70245229722eae -o yaml| grep storageconsumer-276f92ad-8b8f-4aeb-beb5-5a9f6e0b8304 ocs.openshift.io/storageconsumer-name: storageconsumer-276f92ad-8b8f-4aeb-beb5-5a9f6e0b8304 Cephblockpool is ocs-storagecluster-cephblockpool Cephfilesystem is ocs-storagecluster-cephfilesystem The storageconsumer is deleted from UI(Storage--> StorageClients page) % oc get storageconsumer storageconsumer-276f92ad-8b8f-4aeb-beb5-5a9f6e0b8304 -n openshift-storage Error from server (NotFound): storageconsumers.ocs.openshift.io "storageconsumer-276f92ad-8b8f-4aeb-beb5-5a9f6e0b8304" not found cephfilesystemsubvolumegroups cephfilesystemsubvolumegroup-822d83fa5a6f6ca4cb3dc8502a32bbae is not deleted. % oc get cephfilesystemsubvolumegroups cephfilesystemsubvolumegroup-822d83fa5a6f6ca4cb3dc8502a32bbae -o yaml | grep storageconsumer-276f92ad-8b8f-4aeb-beb5-5a9f6e0b8304 ocs.openshift.io/storageconsumer-name: storageconsumer-276f92ad-8b8f-4aeb-beb5-5a9f6e0b8304 Cephblockpoolradosnamespace cephradosnamespace-d1e7af4edd1801d99d70245229722eae is not deleted. % oc get cephblockpoolradosnamespace cephradosnamespace-d1e7af4edd1801d99d70245229722eae -o yaml| grep storageconsumer-276f92ad-8b8f-4aeb-beb5-5a9f6e0b8304 ocs.openshift.io/storageconsumer-name: storageconsumer-276f92ad-8b8f-4aeb-beb5-5a9f6e0b8304 Storagerequests are not deleted % oc get storagerequest storagerequest-7fcd02b35d53d25c5f9377818ca4ba4c storagerequest-cbebe0ef42adfc73682d262fdf210ae7 NAME STORAGETYPE PHASE storagerequest-7fcd02b35d53d25c5f9377818ca4ba4c block Initializing storagerequest-cbebe0ef42adfc73682d262fdf210ae7 sharedfile Initializing % oc get storagerequest storagerequest-7fcd02b35d53d25c5f9377818ca4ba4c storagerequest-cbebe0ef42adfc73682d262fdf210ae7 -o yaml | grep storageconsumer-276f92ad-8b8f-4aeb-beb5-5a9f6e0b8304 name: storageconsumer-276f92ad-8b8f-4aeb-beb5-5a9f6e0b8304 name: storageconsumer-276f92ad-8b8f-4aeb-beb5-5a9f6e0b8304 Logs http://magna002.ceph.redhat.com/ocsci-jenkins/openshift-clusters/ibm-baremetal2/ibm-baremetal2_20240429T123517/logs/testcases_1715860519/ibm-baremetal2/ ======================================================================= Version of all relevant components (if applicable): % oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.16.0-ec.5 True False 16d Cluster version is 4.16.0-ec.5 % oc get csv NAME DISPLAY VERSION REPLACES PHASE mcg-operator.v4.16.0-96.stable NooBaa Operator 4.16.0-96.stable mcg-operator.v4.16.0-92.stable Succeeded ocs-client-operator.v4.16.0-96.stable OpenShift Data Foundation Client 4.16.0-96.stable ocs-client-operator.v4.16.0-92.stable Succeeded ocs-operator.v4.16.0-96.stable OpenShift Container Storage 4.16.0-96.stable ocs-operator.v4.16.0-92.stable Succeeded odf-csi-addons-operator.v4.16.0-96.stable CSI Addons 4.16.0-96.stable odf-csi-addons-operator.v4.16.0-92.stable Succeeded odf-operator.v4.16.0-96.stable OpenShift Data Foundation 4.16.0-96.stable odf-operator.v4.16.0-92.stable Succeeded odf-prometheus-operator.v4.16.0-96.stable Prometheus Operator 4.16.0-96.stable odf-prometheus-operator.v4.16.0-92.stable Succeeded recipe.v4.16.0-96.stable Recipe 4.16.0-96.stable Succeeded rook-ceph-operator.v4.16.0-96.stable Rook-Ceph 4.16.0-96.stable rook-ceph-operator.v4.16.0-92.stable Succeeded Hosted cluster which is having the storageclient also have the same OCP and client operator version. ======================================================================= Does this issue impact your ability to continue to work with the product (please explain in detail what is the user impact)? Removing the storageconsumer does not do the cleanup correctly Is there any workaround available to the best of your knowledge? Rate from 1 - 5 the complexity of the scenario you performed that caused this bug (1 - very simple, 5 - very complex)? 3 Can this issue reproducible? Reporting the fist instance Can this issue reproduce from the UI? Storageconsumer was deleted from UI If this is a regression, please provide more details to justify this: Steps to Reproduce: Follow the steps 1-4 given in https://bugzilla.redhat.com/show_bug.cgi?id=2276055#c8 under "for 3" Actual results: storagerequest, cephblockpoolradosnamespace and cephfilesystemsubvolumegroup are not deleted with storageconsumer when the storageconsumer does not have the heartbeat from the storageclient. Expected results: storagerequest, cephblockpoolradosnamespace and cephfilesystemsubvolumegroup and any images or subvolumes should be removed Additional info: Provider is on BM. Hostedcluster is created using agent.
Verified in version: OCP 4.16.0-ec.6 ODF 4.16.0-110 Same version of client and OCP in the hosted cluster. Client created on hosted cluster using agent. Created two additional storageclaims on client side. This created new cephblockpoolradosnamespace and cephfilesystemsubvolumegroup. So there are two cephblockpoolradosnamespaces and cephfilesystemsubvolumegroups associated with the storageconsumer. Created PVCs using all the 4 storageclasses. Created PVCs are RBD - Block and Filesystem volume mode. CephFS - Filesystem volume mode. 1 GB of data was present on each volume at the time of deleting the hosted cluster. Snapshot of the PVCs were present. Storageclient is % oc get storageclient NAME PHASE CONSUMER ocs-storagecluster Connected 4f6ff6c5-cbbe-4ba0-a050-e72c694c8a52 Storageconsumer on provider is % oc get storageconsumer storageconsumer-b6628a92-4780-4f93-a94b-adb7024dc449 NAME AGE storageconsumer-b6628a92-4780-4f93-a94b-adb7024dc449 6d14h % oc get storageconsumer storageconsumer-b6628a92-4780-4f93-a94b-adb7024dc449 -o yaml apiVersion: ocs.openshift.io/v1alpha1 kind: StorageConsumer metadata: annotations: ocs.openshift.io/provider-onboarding-ticket: eyJpZCI6ImU0ZmEzZjAxLWMyMjctNGFiNi1hY2Y2LWNmNmI3ZGZkNDg3YSIsImV4cGlyYXRpb25EYXRlIjoiMTcxNzE3NTMzMSJ9.JxueAyQ3DNmcWL8KaQsWWdUAHFUo/M2rcmOzRhzk/J2uK7CCPHMGlUt5+/Tp+tyq6jM0h628fWXO88ZNPIOON0QJKfF3Btw9FxLe1bOSWBfIpvIT4AcPfKJTWJcH2xVPM7QV6VghBoTDzA7oZkgxlrWeRyqPDP8GLbPnIrr1UR0FgfZ5i4ZdYSjk2tHy8v/n1lC7WqRvpM+OoKG8c0NvPNF0t/wBqfgFNy6Nh62CbAl7HLiJn+MpguhDZkBWLd+v+GfwfwUXNDr/MhGEnECnp483cIHDdQ5rLgOqSb8F7Nhd4GppKJgedPPKqEieyHXyDK9S65jOckY48z4g0FGgbf+g6MyTwZD0wPXtm8i6Y90xX5EPRuxrpYglAjqMT1jxYc4VJK0R5ZS1C1Ld5ZohM/KcTHVOa9KYOnxvB4dkCFIa6roQj+Zzn5nXXhrIE80Wu3iBsJvgNI4QASfHf2/P24xFrnRKztSi6XcbQKTkc44St8r9n45GMoEPnolD5vxY1b2UD7jA7ozbyKBOXocNv01Q4W1dmcyCvVGBNlRo9e6v32ORuQFhFGbUQv1EMFJBjozeWEK5bs86/wmsmJnv8m9hooCwl9LpcFG+uwHmyeZ5ll8T50vdHLdJoZxsOL1mDUdaXZYDVH4yOSXoQQC8VfSCrW7slteOfX08yvHpUXU= creationTimestamp: "2024-05-29T17:09:30Z" generation: 2 name: storageconsumer-b6628a92-4780-4f93-a94b-adb7024dc449 namespace: openshift-storage resourceVersion: "55593964" uid: 4f6ff6c5-cbbe-4ba0-a050-e72c694c8a52 spec: enable: true status: cephResources: - kind: CephClient name: a850871d1f8bfe7b84af4dc61f335b6d status: Ready client: clusterId: b6628a92-4780-4f93-a94b-adb7024dc449 clusterName: "" name: ocs-storagecluster operatorVersion: 4.16.0-110.stable platformVersion: 4.16.0-ec.6 lastHeartbeat: "2024-06-05T07:45:15Z" state: Ready Storage requests - Created additional storage claim from client side storagerequest-c6c7b03b46ffc2c135dfb3960ae8c740 - sharedfile storagerequest-cd23231e044de2f2775320bbcf508999 - block storagerequest-be9ad34a8be5c58875b1fd63fce10798 - additional ‘sharedfile’ storagerequest-0201bdac5d84f1b480a0581e0c308727 - additional ‘block’ Subvolumegroups 1. cephfilesystemsubvolumegroup-6a395559002d3087b38215eb641637ab % oc rsh rook-ceph-tools-f6958bb97-f5nrj ceph fs subvolume ls ocs-storagecluster-cephfilesystem cephfilesystemsubvolumegroup-6a395559002d3087b38215eb641637ab [ { "name": "csi-vol-7ad367ec-4956-4583-b88e-d10f4826ecc9" }, { "name": "csi-vol-942a89b8-3b57-4b7e-984d-ce81a53bbc16" }, { "name": "csi-vol-3d49d1f2-8e77-4d76-8fd3-1588d0078103" }, { "name": "csi-vol-f984dd06-b1f7-4e90-a7b4-402203ff2840" }, { "name": "csi-vol-8b5d464e-7017-48e6-a040-12bc7ef322c4" }, { "name": "csi-vol-e9e26adc-a74a-4c13-8614-db8b60e1f122" } ] 2. cephfilesystemsubvolumegroup-e5ce334603fe1b1ea7eb913969ec3399 % oc rsh rook-ceph-tools-f6958bb97-f5nrj ceph fs subvolume ls ocs-storagecluster-cephfilesystem cephfilesystemsubvolumegroup-e5ce334603fe1b1ea7eb913969ec3399 [ { "name": "csi-vol-d2d710cf-9133-416d-81ef-88617b304533" }, { "name": "csi-vol-97c4a7cf-7fa2-4d8f-abf7-55bedab1cdf4" }, { "name": "csi-vol-d23576a6-767a-49fe-aedb-2ed56992a301" }, { "name": "csi-vol-cfffa01f-5c83-4229-ab13-54df4da05aac" }, { "name": "csi-vol-d515a7b1-7ff3-4ecf-8ae9-239bfb12d749" }, { "name": "csi-vol-f300b491-5ab8-4f65-a1d8-75c8e903ca1c" }, { "name": "csi-vol-206ee0fd-a542-46ff-bb22-43ce58c601f4" }, { "name": "csi-vol-d0ead75a-23f7-4a37-88ea-9c6728a76b85" } ] Radosnamespaces 2. cephradosnamespace-5ee26effcdd716c681549c517645d8f0 % oc rsh rook-ceph-tools-f6958bb97-f5nrj rbd ls -p ocs-storagecluster-cephblockpool --namespace cephradosnamespace-5ee26effcdd716c681549c517645d8f0 csi-snap-28e7c4eb-c60d-4817-902c-f81f7fd29954 csi-snap-2a423828-1b68-4a28-b6cc-3b198f84420b csi-snap-50888002-4dfd-4c40-9c80-2e1bc152eda8 csi-vol-21e66cc2-40e9-42e3-b511-eefe39725ddf csi-vol-33c68641-2cd3-4373-8831-acff19b1ca04 csi-vol-3452a7bf-2bd3-4d3b-8d9f-d955d48c00eb csi-vol-423afeef-4e62-425d-a884-863688f6673a csi-vol-5e572257-a507-4f0c-96d0-6a53ba678a27 csi-vol-64cd3738-6983-4c42-b8c0-ced3ed5e3d75 csi-vol-ae71c567-16ba-4219-851c-f1197d557eff csi-vol-b7d6ac3f-d339-447e-aec8-79fef319a8fc % oc rsh rook-ceph-tools-f6958bb97-f5nrj rbd ls -l ocs-storagecluster-cephblockpool --namespace cephradosnamespace-5ee26effcdd716c681549c517645d8f0 NAME SIZE PARENT FMT PROT LOCK csi-snap-28e7c4eb-c60d-4817-902c-f81f7fd29954 3 GiB ocs-storagecluster-cephblockpool/cephradosnamespace-5ee26effcdd716c681549c517645d8f0/csi-vol-64cd3738-6983-4c42-b8c0-ced3ed5e3d75@36f6299d-ec19-4023-9736-26c729349340 2 csi-snap-28e7c4eb-c60d-4817-902c-f81f7fd29954@csi-snap-28e7c4eb-c60d-4817-902c-f81f7fd29954 3 GiB ocs-storagecluster-cephblockpool/cephradosnamespace-5ee26effcdd716c681549c517645d8f0/csi-vol-64cd3738-6983-4c42-b8c0-ced3ed5e3d75@36f6299d-ec19-4023-9736-26c729349340 2 csi-snap-2a423828-1b68-4a28-b6cc-3b198f84420b 3 GiB ocs-storagecluster-cephblockpool/cephradosnamespace-5ee26effcdd716c681549c517645d8f0/csi-vol-33c68641-2cd3-4373-8831-acff19b1ca04@8f0dd594-f02e-44ae-b852-484dc43ec2f8 2 csi-snap-2a423828-1b68-4a28-b6cc-3b198f84420b@csi-snap-2a423828-1b68-4a28-b6cc-3b198f84420b 3 GiB ocs-storagecluster-cephblockpool/cephradosnamespace-5ee26effcdd716c681549c517645d8f0/csi-vol-33c68641-2cd3-4373-8831-acff19b1ca04@8f0dd594-f02e-44ae-b852-484dc43ec2f8 2 csi-snap-50888002-4dfd-4c40-9c80-2e1bc152eda8 3 GiB ocs-storagecluster-cephblockpool/cephradosnamespace-5ee26effcdd716c681549c517645d8f0/csi-vol-21e66cc2-40e9-42e3-b511-eefe39725ddf@7e2b4223-f09f-466d-9541-19afa85e2796 2 csi-snap-50888002-4dfd-4c40-9c80-2e1bc152eda8@csi-snap-50888002-4dfd-4c40-9c80-2e1bc152eda8 3 GiB ocs-storagecluster-cephblockpool/cephradosnamespace-5ee26effcdd716c681549c517645d8f0/csi-vol-21e66cc2-40e9-42e3-b511-eefe39725ddf@7e2b4223-f09f-466d-9541-19afa85e2796 2 csi-vol-21e66cc2-40e9-42e3-b511-eefe39725ddf 3 GiB 2 csi-vol-33c68641-2cd3-4373-8831-acff19b1ca04 3 GiB 2 csi-vol-3452a7bf-2bd3-4d3b-8d9f-d955d48c00eb 3 GiB ocs-storagecluster-cephblockpool/cephradosnamespace-5ee26effcdd716c681549c517645d8f0/csi-snap-28e7c4eb-c60d-4817-902c-f81f7fd29954@csi-snap-28e7c4eb-c60d-4817-902c-f81f7fd29954 2 csi-vol-423afeef-4e62-425d-a884-863688f6673a 3 GiB ocs-storagecluster-cephblockpool/cephradosnamespace-5ee26effcdd716c681549c517645d8f0/csi-snap-2a423828-1b68-4a28-b6cc-3b198f84420b@csi-snap-2a423828-1b68-4a28-b6cc-3b198f84420b 2 csi-vol-5e572257-a507-4f0c-96d0-6a53ba678a27 3 GiB ocs-storagecluster-cephblockpool/cephradosnamespace-5ee26effcdd716c681549c517645d8f0/csi-snap-2a423828-1b68-4a28-b6cc-3b198f84420b@csi-snap-2a423828-1b68-4a28-b6cc-3b198f84420b 2 csi-vol-64cd3738-6983-4c42-b8c0-ced3ed5e3d75 3 GiB 2 csi-vol-ae71c567-16ba-4219-851c-f1197d557eff 3 GiB ocs-storagecluster-cephblockpool/cephradosnamespace-5ee26effcdd716c681549c517645d8f0/csi-snap-50888002-4dfd-4c40-9c80-2e1bc152eda8@csi-snap-50888002-4dfd-4c40-9c80-2e1bc152eda8 2 csi-vol-b7d6ac3f-d339-447e-aec8-79fef319a8fc 3 GiB ocs-storagecluster-cephblockpool/cephradosnamespace-5ee26effcdd716c681549c517645d8f0/csi-snap-50888002-4dfd-4c40-9c80-2e1bc152eda8@csi-snap-50888002-4dfd-4c40-9c80-2e1bc152eda8 2 2. cephradosnamespace-e3a6585f78f7f36c3202ebae61d96b20 % oc rsh rook-ceph-tools-f6958bb97-f5nrj rbd ls -p ocs-storagecluster-cephblockpool --namespace cephradosnamespace-e3a6585f78f7f36c3202ebae61d96b20 csi-snap-0d94abe9-e958-4ca0-8a53-d3d36b46a25c csi-snap-cd09cde4-a87e-4d2a-abb6-263c3f7cd7d6 csi-snap-d10722bc-34e2-478c-82a6-b8e515eeb3c5 csi-vol-22b37953-e61b-48e5-9d3d-10f393163fa0 csi-vol-23f7faea-c6c8-4a7e-a753-5afede8b0785 csi-vol-3dd19376-8cc3-4938-b0c3-91ef114bcec3 csi-vol-708b4f4b-9829-4d6c-b932-2bf277618e0a csi-vol-7ec4511f-9220-428c-8385-4a8bad7e4aae csi-vol-acc0dc3f-a988-475b-b330-d08d35e9e5c4 csi-vol-b143a421-7f27-4570-a2e3-13e8f9067d20 csi-vol-b5224027-c88a-49e8-9bfe-a64126d95856 csi-vol-ff1a48f8-2aef-4877-b405-a9273a1c0fef % oc rsh rook-ceph-tools-f6958bb97-f5nrj rbd ls -l ocs-storagecluster-cephblockpool --namespace cephradosnamespace-e3a6585f78f7f36c3202ebae61d96b20 NAME SIZE PARENT FMT PROT LOCK csi-snap-0d94abe9-e958-4ca0-8a53-d3d36b46a25c 3 GiB ocs-storagecluster-cephblockpool/cephradosnamespace-e3a6585f78f7f36c3202ebae61d96b20/csi-vol-b5224027-c88a-49e8-9bfe-a64126d95856@9e7f5122-f512-49f5-ac4d-744d2562dba6 2 csi-snap-0d94abe9-e958-4ca0-8a53-d3d36b46a25c@csi-snap-0d94abe9-e958-4ca0-8a53-d3d36b46a25c 3 GiB ocs-storagecluster-cephblockpool/cephradosnamespace-e3a6585f78f7f36c3202ebae61d96b20/csi-vol-b5224027-c88a-49e8-9bfe-a64126d95856@9e7f5122-f512-49f5-ac4d-744d2562dba6 2 csi-snap-cd09cde4-a87e-4d2a-abb6-263c3f7cd7d6 3 GiB ocs-storagecluster-cephblockpool/cephradosnamespace-e3a6585f78f7f36c3202ebae61d96b20/csi-vol-ff1a48f8-2aef-4877-b405-a9273a1c0fef@0dfe253a-b800-4bff-83a5-28a8f3ac554d 2 csi-snap-cd09cde4-a87e-4d2a-abb6-263c3f7cd7d6@csi-snap-cd09cde4-a87e-4d2a-abb6-263c3f7cd7d6 3 GiB ocs-storagecluster-cephblockpool/cephradosnamespace-e3a6585f78f7f36c3202ebae61d96b20/csi-vol-ff1a48f8-2aef-4877-b405-a9273a1c0fef@0dfe253a-b800-4bff-83a5-28a8f3ac554d 2 csi-snap-d10722bc-34e2-478c-82a6-b8e515eeb3c5 3 GiB ocs-storagecluster-cephblockpool/cephradosnamespace-e3a6585f78f7f36c3202ebae61d96b20/csi-vol-22b37953-e61b-48e5-9d3d-10f393163fa0@72109d38-1bbe-46be-b55c-26eedcf4bee9 2 csi-snap-d10722bc-34e2-478c-82a6-b8e515eeb3c5@csi-snap-d10722bc-34e2-478c-82a6-b8e515eeb3c5 3 GiB ocs-storagecluster-cephblockpool/cephradosnamespace-e3a6585f78f7f36c3202ebae61d96b20/csi-vol-22b37953-e61b-48e5-9d3d-10f393163fa0@72109d38-1bbe-46be-b55c-26eedcf4bee9 2 csi-vol-22b37953-e61b-48e5-9d3d-10f393163fa0 3 GiB 2 csi-vol-23f7faea-c6c8-4a7e-a753-5afede8b0785 3 GiB ocs-storagecluster-cephblockpool/cephradosnamespace-e3a6585f78f7f36c3202ebae61d96b20/csi-snap-d10722bc-34e2-478c-82a6-b8e515eeb3c5@csi-snap-d10722bc-34e2-478c-82a6-b8e515eeb3c5 2 csi-vol-3dd19376-8cc3-4938-b0c3-91ef114bcec3 3 GiB ocs-storagecluster-cephblockpool/cephradosnamespace-e3a6585f78f7f36c3202ebae61d96b20/csi-snap-cd09cde4-a87e-4d2a-abb6-263c3f7cd7d6@csi-snap-cd09cde4-a87e-4d2a-abb6-263c3f7cd7d6 2 csi-vol-708b4f4b-9829-4d6c-b932-2bf277618e0a 3 GiB 2 excl csi-vol-7ec4511f-9220-428c-8385-4a8bad7e4aae 3 GiB ocs-storagecluster-cephblockpool/cephradosnamespace-e3a6585f78f7f36c3202ebae61d96b20/csi-snap-cd09cde4-a87e-4d2a-abb6-263c3f7cd7d6@csi-snap-cd09cde4-a87e-4d2a-abb6-263c3f7cd7d6 2 csi-vol-acc0dc3f-a988-475b-b330-d08d35e9e5c4 3 GiB ocs-storagecluster-cephblockpool/cephradosnamespace-e3a6585f78f7f36c3202ebae61d96b20/csi-snap-0d94abe9-e958-4ca0-8a53-d3d36b46a25c@csi-snap-0d94abe9-e958-4ca0-8a53-d3d36b46a25c 2 csi-vol-b143a421-7f27-4570-a2e3-13e8f9067d20 3 GiB ocs-storagecluster-cephblockpool/cephradosnamespace-e3a6585f78f7f36c3202ebae61d96b20/csi-snap-d10722bc-34e2-478c-82a6-b8e515eeb3c5@csi-snap-d10722bc-34e2-478c-82a6-b8e515eeb3c5 2 csi-vol-b5224027-c88a-49e8-9bfe-a64126d95856 3 GiB 2 csi-vol-ff1a48f8-2aef-4877-b405-a9273a1c0fef 3 GiB 2 Deleted the hosted cluster where the storageclient is present. After 20 minutes, deleted the storageclient from storage clients page in provider cluster UI. At the time of deletion, last heartbeat was showing as 20 minutes ago in the UI. Storageconsumer was removed quickly. Storageconsumer is deleted: % oc get storageconsumer storageconsumer-b6628a92-4780-4f93-a94b-adb7024dc449 Error from server (NotFound): storageconsumers.ocs.openshift.io "storageconsumer-b6628a92-4780-4f93-a94b-adb7024dc449" not found Storagerequests deleted. % oc get storagerequests storagerequest-c6c7b03b46ffc2c135dfb3960ae8c740 storagerequest-cd23231e044de2f2775320bbcf508999 storagerequest-be9ad34a8be5c58875b1fd63fce10798 storagerequest-0201bdac5d84f1b480a0581e0c308727 Error from server (NotFound): storagerequests.ocs.openshift.io "storagerequest-c6c7b03b46ffc2c135dfb3960ae8c740" not found Error from server (NotFound): storagerequests.ocs.openshift.io "storagerequest-cd23231e044de2f2775320bbcf508999" not found Error from server (NotFound): storagerequests.ocs.openshift.io "storagerequest-be9ad34a8be5c58875b1fd63fce10798" not found Error from server (NotFound): storagerequests.ocs.openshift.io "storagerequest-0201bdac5d84f1b480a0581e0c308727" not found Subvolumegroups deleted. % oc rsh rook-ceph-tools-f6958bb97-f5nrj ceph fs subvolume ls ocs-storagecluster-cephfilesystem cephfilesystemsubvolumegroup-6a395559002d3087b38215eb641637ab Error ENOENT: subvolume group 'cephfilesystemsubvolumegroup-6a395559002d3087b38215eb641637ab' does not exist command terminated with exit code 2 % oc rsh rook-ceph-tools-f6958bb97-f5nrj ceph fs subvolume ls ocs-storagecluster-cephfilesystem cephfilesystemsubvolumegroup-e5ce334603fe1b1ea7eb913969ec3399 Error ENOENT: subvolume group 'cephfilesystemsubvolumegroup-e5ce334603fe1b1ea7eb913969ec3399' does not exist command terminated with exit code 2 Radosnamespaces deleted % oc rsh rook-ceph-tools-f6958bb97-f5nrj rbd ls -l ocs-storagecluster-cephblockpool --namespace cephradosnamespace-5ee26effcdd716c681549c517645d8f0 rbd: namespace 'cephradosnamespace-5ee26effcdd716c681549c517645d8f0' does not exist. rbd: listing images failed: (2) No such file or directory command terminated with exit code 2 % oc rsh rook-ceph-tools-f6958bb97-f5nrj rbd ls -l ocs-storagecluster-cephblockpool --namespace cephradosnamespace-e3a6585f78f7f36c3202ebae61d96b20 rbd: namespace 'cephradosnamespace-e3a6585f78f7f36c3202ebae61d96b20' does not exist. rbd: listing images failed: (2) No such file or directory command terminated with exit code 2 The radosnamespace cephradosnamespace-e3a6585f78f7f36c3202ebae61d96b20 and cephradosnamespace-5ee26effcdd716c681549c517645d8f0 are not listed. % oc rsh rook-ceph-tools-f6958bb97-f5nrj rbd namespace list ocs-storagecluster-cephblockpool NAME cephradosnamespace-1518c2a5c832628b5140165c952bb2db cephradosnamespace-279a78543c6d8f28fba43c9046d0ba77 cephradosnamespace-348fc3351f506121ec3444c7ac7f814a cephradosnamespace-4075113d4b7cee60e638c64f77d35b2f cephradosnamespace-4a4263fa3634feb0b74f8ecefb25fa3d cephradosnamespace-5922ae1f367ecab41f8aadaa7c327ca9 cephradosnamespace-d0e0ee9087ba526a589f36e9a38959fa cephradosnamespace-e09b8c7b6b2f387870fe943118e26219
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Important: Red Hat OpenShift Data Foundation 4.16.0 security, enhancement & bug fix update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2024:4591