Back to bug 1974344

Who When What Removed Added
Martin Bukatovic 2021-06-21 12:26:54 UTC Keywords Regression
Nimrod Becker 2021-06-21 12:34:35 UTC Flags needinfo?(mbukatov)
Martin Bukatovic 2021-06-21 13:00:17 UTC Component Multi-Cloud Object Gateway ceph-monitoring
Assignee nbecker nthomas
QA Contact ratamir ebenahar
Flags needinfo?(mbukatov)
Nishanth Thomas 2021-06-22 05:49:43 UTC CC nthomas
Assignee nthomas asachan
Martin Bukatovic 2021-06-22 10:00:07 UTC CC madam
Component ceph-monitoring rook
Assignee asachan tnielsen
Summary [arbiter] critical ClusterObjectStoreState alert firing after installation of arbiter storage cluster [arbiter] critical ClusterObjectStoreState alert firing after installation of arbiter storage cluster, likely because ceph object user for cephobjectstore fails to be created
Travis Nielsen 2021-06-22 17:46:25 UTC Flags needinfo?(mbukatov)
Martin Bukatovic 2021-06-22 19:39:46 UTC Flags needinfo?(mbukatov)
Travis Nielsen 2021-06-22 20:52:54 UTC Assignee tnielsen shan
Sébastien Han 2021-06-23 08:06:43 UTC Flags needinfo?(mbukatov)
Sébastien Han 2021-06-23 08:07:53 UTC Status NEW ASSIGNED
Martin Bukatovic 2021-06-23 11:09:20 UTC Flags needinfo?(mbukatov)
Sébastien Han 2021-06-23 15:24:01 UTC Status ASSIGNED NEW
CC shan
Assignee shan tnielsen
Sébastien Han 2021-06-23 16:03:38 UTC Assignee tnielsen shan
Sébastien Han 2021-06-23 16:04:08 UTC Status NEW ASSIGNED
Martin Bukatovic 2021-06-23 21:32:14 UTC Summary [arbiter] critical ClusterObjectStoreState alert firing after installation of arbiter storage cluster, likely because ceph object user for cephobjectstore fails to be created critical ClusterObjectStoreState alert firing after installation of arbiter storage cluster, likely because ceph object user for cephobjectstore fails to be created, when storagecluster is reinstalled
Travis Nielsen 2021-07-02 19:43:58 UTC CC tnielsen
Flags needinfo?(shan)
Sébastien Han 2021-07-06 08:49:49 UTC Flags needinfo?(shan) needinfo?(mbukatov)
Sébastien Han 2021-07-06 12:39:26 UTC Status ASSIGNED POST
Link ID Github rook/rook/pull/8208
Raz Tamir 2021-07-06 14:14:28 UTC CC ratamir
RHEL Program Management 2021-07-06 14:14:34 UTC Target Release --- OCS 4.8.0
Sébastien Han 2021-07-06 15:44:56 UTC Assignee shan brgardne
Blaine Gardner 2021-07-06 18:07:55 UTC Flags needinfo?(mbukatov)
OpenShift BugZilla Robot 2021-07-06 19:49:57 UTC Status POST MODIFIED
Blaine Gardner 2021-07-06 20:56:30 UTC Flags needinfo?(mbukatov)
Mudit Agarwal 2021-07-07 12:29:45 UTC Status MODIFIED ON_QA
CC muagarwa
Fixed In Version 4.8.0-446.ci
Mudit Agarwal 2021-07-08 04:12:13 UTC Status ON_QA ASSIGNED
Mudit Agarwal 2021-07-08 04:12:32 UTC CC brgardne
Flags needinfo?(brgardne)
Sébastien Han 2021-07-08 08:24:54 UTC Flags needinfo?(mbukatov) needinfo?(mbukatov) needinfo?(mbukatov) needinfo?(brgardne)
Mudit Agarwal 2021-07-08 12:11:25 UTC Fixed In Version 4.8.0-446.ci
Doc Type If docs needed, set a value Known Issue
Flags needinfo?(brgardne)
RHEL Program Management 2021-07-08 12:11:33 UTC Target Release OCS 4.8.0 OCS 4.9.0
Mudit Agarwal 2021-07-08 12:15:06 UTC Blocks 1966894
Orit Wasserman 2021-07-08 14:09:09 UTC CC owasserm
Blaine Gardner 2021-07-08 16:40:14 UTC Doc Text In a storage cluster containing a CephCluster and one or more CephObjectStores, if the CephCluster resource is deleted before all of the CephObjectStore resources are fully deleted, the Rook Operator can still keep connection details about the CephObjectStore(s) in memory. If the same CephCluster and CephObjectStore(s) are re-created, the CephObjectStore(s) may enter "Failed" state.

To avoid this issue, it is recommended to delete the CephObjectStore(s) completely before removing the CephCluster.

If the user does not wish to wait for the CephObjectStore(s) to be deleted, restarting the Rook Operator (by deleting the Operator Pod) will avoid the issue if done after uninstall.

If a user is actively experiencing this issue, restarting the Rook Operator will resolve it by clearing the Operator's memory of old CephObjectStore connection details.
Flags needinfo?(brgardne)
Sébastien Han 2021-07-12 08:08:38 UTC Status ASSIGNED MODIFIED
Mudit Agarwal 2021-07-12 08:21:25 UTC Blocks 1981258
Olive Lakra 2021-07-28 05:30:07 UTC CC olakra
Doc Text In a storage cluster containing a CephCluster and one or more CephObjectStores, if the CephCluster resource is deleted before all of the CephObjectStore resources are fully deleted, the Rook Operator can still keep connection details about the CephObjectStore(s) in memory. If the same CephCluster and CephObjectStore(s) are re-created, the CephObjectStore(s) may enter "Failed" state.

To avoid this issue, it is recommended to delete the CephObjectStore(s) completely before removing the CephCluster.

If the user does not wish to wait for the CephObjectStore(s) to be deleted, restarting the Rook Operator (by deleting the Operator Pod) will avoid the issue if done after uninstall.

If a user is actively experiencing this issue, restarting the Rook Operator will resolve it by clearing the Operator's memory of old CephObjectStore connection details.
.Critical alert notification is sent after installation of the arbiter storage cluster when ceph object user for `cephobjectstore` failed to be created during storage cluster re-installation.
In a storage cluster containing a `CephCluster` and one or more `CephObjectStores`, if the `CephCluster` resource is deleted before all of the `CephObjectStore` resources are fully deleted, the Rook Operator can still keep connection details about the CephObjectStore(s) in memory. If the same `CephCluster` and CephObjectStore(s) are re-created, the CephObjectStore(s) may enter "Failed" state.

To avoid this issue, delete the CephObjectStore(s) completely before removing the `CephCluster`.

* If you do not wish to wait for the CephObjectStore(s) to be deleted, restarting the Rook Operator (by deleting the Operator Pod) will avoid the issue if done after uninstalling.

* If you are actively experiencing this issue, restarting the Rook Operator will resolve it by clearing the Operator's memory of old `CephObjectStore` connection details.
Flags needinfo?(muagarwa)
Mudit Agarwal 2021-07-28 05:42:57 UTC Flags needinfo?(muagarwa)
errata-xmlrpc 2021-08-12 13:04:14 UTC Status MODIFIED ON_QA
Martin Bukatovic 2021-08-19 08:45:51 UTC QA Contact ebenahar mbukatov
Elad 2021-08-25 09:24:45 UTC Keywords AutomationBackLog
Rejy M Cyriac 2021-09-26 22:18:43 UTC Target Release OCS 4.9.0 ---
Rejy M Cyriac 2021-09-26 22:20:59 UTC Component rook rook
Product Red Hat OpenShift Container Storage Red Hat OpenShift Data Foundation
RHEL Program Management 2021-09-26 22:23:05 UTC Target Release --- ODF 4.9.0
Olive Lakra 2021-10-05 05:14:53 UTC CC olakra
Mudit Agarwal 2021-11-03 04:24:50 UTC Doc Type Known Issue Bug Fix
Flags needinfo?(brgardne)
Mudit Agarwal 2021-11-03 04:25:02 UTC Blocks 2011326
Elad 2021-11-03 08:25:27 UTC Depends On 2005040
CC ebenahar
Blaine Gardner 2021-11-03 17:39:40 UTC Doc Text .Critical alert notification is sent after installation of the arbiter storage cluster when ceph object user for `cephobjectstore` failed to be created during storage cluster re-installation.
In a storage cluster containing a `CephCluster` and one or more `CephObjectStores`, if the `CephCluster` resource is deleted before all of the `CephObjectStore` resources are fully deleted, the Rook Operator can still keep connection details about the CephObjectStore(s) in memory. If the same `CephCluster` and CephObjectStore(s) are re-created, the CephObjectStore(s) may enter "Failed" state.

To avoid this issue, delete the CephObjectStore(s) completely before removing the `CephCluster`.

* If you do not wish to wait for the CephObjectStore(s) to be deleted, restarting the Rook Operator (by deleting the Operator Pod) will avoid the issue if done after uninstalling.

* If you are actively experiencing this issue, restarting the Rook Operator will resolve it by clearing the Operator's memory of old `CephObjectStore` connection details.
Fixed an issue where uninstalling and reinstalling the same CephCluster and CephObjectStore would result in the newest CephObjectStore not reaching "Ready" state.
Flags needinfo?(brgardne)
Elad 2021-11-04 07:46:25 UTC QA Contact mbukatov akrai
Harish NV Rao 2021-11-16 12:17:30 UTC Depends On 2021068
akarsha 2021-11-24 14:55:45 UTC Flags needinfo?(mbukatov) needinfo?(brgardne)
Elad 2021-11-25 10:36:11 UTC QA Contact akrai mbukatov
Martin Bukatovic 2021-11-25 23:48:24 UTC Flags needinfo?(mbukatov)
Blaine Gardner 2021-12-01 16:35:10 UTC Flags needinfo?(brgardne)
Martin Bukatovic 2021-12-02 13:39:45 UTC Status ON_QA ASSIGNED
Michael Adam 2021-12-02 18:33:26 UTC Keywords Regression
Flags needinfo?(ebenahar)
Elad 2021-12-02 20:13:50 UTC Flags needinfo?(ebenahar)
Elad 2021-12-02 20:16:26 UTC Flags needinfo?(brgardne)
Blaine Gardner 2021-12-02 21:51:55 UTC Flags needinfo?(brgardne) needinfo?(ebenahar)
Mudit Agarwal 2021-12-06 12:31:28 UTC Doc Type Bug Fix Known Issue
Flags needinfo?(ebenahar) needinfo?(brgardne)
Mudit Agarwal 2021-12-06 12:55:37 UTC Target Release ODF 4.9.0 ---
RHEL Program Management 2021-12-07 08:30:54 UTC Target Release --- ODF 4.10.0
Mudit Agarwal 2021-12-07 08:31:59 UTC Blocks 2029744
Blaine Gardner 2021-12-07 14:50:57 UTC Doc Text Fixed an issue where uninstalling and reinstalling the same CephCluster and CephObjectStore would result in the newest CephObjectStore not reaching "Ready" state. In a storage cluster containing a CephCluster and one or more CephObjectStores, if the CephCluster resource is deleted before all of the CephObjectStore resources are fully deleted, the Rook Operator can still keep connection details about the CephObjectStore(s) in memory. If the same CephCluster and CephObjectStore(s) are re-created, the CephObjectStore(s) may enter "Failed" state.

To avoid this issue, it is recommended to delete the CephObjectStore(s) completely before removing the CephCluster. If the user does not wish to wait for the CephObjectStore(s) to be deleted, restarting the Rook Operator (by deleting the Operator Pod) will avoid the issue if done after uninstall. If a user is actively experiencing this issue, restarting the Rook Operator will resolve it by clearing the Operator's memory of old CephObjectStore connection details.
Kusuma 2021-12-08 17:42:23 UTC Doc Text In a storage cluster containing a CephCluster and one or more CephObjectStores, if the CephCluster resource is deleted before all of the CephObjectStore resources are fully deleted, the Rook Operator can still keep connection details about the CephObjectStore(s) in memory. If the same CephCluster and CephObjectStore(s) are re-created, the CephObjectStore(s) may enter "Failed" state.

To avoid this issue, it is recommended to delete the CephObjectStore(s) completely before removing the CephCluster. If the user does not wish to wait for the CephObjectStore(s) to be deleted, restarting the Rook Operator (by deleting the Operator Pod) will avoid the issue if done after uninstall. If a user is actively experiencing this issue, restarting the Rook Operator will resolve it by clearing the Operator's memory of old CephObjectStore connection details.
.Critical alert notification is sent after installation of arbiter storage cluster, when Ceph object user for `cephobjectstore` fails to be created during storage cluster reinstallation

In a storage cluster containing a CephCluster and one or more `CephObjectStores`, if the `CephCluster` resource is deleted before all of the `CephObjectStore` resources are fully deleted, the Rook Operator can still keep connection details about the `CephObjectStores` in memory. If the same `CephCluster` and `CephObjectStores` are re-created, the `CephObjectStores` might enter `Failed` state.

To avoid this issue, you can delete the `CephObjectStores` completely before removing the CephCluster. If you do not want to wait for the CephObjectStores to be deleted, restart the Rook Operator (by deleting the Operator Pod) to avoid the issue if done after uninstall. If you are actively experiencing this issue, restart the Rook Operator to resolve it by clearing the Operator's memory of old CephObjectStore connection details.
CC kbg
Blaine Gardner 2021-12-13 17:37:40 UTC Flags needinfo?(brgardne)
Blaine Gardner 2021-12-13 17:40:35 UTC Flags needinfo?(muagarwa)
Doc Text .Critical alert notification is sent after installation of arbiter storage cluster, when Ceph object user for `cephobjectstore` fails to be created during storage cluster reinstallation

In a storage cluster containing a CephCluster and one or more `CephObjectStores`, if the `CephCluster` resource is deleted before all of the `CephObjectStore` resources are fully deleted, the Rook Operator can still keep connection details about the `CephObjectStores` in memory. If the same `CephCluster` and `CephObjectStores` are re-created, the `CephObjectStores` might enter `Failed` state.

To avoid this issue, you can delete the `CephObjectStores` completely before removing the CephCluster. If you do not want to wait for the CephObjectStores to be deleted, restart the Rook Operator (by deleting the Operator Pod) to avoid the issue if done after uninstall. If you are actively experiencing this issue, restart the Rook Operator to resolve it by clearing the Operator's memory of old CephObjectStore connection details.
Critical alert notification is sent after installation of arbiter storage cluster, when Ceph object user for `cephobjectstore` fails to be created during storage cluster reinstallation

In a storage cluster containing a CephCluster and one or more `CephObjectStores`, if the `CephCluster` resource is deleted before all of the `CephObjectStore` resources are fully deleted, the Rook Operator can still keep connection details about the `CephObjectStores` in memory. If the same `CephCluster` and `CephObjectStores` are re-created, the `CephObjectStores` might enter `Failed` state.

To avoid this issue, you can delete the `CephObjectStores` completely before removing the CephCluster. If you do not want to wait for the CephObjectStores to be deleted, restart the Rook Operator (by deleting the Operator Pod) to avoid the issue if done after uninstall. If you are actively experiencing this issue, restart the Rook Operator to resolve it by clearing the Operator's memory of old CephObjectStore connection details.
Kusuma 2021-12-13 18:01:20 UTC Doc Text Critical alert notification is sent after installation of arbiter storage cluster, when Ceph object user for `cephobjectstore` fails to be created during storage cluster reinstallation

In a storage cluster containing a CephCluster and one or more `CephObjectStores`, if the `CephCluster` resource is deleted before all of the `CephObjectStore` resources are fully deleted, the Rook Operator can still keep connection details about the `CephObjectStores` in memory. If the same `CephCluster` and `CephObjectStores` are re-created, the `CephObjectStores` might enter `Failed` state.

To avoid this issue, you can delete the `CephObjectStores` completely before removing the CephCluster. If you do not want to wait for the CephObjectStores to be deleted, restart the Rook Operator (by deleting the Operator Pod) to avoid the issue if done after uninstall. If you are actively experiencing this issue, restart the Rook Operator to resolve it by clearing the Operator's memory of old CephObjectStore connection details.
.Critical alert notification is sent after installation of arbiter storage cluster, when Ceph object user for `cephobjectstore` fails to be created during storage cluster reinstallation

In a storage cluster containing a CephCluster and one or more `CephObjectStores`, if the `CephCluster` resource is deleted before all of the `CephObjectStore` resources are fully deleted, the Rook Operator can still keep connection details about the `CephObjectStores` in memory. If the same `CephCluster` and `CephObjectStores` are re-created, the `CephObjectStores` might enter `Failed` state.

To avoid this issue, you can delete the `CephObjectStores` completely before removing the CephCluster. If you do not want to wait for the CephObjectStores to be deleted, restart the Rook Operator (by deleting the Operator Pod) to avoid the issue if done after uninstall. If you are actively experiencing this issue, restart the Rook Operator to resolve it by clearing the Operator's memory of old CephObjectStore connection details.
Blaine Gardner 2021-12-13 23:11:50 UTC Link ID Github rook/rook/pull/9417
Mudit Agarwal 2021-12-14 12:55:58 UTC Flags needinfo?(muagarwa)
Blaine Gardner 2021-12-17 18:27:12 UTC Status ASSIGNED MODIFIED
Link ID Github red-hat-storage/rook/pull/318
Red Hat Bugzilla 2022-01-10 10:25:22 UTC CC ratamir
Mudit Agarwal 2022-01-20 15:42:52 UTC Status MODIFIED ON_QA
Fixed In Version 4.10.0-113
Blaine Gardner 2022-01-27 16:48:16 UTC Flags needinfo?(mbukatov)
Martin Bukatovic 2022-02-17 18:57:35 UTC Flags needinfo?(mbukatov)
Martin Bukatovic 2022-02-17 19:34:30 UTC Status ON_QA VERIFIED
Mudit Agarwal 2022-03-04 12:15:58 UTC Flags needinfo?(brgardne)
Doc Text .Critical alert notification is sent after installation of arbiter storage cluster, when Ceph object user for `cephobjectstore` fails to be created during storage cluster reinstallation

In a storage cluster containing a CephCluster and one or more `CephObjectStores`, if the `CephCluster` resource is deleted before all of the `CephObjectStore` resources are fully deleted, the Rook Operator can still keep connection details about the `CephObjectStores` in memory. If the same `CephCluster` and `CephObjectStores` are re-created, the `CephObjectStores` might enter `Failed` state.

To avoid this issue, you can delete the `CephObjectStores` completely before removing the CephCluster. If you do not want to wait for the CephObjectStores to be deleted, restart the Rook Operator (by deleting the Operator Pod) to avoid the issue if done after uninstall. If you are actively experiencing this issue, restart the Rook Operator to resolve it by clearing the Operator's memory of old CephObjectStore connection details.
Doc Type Known Issue Bug Fix
Mudit Agarwal 2022-03-04 12:16:14 UTC Blocks 2056571
Blaine Gardner 2022-03-07 23:15:41 UTC Doc Text Fixed an issue where deleting a CephObjectStore and then creating a new CephObjectStore with the same name would result in the new store never reaching Ready state.
Flags needinfo?(brgardne)
Sonal 2022-03-31 11:24:02 UTC Flags needinfo?(mbukatov)
CC sarora
Shilpi Sharma 2022-04-05 10:01:42 UTC Flags needinfo?(brgardne)
CC shilpsha
Martin Bukatovic 2022-04-05 10:55:56 UTC Group redhat
Martin Bukatovic 2022-04-05 10:58:39 UTC Flags needinfo?(mbukatov)
Blaine Gardner 2022-04-05 14:56:02 UTC Flags needinfo?(brgardne)
Shilpi Sharma 2022-04-05 15:32:21 UTC Doc Text Fixed an issue where deleting a CephObjectStore and then creating a new CephObjectStore with the same name would result in the new store never reaching Ready state. .CephObjectStore is reaching to Ready state


Previously, there was an error with CephObjectStore where it would not attain the Ready state if it was deleted and then a new CephObjectStore was created with the same name.

With this update, any new CephObjectStore can be able to reach the Ready state even if a previously deleted CephObjectStore has the same name.
errata-xmlrpc 2022-04-13 15:06:15 UTC Status VERIFIED RELEASE_PENDING
errata-xmlrpc 2022-04-13 18:49:40 UTC Resolution --- ERRATA
Status RELEASE_PENDING CLOSED
Last Closed 2022-04-13 18:49:40 UTC
errata-xmlrpc 2022-04-13 18:50:11 UTC Link ID Red Hat Product Errata RHSA-2022:1372
Ramakrishnan Periyasamy 2022-08-17 10:01:24 UTC CC rperiyas
Elad 2023-08-09 17:03:01 UTC CC odf-bz-bot

Back to bug 1974344