Bug 2186482 - [GSS] [4.12 backport] Object storage in degraded state
Summary: [GSS] [4.12 backport] Object storage in degraded state
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenShift Data Foundation
Classification: Red Hat Storage
Component: Multi-Cloud Object Gateway
Version: 4.11
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: ODF 4.12.3
Assignee: Utkarsh Srivastava
QA Contact: Tiffany Nguyen
URL:
Whiteboard:
Depends On: 2181535
Blocks: 2186485
TreeView+ depends on / blocked
 
Reported: 2023-04-13 12:57 UTC by Bipin Kunal
Modified: 2023-08-09 16:49 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Previously, non optimized database related flows on deletions caused Multicloud Object Gateway to spike in CPU usage and perform slowly on mass delete scenarios. For example, reclaiming a deleted object bucket claim (OBC). With this fix, indexes for the bucket reclaimer process are optimized, a new index is added to the database to speed up the database cleaner flows, and bucket reclaimer changes are introduced to work on batches of objects.
Clone Of:
Environment:
Last Closed: 2023-05-23 09:17:30 UTC
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github noobaa noobaa-core pull 7262 0 None Merged Optimize NooBaa deletion paths 2023-04-24 06:02:11 UTC
Github noobaa noobaa-core pull 7280 0 None Merged [Backport to 5.12] improve performance and fix lifecycle BG 2023-05-02 07:17:32 UTC
Red Hat Product Errata RHSA-2023:3265 0 None None None 2023-05-23 09:17:46 UTC

Comment 7 Tiffany Nguyen 2023-05-11 00:30:50 UTC 
Verified this issue using ODF 4.12.3-16.  With noobaa pods resources as below, CPU usage of noobaa-core and noobaa-db didn't reach over 100% when upload and list 1M objects.  Backingstore is working as expected, obc and backingstore are deleting without any issue.

$oc get storagecluster -n openshift-storage ocs-storagecluster -oyaml | yq '.spec.resources'

mgr:
  limits:
    cpu: "3"
    memory: 3Gi
  requests:
    cpu: "3"
    memory: 3Gi
noobaa-core:
  limits:
    cpu: "3"
    memory: 4Gi
  requests:
    cpu: "3"
    memory: 4Gi
noobaa-db:
  limits:
    cpu: "3"
    memory: 4Gi
  requests:
    cpu: "3"
    memory: 4Gi
noobaa-endpoint:
  limits:
    cpu: "3"
    memory: 4Gi
  requests:
    cpu: "3"
    memory: 4Gi
Comment 13 errata-xmlrpc 2023-05-23 09:17:30 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: Red Hat OpenShift Data Foundation 4.12.3 Security and Bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2023:3265
Note You need to log in before you can comment on or make changes to this bug.