Bug 1391650 – [RFE] [RGW] radosgw-admin bucket quota warning

Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.

Bug 1391650 - [RFE] [RGW] radosgw-admin bucket quota warning

Summary:	[RFE] [RGW] radosgw-admin bucket quota warning

Status:	CLOSED ERRATA

Aliases:	None

Product:	Red Hat Ceph Storage
Classification:	Red Hat
Component:	RGW (Show other bugs)
Sub Component:
Version:	2.1
Hardware:	All All

Priority	unspecified Severity high
Target Milestone:	rc
Target Release:	2.3
Assigned To:	Matt Benjamin (redhat)
QA Contact:	Parikshith
Docs Contact:	Erin Donnelly

URL:
Whiteboard:
Keywords:	FutureFeature

Depends On:
Blocks:	1437916
	Show dependency tree / graph

Reported:	2016-11-03 12:43 EDT by Matt Benjamin (redhat)
Modified:	2017-07-30 12:04 EDT (History)
CC List:	12 users (show)

See Also:
Fixed In Version:	RHEL: ceph-10.2.7-22.el7cp Ubuntu: ceph_10.2.7-24redhat1
Doc Type:	Enhancement
Doc Text:	.The radosgw-admin utility supports a new bucket limitcheck command The `radosgw-admin` utility has a new command `bucket limitcheck` to warn the administrator when a bucket needs resharding. Previously, buckets with more objects than is recommended could be unnoticed and cause performance issues. This new command reports on bucket status with respect to the configured bucket sharding recommendations ensuring that administrators can detect overloaded buckets easily.
Story Points:	---
Clone Of:
Environment:
Last Closed:	2017-06-19 09:27:19 EDT
Type:	Bug
Regression:	---
Mount Type:	---
Documentation:	---
CRM:
Verified Versions:
Category:	---
oVirt Team:	---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---

Attachments	(Terms of Use)
Add an attachment (proposed patch, testcase, etc.)

External Trackers
Tracker	ID	Priority	Status	Summary	Last Updated
Ceph Project Bug Tracker	17925	None	None	None	2017-01-30 15:37 EST
Red Hat Product Errata	RHBA-2017:1497	normal	SHIPPED_LIVE	Red Hat Ceph Storage 2.3 bug fix and enhancement update	2017-06-19 13:24:11 EDT

Groups: None (edit)

Description Matt Benjamin (redhat) 2016-11-03 12:43:06 EDT

Description of problem:

RGW should generate warnings when buckets are in danger exceeding the same maximum number of entries per index shard (appx. 100000, for buckets with indexes)

Version-Release number of selected component (if applicable):
2.1

Actual results:
No comparable feature exists.


Expected results:
We propose to address the issue by providing a new radosgw-admin sub-command which scans existing, indexed buckets and reports on those which have exceeded or have approached within a defined percentage (default 10%) of the safe maximum entry limit of 100K per index shard.

Additional info:
None

Comment 6 Matt Benjamin (redhat) 2016-11-05 16:49:05 EDT

Proposed upstream PR:
https://github.com/ceph/ceph/pull/11796

Comment 11 Ken Dreyer (Red Hat) 2017-04-07 14:18:09 EDT

Matt would you please rebase the PR at https://github.com/ceph/ceph/pull/11796 so we can get this upstream?

Comment 12 Matt Benjamin (redhat) 2017-04-19 16:08:34 EDT

(In reply to Ken Dreyer (Red Hat) from comment #11)
> Matt would you please rebase the PR at
> https://github.com/ceph/ceph/pull/11796 so we can get this upstream?

@ken, rebased last week, I asked Casey to re-review.

Matt

Comment 34 Parikshith 2017-06-01 02:08:37 EDT

Made the changes. Verified with 5 different buckets with varying objects.
Warning messages are displayed as per the configuration set.

rgw_override_bucket_index_max_shards = 10 
rgw_safe_max_objects_per_shard = 1000

radosgw-admin bucket limit check --warnings-only
[
    {
        "user_id": "johndoe",
        "buckets": [
            {
                "bucket": "general_2_3_10000",
                "tenant": "",
                "num_objects": 10000,
                "num_shards": 10,
                "objects_per_shard": 1000,
                "fill_status": "WARN 100.000000%"
            },
            {
                "bucket": "general_2_3_20000",
                "tenant": "",
                "num_objects": 20000,
                "num_shards": 10,
                "objects_per_shard": 2000,
                "fill_status": "OVER 100.000000%"
            }
        ]
    }
]

Comment 36 errata-xmlrpc 2017-06-19 09:27:19 EDT

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:1497

Note You need to log in before you can comment on or make changes to this bug.