Bug 1701970 - Inefficient unordered bucket listing
Summary: Inefficient unordered bucket listing
Alias: None
Product: Red Hat Ceph Storage
Classification: Red Hat
Component: RGW
Version: 3.2
Hardware: Unspecified
OS: Unspecified
Target Milestone: z2
: 3.2
Assignee: Matt Benjamin (redhat)
QA Contact: ceph-qe-bugs
Depends On:
TreeView+ depends on / blocked
Reported: 2019-04-22 15:16 UTC by Casey Bodley
Modified: 2019-04-30 15:57 UTC (History)
10 users (show)

Fixed In Version: RHEL: ceph-12.2.8-120.el7cp Ubuntu: ceph_12.2.8-103redhat1
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Last Closed: 2019-04-30 15:57:08 UTC
Target Upstream Version:

Attachments (Terms of Use)

System ID Priority Status Summary Last Updated
Ceph Project Bug Tracker 39393 None None None 2019-04-22 15:16:21 UTC
Red Hat Product Errata RHSA-2019:0911 None None None 2019-04-30 15:57:23 UTC

Description Casey Bodley 2019-04-22 15:16:21 UTC
Description of problem:

A bug in unordered bucket listing causes it to scale poorly with the number of bucket shards N, issuing N^2 osd requests instead of N.

Version-Release number of selected component (if applicable):

How reproducible:

On buckets with more than one shard.

Steps to Reproduce:

1. Configure radosgw with rgw_override_bucket_index_max_shards=8 and debug_ms=1, restart radosgw

2. Create a bucket with public access:
$ s3cmd mb s3://bucket --acl-public

3. List the bucket once with the 'allow-unordered' parameter:
$ curl http://[hostname:port]/bucket?allow-unordered=1

4. In the radosgw log, count the number of [call rgw.bucket_list] requests sent to osds:
$ grep --count 'call rgw.bucket_list' /path/to/radosgw.log

Actual results:


Expected results:


Additional info:

Comment 9 errata-xmlrpc 2019-04-30 15:57:08 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.