Bug 1701970 - Inefficient unordered bucket listing
Summary: Inefficient unordered bucket listing
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Ceph Storage
Classification: Red Hat
Component: RGW
Version: 3.2
Hardware: Unspecified
OS: Unspecified
high
low
Target Milestone: z2
: 3.2
Assignee: Matt Benjamin (redhat)
QA Contact: ceph-qe-bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-04-22 15:16 UTC by Casey Bodley
Modified: 2019-04-30 15:57 UTC (History)
10 users (show)

Fixed In Version: RHEL: ceph-12.2.8-120.el7cp Ubuntu: ceph_12.2.8-103redhat1
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-04-30 15:57:08 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Ceph Project Bug Tracker 39393 None None None 2019-04-22 15:16:21 UTC
Red Hat Product Errata RHSA-2019:0911 None None None 2019-04-30 15:57:23 UTC

Description Casey Bodley 2019-04-22 15:16:21 UTC
Description of problem:

A bug in unordered bucket listing causes it to scale poorly with the number of bucket shards N, issuing N^2 osd requests instead of N.

Version-Release number of selected component (if applicable):


How reproducible:

On buckets with more than one shard.


Steps to Reproduce:

1. Configure radosgw with rgw_override_bucket_index_max_shards=8 and debug_ms=1, restart radosgw

2. Create a bucket with public access:
$ s3cmd mb s3://bucket --acl-public

3. List the bucket once with the 'allow-unordered' parameter:
$ curl http://[hostname:port]/bucket?allow-unordered=1

4. In the radosgw log, count the number of [call rgw.bucket_list] requests sent to osds:
$ grep --count 'call rgw.bucket_list' /path/to/radosgw.log


Actual results:

64

Expected results:

8

Additional info:

Comment 9 errata-xmlrpc 2019-04-30 15:57:08 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2019:0911


Note You need to log in before you can comment on or make changes to this bug.