Bug 1701970

Summary: Inefficient unordered bucket listing
Product: [Red Hat Storage] Red Hat Ceph Storage Reporter: Casey Bodley <cbodley>
Component: RGWAssignee: Matt Benjamin (redhat) <mbenjamin>
Status: CLOSED ERRATA QA Contact: ceph-qe-bugs <ceph-qe-bugs>
Severity: low Docs Contact:
Priority: high    
Version: 3.2CC: assingh, cbodley, ceph-eng-bugs, ceph-qe-bugs, kbader, mbenjamin, sweil, tchandra, tserlin, vimishra
Target Milestone: z2   
Target Release: 3.2   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: RHEL: ceph-12.2.8-120.el7cp Ubuntu: ceph_12.2.8-103redhat1 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-04-30 15:57:08 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Casey Bodley 2019-04-22 15:16:21 UTC 
Description of problem:

A bug in unordered bucket listing causes it to scale poorly with the number of bucket shards N, issuing N^2 osd requests instead of N.

Version-Release number of selected component (if applicable):


How reproducible:

On buckets with more than one shard.


Steps to Reproduce:

1. Configure radosgw with rgw_override_bucket_index_max_shards=8 and debug_ms=1, restart radosgw

2. Create a bucket with public access:
$ s3cmd mb s3://bucket --acl-public

3. List the bucket once with the 'allow-unordered' parameter:
$ curl http://[hostname:port]/bucket?allow-unordered=1

4. In the radosgw log, count the number of [call rgw.bucket_list] requests sent to osds:
$ grep --count 'call rgw.bucket_list' /path/to/radosgw.log


Actual results:

64

Expected results:

8

Additional info:
Comment 9 errata-xmlrpc 2019-04-30 15:57:08 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2019:0911