Bug 1781866 - Parallel querying of buckets.index listomapkeys
Summary: Parallel querying of buckets.index listomapkeys
Keywords:
Status: CLOSED DEFERRED
Alias: None
Product: Red Hat Ceph Storage
Classification: Red Hat Storage
Component: RADOS
Version: 3.3
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: ---
: 5.*
Assignee: Neha Ojha
QA Contact: Manohar Murthy
URL:
Whiteboard:
Depends On:
Blocks: 1723524 1727980
TreeView+ depends on / blocked
 
Reported: 2019-12-10 18:15 UTC by Tim Wilkinson
Modified: 2020-11-25 16:50 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-11-25 16:50:15 UTC
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Ceph Project Bug Tracker 48361 0 None None None 2020-11-25 16:50:13 UTC

Description Tim Wilkinson 2019-12-10 18:15:38 UTC
Description of problem:
----------------------
Querying listomapkeys for rgw.buckets.index takes 45 min in our test env (5 buckets * 521 shards). Each non-zero index shard key count is taking 4 or 5 sec to return. Hoping for a method of obtaining this debug info in a more timely manner.



Version-Release number of selected component (if applicable):
--------------------------------------------
ceph-radosgw-12.2.12-79



How reproducible:
----------------
consistent



Steps to Reproduce:
------------------
for i in `rados -p site1.rgw.buckets.index ls`; do echo -en ${i}": \t"; rados -p site1.rgw.buckets.index listomapkeys $i | wc -l; done



Actual results:
--------------
2605 shards respond in 43 minutes

real	42m52.516s
user	9m4.966s
sys	13m42.702s



Requested results:
-----------------
parallel queries to expedite the results

Comment 3 Jane smith 2020-11-04 08:05:30 UTC Comment hidden (spam)
Comment 4 Yaniv Kaul 2020-11-25 08:59:40 UTC
Any plans to address this? Otherwise, we can move it to upstream or close.

Comment 5 Neha Ojha 2020-11-25 16:50:15 UTC
(In reply to Yaniv Kaul from comment #4)
> Any plans to address this? Otherwise, we can move it to upstream or close.

Not in the immediate term. Sure, tracked in https://tracker.ceph.com/issues/48361


Note You need to log in before you can comment on or make changes to this bug.