Bug 1445891 - [RFE] Add custom user data support in bucket index
Summary: [RFE] Add custom user data support in bucket index
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Ceph Storage
Classification: Red Hat Storage
Component: RGW
Version: 2.2
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: rc
: 2.3
Assignee: Matt Benjamin (redhat)
QA Contact: Vasishta
Bara Ancincova
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-04-26 17:38 UTC by Benjamin Schmaus
Modified: 2020-08-13 09:07 UTC (History)
13 users (show)

Fixed In Version: RHEL: ceph-10.2.7-28.el7cp Ubuntu: ceph_10.2.7-30redhat1
Doc Type: Bug Fix
Doc Text:
The Ceph Object Gateway now supports a mechanism to associate opaque data to the object entries in internal bucket index metadata, and to recover this uninterpreted data in a custom HTTP header in bucket listing operations. This new feature enables users to store any user-defined data in the bucket index. Currently, interfaces to set and return user data are only available using the Swift access protocol.
Clone Of:
Environment:
Last Closed: 2017-07-17 17:06:06 UTC
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Ceph Project Bug Tracker 19644 0 None None None 2017-04-26 17:38:31 UTC
Red Hat Product Errata RHBA-2017:1755 0 normal SHIPPED_LIVE Red Hat Ceph Storage 2.3 bug fix and enhancement update 2017-07-17 21:05:50 UTC

Description Benjamin Schmaus 2017-04-26 17:38:31 UTC
Description of problem:

One of our big data use cases was to fetch all segments' size of a swift DLO manifest without having to go through each segment file. This feature addresses such requirements by having a 'user data' field in the bucket index structure 'rgw_bucket_dir_entry_meta', which would be populated via a custom http header passed in by the user and returned as appropriate.

This feature enables us to have a meta field (which is generic in nature to store any user defined data) in the bucket index. We are presently using this put/get all segments' size for a swift DLO object. Without this, to know the size of all segments that are part of swift DLO manifest, we have to loop through all segment parts and return the size as part of content length on the DLO manifest object. This operation was pretty heavy for our big data clients to perform, which we now solve by having this user defined field in the bucket index.


Version-Release number of selected component (if applicable):
2.2

How reproducible:
100%

Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

https://github.com/ceph/ceph/pull/14592

Comment 31 Matt Benjamin (redhat) 2017-07-11 15:41:55 UTC
looks great, thanks Bara

Comment 32 Vasishta 2017-07-13 08:47:50 UTC
Hi all,

Working fine for me, moving to VERIFIED state.

$ sudo radosgw-admin bucket list --bucket test_z --cluster 1 |grep user_data
        "user_data": "ABCD_MNOP"
        "user_data": "ABCD_1234"
        "user_data": "1234_5678"

Regards,
Vasishta

Comment 34 errata-xmlrpc 2017-07-17 17:06:06 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:1755


Note You need to log in before you can comment on or make changes to this bug.