Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
This project is now read‑only. Starting Monday, February 2, please use https://ibm-ceph.atlassian.net/ for all bug tracking management.

Bug 1445891

Summary: [RFE] Add custom user data support in bucket index
Product: [Red Hat Storage] Red Hat Ceph Storage Reporter: Benjamin Schmaus <bschmaus>
Component: RGWAssignee: Matt Benjamin (redhat) <mbenjamin>
Status: CLOSED ERRATA QA Contact: Vasishta <vashastr>
Severity: high Docs Contact: Bara Ancincova <bancinco>
Priority: unspecified    
Version: 2.2CC: cbodley, ceph-eng-bugs, kbader, kdreyer, mbenjamin, mhackett, owasserm, sweil, tserlin, uboppana, vakulkar, vashastr, vumrao
Target Milestone: rcKeywords: FutureFeature
Target Release: 2.3   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: RHEL: ceph-10.2.7-28.el7cp Ubuntu: ceph_10.2.7-30redhat1 Doc Type: Bug Fix
Doc Text:
The Ceph Object Gateway now supports a mechanism to associate opaque data to the object entries in internal bucket index metadata, and to recover this uninterpreted data in a custom HTTP header in bucket listing operations. This new feature enables users to store any user-defined data in the bucket index. Currently, interfaces to set and return user data are only available using the Swift access protocol.
 Story Points: ---
Clone Of: Environment:
Last Closed: 2017-07-17 17:06:06 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Benjamin Schmaus 2017-04-26 17:38:31 UTC 
Description of problem:

One of our big data use cases was to fetch all segments' size of a swift DLO manifest without having to go through each segment file. This feature addresses such requirements by having a 'user data' field in the bucket index structure 'rgw_bucket_dir_entry_meta', which would be populated via a custom http header passed in by the user and returned as appropriate.

This feature enables us to have a meta field (which is generic in nature to store any user defined data) in the bucket index. We are presently using this put/get all segments' size for a swift DLO object. Without this, to know the size of all segments that are part of swift DLO manifest, we have to loop through all segment parts and return the size as part of content length on the DLO manifest object. This operation was pretty heavy for our big data clients to perform, which we now solve by having this user defined field in the bucket index.


Version-Release number of selected component (if applicable):
2.2

How reproducible:
100%

Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

https://github.com/ceph/ceph/pull/14592
Comment 31 Matt Benjamin (redhat) 2017-07-11 15:41:55 UTC 
looks great, thanks Bara
Comment 32 Vasishta 2017-07-13 08:47:50 UTC 
Hi all,

Working fine for me, moving to VERIFIED state.

$ sudo radosgw-admin bucket list --bucket test_z --cluster 1 |grep user_data
        "user_data": "ABCD_MNOP"
        "user_data": "ABCD_1234"
        "user_data": "1234_5678"

Regards,
Vasishta
Comment 34 errata-xmlrpc 2017-07-17 17:06:06 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:1755