Bug 2341711 - rgw: bucket logging fixes and enhancements
Summary: rgw: bucket logging fixes and enhancements
Keywords:
Status: VERIFIED
Alias: None
Product: Red Hat Ceph Storage
Classification: Red Hat Storage
Component: RGW
Version: 8.1
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
: 8.1
Assignee: Yuval Lifshitz
QA Contact: Hemanth Sai
Rivka Pollack
URL:
Whiteboard:
Depends On:
Blocks: 2351689
TreeView+ depends on / blocked
 
Reported: 2025-01-23 08:53 UTC by Yuval Lifshitz
Modified: 2025-06-16 12:14 UTC (History)
5 users (show)

Fixed In Version: ceph-19.2.1-3.el9cp
Doc Type: Enhancement
Doc Text:
.New fixes and enhancements for Ceph Object Gateway bucket logging This enhancement introduces several improvements to bucket logging, including support for source and destination buckets across different tenants, suffix/prefix-based key filtering, and standardized AWS operation names in log records. A new REST-based flush (POST) API has been added, along with the `bucket logging info admin` command for retrieving logging configurations. Fixes address concurrency issues causing multiple temporary objects, missing object size in certain cases, and retry attributes in race conditions. Additional safeguards now ensure that source and log buckets are distinct and that log buckets do not have encryption. Cleanup mechanisms have been improved to remove pending objects when source buckets are deleted, logging is disabled or reconfigured, or when target buckets are removed. Logging records now include missing fields related to authentication and transport layer information, ensuring more comprehensive logging capabilities.
Clone Of:
Environment:
Last Closed:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHCEPH-10483 0 None None None 2025-01-23 08:55:12 UTC

Description Yuval Lifshitz 2025-01-23 08:53:20 UTC
Description of problem:
in 8.0 we provided partial bucket logging support (see: https://bugzilla.redhat.com/show_bug.cgi?id=2308169), it was mising the following fixes and enhancements:
* add REST (POST) based flush API
* doc and examples fixes
* object size fix
* fix concurrency issue causing multiple temporary objects
* add suffix/prefix based key filtering
* using standard (AWS) operation names in logs records
* support source and destination buckets on different tenants
* verify source and log bucket must be different (original bz: https://bugzilla.redhat.com/show_bug.cgi?id=2321568)
* log bucket must not have encryption
* retry attribuite set in case of race
* clean pending objects
  * when source bucket is deleted
  * when logging is disabled
  * when logging conf changes
  * when target bucket is deleted
* add "bucket logging info" admin command
  * returning logging conf for source bucket
  * list of source buckets for log bucket
* add missing fields in lof records regarding authentication and transport layer info

Comment 1 Storage PM bot 2025-01-23 08:53:29 UTC
Please specify the severity of this bug. Severity is defined here:
https://bugzilla.redhat.com/page.cgi?id=fields.html#bug_severity.


Note You need to log in before you can comment on or make changes to this bug.