Bug 2341711 - rgw: bucket logging fixes and enhancements
Summary: rgw: bucket logging fixes and enhancements
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Ceph Storage
Classification: Red Hat Storage
Component: RGW
Version: 8.1
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
: 8.1
Assignee: Yuval Lifshitz
QA Contact: Hemanth Sai
Rivka Pollack
URL:
Whiteboard:
Depends On:
Blocks: 2351689
TreeView+ depends on / blocked
 
Reported: 2025-01-23 08:53 UTC by Yuval Lifshitz
Modified: 2025-08-07 15:40 UTC (History)
5 users (show)

Fixed In Version: ceph-19.2.1-3.el9cp
Doc Type: Technology Preview
Doc Text:
.Bucket logging support for Ceph Object Gateway with bug fixes and enhancements Bucket logging was introduced in Red Hat Ceph Storage 8.0. Bucket logging provides a mechanism for logging all access to a bucket. The log data can be used to monitor bucket activity, detect unauthorized access, get insights into the bucket usage and use the logs as a journal for bucket changes. The log records are stored in objects in a separate bucket and can be analyzed later. Logging configuration is done at the bucket level and can be enabled or disabled at any time. The log bucket can accumulate logs from multiple buckets. The configured `prefix` may be used to distinguish between logs from different buckets. For performance reasons, even though the log records are written to persistent storage, the log object appears in the log bucket only after a configurable amount of time or when reaching the maximum object size of 128 MB. Adding a log object to the log bucket is done in such a way that if no more records are written to the object, it might remain outside of the log bucket even after the configured time has passed. There are two logging types: `standard` and `journal`. The default logging type is `standard`. When set to `standard` the log records are written to the log bucket after the bucket operation is completed. As a result the logging operation can fail with no indication to the client. When set to `journal` the records are written to the log bucket before the bucket operation is complete. As a result, the operation does not run if the logging action fails and an error is returned to the client. You can complete the following bucket logging actions: enable, disable, and get. Red Hat Ceph Storage 8.1 enhancements introduce several improvements to bucket logging, including support for source and destination buckets across different tenants, suffix/prefix-based key filtering, and standardized AWS operation names in log records. A new REST-based flush (POST) API has been added, along with the `bucket logging info admin` command for retrieving logging configurations. Fixes address concurrency issues causing multiple temporary objects, missing object size in certain cases, and retry attributes in race conditions. Additional safeguards now ensure that source and log buckets are distinct and that log buckets do not have encryption. Cleanup mechanisms have been improved to remove pending objects when source buckets are deleted, logging is disabled or reconfigured, or when target buckets are removed. Logging records now include missing fields related to authentication and transport layer information, ensuring more comprehensive logging capabilities.
Clone Of:
Environment:
Last Closed: 2025-06-26 12:24:09 UTC
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHCEPH-10483 0 None None None 2025-01-23 08:55:12 UTC
Red Hat Product Errata RHSA-2025:9775 0 None None None 2025-06-26 12:24:17 UTC

Description Yuval Lifshitz 2025-01-23 08:53:20 UTC
Description of problem:
in 8.0 we provided partial bucket logging support (see: https://bugzilla.redhat.com/show_bug.cgi?id=2308169), it was mising the following fixes and enhancements:
* add REST (POST) based flush API
* doc and examples fixes
* object size fix
* fix concurrency issue causing multiple temporary objects
* add suffix/prefix based key filtering
* using standard (AWS) operation names in logs records
* support source and destination buckets on different tenants
* verify source and log bucket must be different (original bz: https://bugzilla.redhat.com/show_bug.cgi?id=2321568)
* log bucket must not have encryption
* retry attribuite set in case of race
* clean pending objects
  * when source bucket is deleted
  * when logging is disabled
  * when logging conf changes
  * when target bucket is deleted
* add "bucket logging info" admin command
  * returning logging conf for source bucket
  * list of source buckets for log bucket
* add missing fields in lof records regarding authentication and transport layer info

Comment 1 Storage PM bot 2025-01-23 08:53:29 UTC
Please specify the severity of this bug. Severity is defined here:
https://bugzilla.redhat.com/page.cgi?id=fields.html#bug_severity.

Comment 7 errata-xmlrpc 2025-06-26 12:24:09 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Red Hat Ceph Storage 8.1 security, bug fix, and enhancement updates), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2025:9775


Note You need to log in before you can comment on or make changes to this bug.