Bug 2102358 - Setting rgw_data_notify_interval_msec=0 dos not disable async data notifications
Summary: Setting rgw_data_notify_interval_msec=0 dos not disable async data notifications
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Ceph Storage
Classification: Red Hat Storage
Component: RGW-Multisite
Version: 4.1
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 4.3z1
Assignee: Adam C. Emerson
QA Contact: Tejas
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-06-29 18:59 UTC by Adam C. Emerson
Modified: 2022-09-22 11:21 UTC (History)
6 users (show)

Fixed In Version: ceph-14.2.22-117.el8cp, ceph-14.2.22-117.el7cp
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-09-22 11:21:10 UTC
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Ceph Project Bug Tracker 49723 0 None None None 2022-06-29 18:59:15 UTC
Github ceph ceph pull 40011 0 None Merged rgw: allow rgw_data_notify_interval_msec=0 to disable notifications 2022-06-29 18:59:15 UTC
Github https://github.com/red-hat-storage ceph-qe-scripts pull 293/ 0 None None None 2022-08-02 12:31:42 UTC
Github https://github.com/red-hat-storage cephci pull 1687 0 None None None 2022-08-02 12:29:29 UTC
Red Hat Issue Tracker RHCEPH-4639 0 None None None 2022-06-29 19:19:42 UTC
Red Hat Product Errata RHBA-2022:6684 0 None None None 2022-09-22 11:21:37 UTC

Description Adam C. Emerson 2022-06-29 18:59:15 UTC
Description of problem:

In upstream Ceph we have found that async data notifications solve sync consistency problems. In downstream Ceph this seemed to have no effect or made it worse.

We recently discovered a patch that was originally introduced as part of the testing framework is required to make disabling async data notifications possible. Without it, attempting to disable them just resulted in more of them being sent more frequently.

Version-Release number of selected component (if applicable):

All 4.x and 5.x

How reproducible:

Deterministically.

Steps to Reproduce:
1. Set up a multisite cluster 
2. Set rgw_data_notify_interval_msec to 0
3. Ingest data into it

Actual results:

No async data notifications.

At level 20, the log does not show 'notifying datalog change' entries

Expected results:

Oh so many async data notifications.


At level 20, the log DOES show 'notifying datalog change'

Additional info:

As async data log notifications may be the ultimate root of sync inconsistency errors we've been seeing, this needs to be fixed so we can test disabling them properly.

Comment 10 errata-xmlrpc 2022-09-22 11:21:10 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat Ceph Storage 4.3 Bug Fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:6684


Note You need to log in before you can comment on or make changes to this bug.