Bug 1663570 - multisite sync errors from operations on a versioning-suspended bucket
Summary: multisite sync errors from operations on a versioning-suspended bucket
Alias: None
Product: Red Hat Ceph Storage
Classification: Red Hat
Component: RGW-Multisite
Version: 3.2
Hardware: Unspecified
OS: Unspecified
Target Milestone: z1
: 3.2
Assignee: Casey Bodley
QA Contact: Tejas
Bara Ancincova
Depends On:
Blocks: 1629656 1690927
TreeView+ depends on / blocked
Reported: 2019-01-04 21:50 UTC by Casey Bodley
Modified: 2019-03-20 13:35 UTC (History)
13 users (show)

Fixed In Version: RHEL: ceph-12.2.8-65.el7cp Ubuntu: ceph_12.2.8-52redhat1xenial
Doc Type: Bug Fix
Doc Text:
.Objects are now synced correctly in versioning-suspended buckets Due to a bug in multi-site sync of versioning-suspended buckets, certain object versioning attributes were overwritten with incorrect values. Consequently, the objects failed to sync and attempted to retry endlessly, blocking further sync progress. With this update, the sync process no longer overwrites versioning attributes. In addition, any broken attributes are now detected and repaired. As a result, objects are synced correctly in versioning-suspended buckets.
Clone Of:
: 1690927 (view as bug list)
Last Closed: 2019-03-07 15:51:27 UTC

Attachments (Terms of Use)

System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2019:0475 None None None 2019-03-07 15:51:36 UTC
Github ceph ceph pull 25794 None None None 2019-01-04 21:53:46 UTC
Github ceph ceph pull 26157 None None None 2019-01-26 20:23:19 UTC
Ceph Project Bug Tracker 37792 None None None 2019-01-04 21:50:35 UTC

Description Casey Bodley 2019-01-04 21:50:35 UTC
Description of problem:

Overwrites in a versioning-suspended bucket will fail to sync with errors like "cls_rgw_bucket_link_olh() returned r=-125" and block further progress on the bucket.

How reproducible: 100%

Steps to Reproduce:

1. deploy a two-zone multisite configuration
2. on the master zone, create a bucket
3. upload an object "obj"
4. enable versioning on the bucket
5. reupload the same object "obj"
6. suspend versioning on the bucket
7. reupload the same object "obj"

Actual results:

Multisite sync on the secondary zone gets stuck attempting to sync the third upload, producing radosgw log errors like "cls_rgw_bucket_link_olh() returned r=-125" and osd log errors like "NOTICE: op.olh_tag (zxopy27aag3jjr38ddtow7517gdpgz4c) != olh.tag (bne5h7ou7gingobf89ae5crr2p3p284y)".

Expected results:

Sync in versioning-suspended buckets should succeed and converge on the same objects/versions in both zones.

Comment 38 errata-xmlrpc 2019-03-07 15:51:27 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.