Bug 2241146

Summary:	[7.0][rgw-archive]: On archive zone, bucket versioning shows disabled on 7.0 even though it is enabled
Product:	[Red Hat Storage] Red Hat Ceph Storage	Reporter:	Vidushi Mishra <vimishra>
Component:	RGW-Multisite	Assignee:	shilpa <smanjara>
Status:	CLOSED ERRATA	QA Contact:	Vidushi Mishra <vimishra>
Severity:	medium	Docs Contact:	Rivka Pollack <rpollack>
Priority:	unspecified
Version:	7.0	CC:	akraj, ceph-eng-bugs, cephqe-warriors, mbenjamin, smanjara, tserlin
Target Milestone:	---
Target Release:	7.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	ceph-18.2.0-98.el9cp	Doc Type:	No Doc Update
Doc Text:		Story Points:	---
Clone Of:
Clones:	2256172 (view as bug list)		Environment:
Last Closed:	2023-12-13 15:23:48 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	2237662, 2256172, 2257221

Description Vidushi Mishra 2023-09-28 09:05:30 UTC

Description of problem:


On the archive zone, bucket versioning shows disabled on 7.0 even though it is enabled

Version-Release number of selected component (if applicable):

ceph version 18.2.0-56.el9cp (186d946288ce5dc6d8f6c33174852688b767b521) reef (stable

How reproducible:

2/2

Steps to Reproduce:

1. On a multisite + archive environment, create a bucket 'test-1' on the primary zone.
[root@ceph-pri-archies-8k924y-node6 ~]# s3cmd mb s3://test-1
Bucket 's3://test-1/' created

2. Upload an object with the name 'obj-4k' to the bucket from the primary zone, and overwrite it a few more times from the secondary zone.

[root@ceph-pri-archies-8k924y-node6 ~]# s3cmd put obj-4k s3://test-1/
upload: 'obj-4k' -> 's3://test-1/obj-4k'  [1 of 1]
 4096 of 4096   100% in    0s   137.78 KB/s  done

3. Test the versioning status of the bucket from all the zones.

Actual results:

- Bucket Versioning shows disabled in the archive zone even though the bucket list shows object versions (for obj-4k overwrite.)


Expected results:

Versioning status should be enabled at the archive zone.

Additional info:

Comment 1 Vidushi Mishra 2023-09-28 09:20:02 UTC

bucket list at the primary 
===========================

[root@ceph-pri-archies-8k924y-node6 ~]# radosgw-admin bucket list  --bucket test-1
[
    {
        "name": "obj-4k",
        "instance": "",
        "ver": {
            "pool": 11,
            "epoch": 577
        },
        "locator": "",
        "exists": true,
        "meta": {
            "category": 1,
            "size": 4096,
            "mtime": "2023-09-28T06:53:38.869223Z",
            "etag": "620f0b67a91f7f74151bc5be745b7110",
            "storage_class": "STANDARD",
            "owner": "user1",
            "owner_display_name": "user1",
            "content_type": "application/octet-stream",
            "accounted_size": 4096,
            "user_data": "",
            "appendable": false
        },
        "tag": "98cc81b5-8deb-4296-9af5-645aab0beefe.44553.15460892259630159912",
        "flags": 0,
        "pending_map": [],
        "versioned_epoch": 0
    }
]
[root@ceph-pri-archies-8k924y-node6 ~]# 

[root@ceph-pri-archies-8k924y-node6 ~]#  radosgw-admin bucket stats --bucket test-1 | egrep 'num_objects|versioning|num_shards'
    "num_shards": 11,
    "versioning": "off",
            "num_objects": 1
[root@ceph-pri-archies-8k924y-node6 ~]#


===================================
bucket list at the archive zone 
===================================

[root@ceph-arc-archies-8k924y-node6 ~]# aws s3api get-bucket-versioning --bucket test-1 --endpoint http://ceph-arc-archies-8k924y-node6:5000
[root@ceph-arc-archies-8k924y-node6 ~]# aws --endpoint http://ceph-arc-archies-8k924y-node6:5000 s3api list-object-versions --bucket test-1
{
    "Versions": [
        {
            "ETag": "\"10121fbb388802a9553a9ba76e5e2bec\"",
            "Size": 24,
            "StorageClass": "STANDARD",
            "Key": "obj-4k",
            "VersionId": "I0Qhfif91hgeRt-r25pJ0TZOi4fJPCm",
            "IsLatest": true,
            "LastModified": "2023-09-28T07:29:31.647Z",
            "Owner": {
                "DisplayName": "user1",
                "ID": "user1"
            }
        },
        {
            "ETag": "\"620f0b67a91f7f74151bc5be745b7110\"",
            "Size": 4096,
            "StorageClass": "STANDARD",
            "Key": "obj-4k",
            "VersionId": "WSDCgxa9r3fADUWZ7ZzJ3aXOMi.NUlN",
            "IsLatest": false,
            "LastModified": "2023-09-28T07:27:32.352Z",
            "Owner": {
                "DisplayName": "user1",
                "ID": "user1"
            }
        },
        {
            "ETag": "\"620f0b67a91f7f74151bc5be745b7110\"",
            "Size": 4096,
            "StorageClass": "STANDARD",
            "Key": "obj-4k",
            "VersionId": "OG1zvrWAKlFlfX.fB0Q4pGyAtYp5ba8",
            "IsLatest": false,
            "LastModified": "2023-09-28T06:53:38.869Z",
            "Owner": {
                "DisplayName": "user1",
                "ID": "user1"
            }
        },
        {
            "ETag": "\"620f0b67a91f7f74151bc5be745b7110\"",
            "Size": 4096,
            "StorageClass": "STANDARD",
            "Key": "obj-4k",
            "VersionId": "xYf4wdcsFEiA4lzHzUe.EDWCpJwr9ba",
            "IsLatest": false,
            "LastModified": "2023-09-28T06:43:20.791Z",
            "Owner": {
                "DisplayName": "user1",
                "ID": "user1"
            }
        }
    ],
    "RequestCharged": null
}
[root@ceph-arc-archies-8k924y-node6 ~]# radosgw-admin bucket stats --bucket test-1 | egrep 'num_objects|versioning|num_shards'
    "num_shards": 11,
    "versioning": "off",
            "num_objects": 4
[root@ceph-arc-archies-8k924y-node6 ~]#

Comment 2 Vidushi Mishra 2023-09-28 09:20:21 UTC

bucket list at the primary 
===========================

[root@ceph-pri-archies-8k924y-node6 ~]# radosgw-admin bucket list  --bucket test-1
[
    {
        "name": "obj-4k",
        "instance": "",
        "ver": {
            "pool": 11,
            "epoch": 577
        },
        "locator": "",
        "exists": true,
        "meta": {
            "category": 1,
            "size": 4096,
            "mtime": "2023-09-28T06:53:38.869223Z",
            "etag": "620f0b67a91f7f74151bc5be745b7110",
            "storage_class": "STANDARD",
            "owner": "user1",
            "owner_display_name": "user1",
            "content_type": "application/octet-stream",
            "accounted_size": 4096,
            "user_data": "",
            "appendable": false
        },
        "tag": "98cc81b5-8deb-4296-9af5-645aab0beefe.44553.15460892259630159912",
        "flags": 0,
        "pending_map": [],
        "versioned_epoch": 0
    }
]
[root@ceph-pri-archies-8k924y-node6 ~]# 

[root@ceph-pri-archies-8k924y-node6 ~]#  radosgw-admin bucket stats --bucket test-1 | egrep 'num_objects|versioning|num_shards'
    "num_shards": 11,
    "versioning": "off",
            "num_objects": 1
[root@ceph-pri-archies-8k924y-node6 ~]#


===================================
bucket list at the archive zone 
===================================

[root@ceph-arc-archies-8k924y-node6 ~]# aws s3api get-bucket-versioning --bucket test-1 --endpoint http://ceph-arc-archies-8k924y-node6:5000
[root@ceph-arc-archies-8k924y-node6 ~]# aws --endpoint http://ceph-arc-archies-8k924y-node6:5000 s3api list-object-versions --bucket test-1
{
    "Versions": [
        {
            "ETag": "\"10121fbb388802a9553a9ba76e5e2bec\"",
            "Size": 24,
            "StorageClass": "STANDARD",
            "Key": "obj-4k",
            "VersionId": "I0Qhfif91hgeRt-r25pJ0TZOi4fJPCm",
            "IsLatest": true,
            "LastModified": "2023-09-28T07:29:31.647Z",
            "Owner": {
                "DisplayName": "user1",
                "ID": "user1"
            }
        },
        {
            "ETag": "\"620f0b67a91f7f74151bc5be745b7110\"",
            "Size": 4096,
            "StorageClass": "STANDARD",
            "Key": "obj-4k",
            "VersionId": "WSDCgxa9r3fADUWZ7ZzJ3aXOMi.NUlN",
            "IsLatest": false,
            "LastModified": "2023-09-28T07:27:32.352Z",
            "Owner": {
                "DisplayName": "user1",
                "ID": "user1"
            }
        },
        {
            "ETag": "\"620f0b67a91f7f74151bc5be745b7110\"",
            "Size": 4096,
            "StorageClass": "STANDARD",
            "Key": "obj-4k",
            "VersionId": "OG1zvrWAKlFlfX.fB0Q4pGyAtYp5ba8",
            "IsLatest": false,
            "LastModified": "2023-09-28T06:53:38.869Z",
            "Owner": {
                "DisplayName": "user1",
                "ID": "user1"
            }
        },
        {
            "ETag": "\"620f0b67a91f7f74151bc5be745b7110\"",
            "Size": 4096,
            "StorageClass": "STANDARD",
            "Key": "obj-4k",
            "VersionId": "xYf4wdcsFEiA4lzHzUe.EDWCpJwr9ba",
            "IsLatest": false,
            "LastModified": "2023-09-28T06:43:20.791Z",
            "Owner": {
                "DisplayName": "user1",
                "ID": "user1"
            }
        }
    ],
    "RequestCharged": null
}
[root@ceph-arc-archies-8k924y-node6 ~]# radosgw-admin bucket stats --bucket test-1 | egrep 'num_objects|versioning|num_shards'
    "num_shards": 11,
    "versioning": "off",
            "num_objects": 4
[root@ceph-arc-archies-8k924y-node6 ~]#

Comment 10 errata-xmlrpc 2023-12-13 15:23:48 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat Ceph Storage 7.0 Bug Fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:7780

Comment 11 Red Hat Bugzilla 2024-05-24 04:25:09 UTC

The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days