Bug 2308333

Summary: [rgw-multiiste][cloud-transition]: Discrepancy in num_objects count in bucket stats on replicated zone for cloud transitioned objects
Product: [Red Hat Storage] Red Hat Ceph Storage Reporter: Vidushi Mishra <vimishra>
Component: RGW-MultisiteAssignee: Soumya Koduri <skoduri>
Status: CLOSED ERRATA QA Contact: Anuchaithra <anrao>
Severity: high Docs Contact:
Priority: unspecified    
Version: 8.0CC: akraj, anrao, ceph-eng-bugs, cephqe-warriors, skoduri, tserlin
Target Milestone: ---Keywords: Automation
Target Release: 8.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: ceph-19.1.0-67.el9cp Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2024-11-25 09:07:41 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2317218    

Description Vidushi Mishra 2024-08-28 12:30:24 UTC
Description of problem: 

In a multisite environment when objects are transitioned to the cloud using the retain head true option, not all transitioned objects are reported as cloud-tiered in the bucket stats on the replicated site.

Version-Release number of selected component (if applicable):

ceph version 19.1.0-51.el9cp 

How reproducible:

2/2

Steps to Reproduce:

1. create an rgw-ms environment and create a bucket on primary zone 'test-2'
2. upload around 90 objects from the primary zone and ensure they have synced to the secondary zone.
3. With the retain head true option enabled, apply an LC policy to transition the objects to the cloud from the primary zone.

Actual results:

1. We observed that in the "secondary zone", the bucket stats for bucket 'test-2' show that 84 objects are cloudtiered and 6 objects are not. 
        "rgw.main": {
            "size": 0,                 <<<<< size is 0 
            "size_actual": 0,
            "size_utilized": 0,
            "size_kb": 0,
            "size_kb_actual": 0,
            "size_kb_utilized": 0,
            "num_objects": 6             <<<<<<<< 
        },
        "rgw.cloudtiered": {
            "size": 0,
            "size_actual": 0,
            "size_utilized": 0,
            "size_kb": 0,
            "size_kb_actual": 0,
            "size_kb_utilized": 0,
            "num_objects": 84
        }

3. The "primary zone" has all the 90 objects cloud-tiered as per the bucket stats as is the expected behavior.

        "rgw.main": {
            "size": 0,
            "size_actual": 0,
            "size_utilized": 0,
            "size_kb": 0,
            "size_kb_actual": 0,
            "size_kb_utilized": 0,
            "num_objects": 0
        },
        "rgw.cloudtiered": {
            "size": 0,
            "size_actual": 0,
            "size_utilized": 0,
            "size_kb": 0,
            "size_kb_actual": 0,
            "size_kb_utilized": 0,
            "num_objects": 90
        }

Expected results:

The bucket stats on the secondary zone should be consistent with the primary zone

Additional info:

LC rule 

[root@ceph-pri-vim-ms-8-0-cdvamy-node4 85fb03c6-63d8-11ef-93b9-fa163e37b0e1]# s3cmd getlifecycle s3://test-2
<?xml version="1.0" ?>
<LifecycleConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
	<Rule>
		<ID>LC_Rule_1</ID>
		<Filter>
			<Prefix>key1</Prefix>
		</Filter>
		<Status>Enabled</Status>
		<Transition>
			<Days>2</Days>
			<StorageClass>CLOUDIBM</StorageClass>
		</Transition>
		<NoncurrentVersionTransition>
			<NoncurrentDays>1</NoncurrentDays>
			<StorageClass>CLOUDIBM</StorageClass>
		</NoncurrentVersionTransition>
	</Rule>
</LifecycleConfiguration>

Comment 9 errata-xmlrpc 2024-11-25 09:07:41 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat Ceph Storage 8.0 security, bug fix, and enhancement updates), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2024:10216