Bug 2317891

Summary: rgw/lifecycle: lifecycle transition fails for non-current object with instance empty
Product: [Red Hat Storage] Red Hat Ceph Storage Reporter: Soumya Koduri <skoduri>
Component: RGWAssignee: Soumya Koduri <skoduri>
Status: CLOSED ERRATA QA Contact: Vidushi Mishra <vimishra>
Severity: high Docs Contact:
Priority: unspecified    
Version: 8.0CC: akraj, ceph-eng-bugs, cephqe-warriors, rpollack, tserlin, vimishra
Target Milestone: ---   
Target Release: 8.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: ceph-19.2.0-16.el9cp Doc Type: Bug Fix
Doc Text:
.Lifecycle transition no longer fails for non-current object with an empty instance Previously, when bucket versioning was enabled, old plain object entries would get converted to versioned by updating its instance as "null" in its raw head/old object. Due to this, the lifecycle transition would fail for a non-current object with an instance empty. With this fix, the code is corrected to keep the instance empty while updating bucket index entries and the lifecycle transition works for all plain entries which are converted to versioned.
 Story Points: ---
Clone Of: Environment:
Last Closed: 2024-11-25 09:13:10 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2317218    

Description Soumya Koduri 2024-10-10 19:36:27 UTC 
Description of problem:

Lifecycle transition (with rule set for non-current versions) fails with below error for the object version created prior to bucket versioning is enabled.

<<<
out/radosgw.8001.log:2024-10-07T00:09:35.620+0530 7fc9eee566c0  0 lifecycle: transition_obj ERROR: failed to transition obj(lc_h1) read_mtime = 2024-10-07T00:07:38.972424+0530 doesn't match mtime = 2024-10-07T00:07:09.220348+0530
>>>

Upstream tracker: https://tracker.ceph.com/issues/68402


Version-Release number of selected component (if applicable):


How reproducible:
Always

Steps to Reproduce:
Lifecycle transition (with rule set for non-current versions) fails with below error for the object version created prior to bucket versioning is enabled.

	out/radosgw.8001.log:2024-10-07T00:09:35.620+0530 7fc9eee566c0  0 lifecycle: transition_obj ERROR: failed to transition obj(lc_h1) read_mtime = 2024-10-07T00:07:38.972424+0530 doesn't match mtime = 2024-10-07T00:07:09.220348+0530
	out/radosgw.8001.log:2024-10-07T00:09:35.620+0530 7fc9eee566c0  0 lifecycle: ERROR: failed to transition obj :bucket4[d5e68941-d38f-45d3-a5b6-cbbb16786734.14117.1]):lc_h1 -> S1 (125) Operation canceled wp_thrd: 2, 1
	out/radosgw.8001.log:2024-10-07T00:10:16.149+0530 7fc9f0e5a6c0  0 lifecycle: transition_obj ERROR: failed to transition obj(lc_h1) read_mtime = 2024-10-07T00:07:38.972424+0530 doesn't match mtime = 2024-10-07T00:07:09.220348+0530
	out/radosgw.8001.log:2024-10-07T00:10:16.149+0530 7fc9f0e5a6c0  0 lifecycle: ERROR: failed to transition obj :bucket4[d5e68941-d38f-45d3-a5b6-cbbb16786734.14117.1]):lc_h1 -> S1 (125) Operation canceled wp_thrd: 1, 1


Steps to reproduce:

	[root@fedora build]# s3cmd mb s3://bucket4
	Bucket 's3://bucket4/' created
	[root@fedora build]# s3cmd put h1 s3://bucket4/lc_h1
	upload: 'h1' -> 's3://bucket4/lc_h1'  [1 of 1]
	 21 of 21   100% in    2s     8.41 B/s  done
	[root@fedora build]# aws s3api --no-verify-ssl --endpoint http://localhost:8000 list-object-versions --bucket=bucket4
	{
	    "Versions": [
		{
		    "ETag": "\"1115173c87f173f4f3cd255ce79d4de5\"",
		    "Size": 21,
		    "StorageClass": "STANDARD",
		    "Key": "lc_h1",
		    "VersionId": "null",
		    "IsLatest": true,
		    "LastModified": "2024-10-06T18:37:09.220000+00:00",
		    "Owner": {
		        "DisplayName": "cosbench_user",
		        "ID": "cosbench"
		    }
		}
	    ],
	    "RequestCharged": null
	}
	[root@fedora build]# aws s3api --no-verify-ssl --endpoint http://localhost:8000 put-bucket-versioning --bucket bucket4 --versioning-configuration Status=Enabled
	[root@fedora build]# s3cmd put h1 s3://bucket4/lc_h1
	upload: 'h1' -> 's3://bucket4/lc_h1'  [1 of 1]
	 21 of 21   100% in    0s   428.37 B/s  done
	[root@fedora build]# aws s3api --no-verify-ssl --endpoint http://localhost:8000 list-object-versions --bucket=bucket4
	{
	    "Versions": [
		{
		    "ETag": "\"1115173c87f173f4f3cd255ce79d4de5\"",
		    "Size": 21,
		    "StorageClass": "STANDARD",
		    "Key": "lc_h1",
		    "VersionId": "ZXoNuSHA.BQiQGmGcrGmxkUD54P0M09",
		    "IsLatest": true,
		    "LastModified": "2024-10-06T18:37:38.972000+00:00",
		    "Owner": {
		        "DisplayName": "cosbench_user",
		        "ID": "cosbench"
		    }
		},
		{
		    "ETag": "\"1115173c87f173f4f3cd255ce79d4de5\"",
		    "Size": 21,
		    "StorageClass": "STANDARD",
		    "Key": "lc_h1",
		    "VersionId": "null",
		    "IsLatest": false,
	[root@fedora build]# 
	[root@fedora build]# s3cmd setlifecycle ../../scripts/lifecycle_S1_noncur.xml s3://bucket4
	s3://bucket4/: Lifecycle Policy updated
	[root@fedora build]# s3cmd getlifecycle s3://bucket4
	<?xml version="1.0" ?>
	<LifecycleConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
		<Rule>
			<ID>Transition and Expiration Rule</ID>
			<Filter>
				<Prefix>lc</Prefix>
			</Filter>
			<Status>Enabled</Status>
			<Transition>
				<Days>1</Days>
				<StorageClass>S1</StorageClass>
			</Transition>
			<NoncurrentVersionTransition>
				<NoncurrentDays>1</NoncurrentDays>
				<StorageClass>S1</StorageClass>
			</NoncurrentVersionTransition>
		</Rule>
	</LifecycleConfiguration>

	[root@fedora build]# aws s3api --no-verify-ssl --endpoint http://localhost:8000 list-object-versions --bucket=bucket4
	{
	    "Versions": [
		{
		    "ETag": "\"1115173c87f173f4f3cd255ce79d4de5\"",
		    "Size": 21,
		    "StorageClass": "S1",
		    "Key": "lc_h1",
		    "VersionId": "ZXoNuSHA.BQiQGmGcrGmxkUD54P0M09",
		    "IsLatest": true,
		    "LastModified": "2024-10-06T18:37:38.972000+00:00",
		    "Owner": {
		        "DisplayName": "cosbench_user",
		        "ID": "cosbench"
		    }
		},
		{
		    "ETag": "\"1115173c87f173f4f3cd255ce79d4de5\"",
		    "Size": 21,
		    "StorageClass": "STANDARD",
		    "Key": "lc_h1",
		    "VersionId": "null",
		    "IsLatest": false,
		    "LastModified": "2024-10-06T18:37:09.220000+00:00",
		    "Owner": {
		        "DisplayName": "cosbench_user",
		        "ID": "cosbench"
		    }
		}
	    ],
	    "RequestCharged": null
	}
	[root@fedora build]# 



Actual results:
Lifecycle transition fails for those versioned objects with instance empty


Expected results:
Transition should succeed

Additional info:
Comment 1 Storage PM bot 2024-10-10 19:36:40 UTC 
Please specify the severity of this bug. Severity is defined here:
https://bugzilla.redhat.com/page.cgi?id=fields.html#bug_severity.
Comment 10 errata-xmlrpc 2024-11-25 09:13:10 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat Ceph Storage 8.0 security, bug fix, and enhancement updates), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2024:10216