Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
This project is now read‑only. Starting Monday, February 2, please use https://ibm-ceph.atlassian.net/ for all bug tracking management.

Bug 2269342

Summary: [6.1z backport] [rgw-multipart/encryption]: Object download fails with "MD5 signatures do not match" if transitioned to other storage class
Product: [Red Hat Storage] Red Hat Ceph Storage Reporter: Bipin Kunal <bkunal>
Component: RGWAssignee: Matt Benjamin (redhat) <mbenjamin>
Status: CLOSED ERRATA QA Contact: Vidushi Mishra <vimishra>
Severity: high Docs Contact:
Priority: unspecified    
Version: 6.1CC: bkunal, ceph-eng-bugs, cephqe-warriors, ckulal, dwalveka, mbenjamin, mkasturi, tserlin, vimishra
Target Milestone: ---   
Target Release: 6.1z9   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: ceph-17.2.6-274.el9cp Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of: 2262766 Environment:
Last Closed: 2025-04-28 05:29:34 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 2262741, 2262766    
Bug Blocks: 2266648    

Description Bipin Kunal 2024-03-13 09:00:47 UTC 
+++ This bug was initially created as a clone of Bug #2262766 +++

+++ This bug was initially created as a clone of Bug #2262741 +++

Description of problem:


On downloading a multipart and encrypted object that was transitioned via LC policy to another storage class, we observed an MD5 signature mismatch error 

[root@ceph-vimpy-zpo62y-node6 ~]# s3cmd info  s3://test-transit1/logs-obj-20M-3
s3://test-transit1/logs-obj-20M-3 (object):
   File size: 20971520
   Last mod:  Mon, 05 Feb 2024 07:36:16 GMT
   MIME type: application/octet-stream
   Storage:   cold                          <<<<<<<<<<<<<<<
   MD5 sum:   8f4e33f3dc3e414ff94e5fb6905cba8c
   SSE:       none
   Policy:    none
   CORS:      none
   ACL:       user1: FULL_CONTROL
   x-amz-meta-s3cmd-attrs: atime:1707114783/ctime:1707114565/gid:0/gname:root/md5:8f4e33f3dc3e414ff94e5fb6905cba8c/mode:33188/mtime:1707114565/uid:0/uname:root
[root@ceph-vimpy-zpo62y-node6 ~]#

[root@ceph-vimpy-zpo62y-node6 ~]# s3cmd get s3://test-transit1/logs-obj-20M-3
download: 's3://test-transit1/logs-obj-20M-3' -> './logs-obj-20M-3'  [1 of 1]
 20971520 of 20971520   100% in    0s   232.76 MB/s  done
WARNING: MD5 signatures do not match: computed=2c2cb58ecd02171724877444eebd85d1, received=8f4e33f3dc3e414ff94e5fb6905cba8c

Version-Release number of selected component (if applicable):

ceph version 18.2.1-10.el9cp

How reproducible:
Always

Steps to Reproduce:
1. Create a ceph cluster with rgw configured.
2. Enable default encryption and restart rgw
[root@ceph-vimpy-zpo62y-node6 ~]# ceph config dump | grep rgw_crypt_default_encryption_key
client.rgw.rgw.1                                       dev       rgw_crypt_default_encryption_key       4YSmvJtBv0aZ7geVgAsdpRnLBEwWSWlMIGnRS8a9TSA=

3. create a bucket 'test-transit1' and perform multipart upload of size  20M objects.

4. Apply an LC rule to transition to a cold storage class in 3 days 

[root@ceph-vimpy-zpo62y-node6 ~]# s3cmd getlifecycle s3://test-transit1
<?xml version="1.0" ?>
<LifecycleConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
	<Rule>
		<ID>example-id</ID>
		<Filter>
			<Prefix>logs</Prefix>
		</Filter>
		<Status>Enabled</Status>
		<Transition>
			<Days>3</Days>
			<StorageClass>cold</StorageClass>
		</Transition>
	</Rule>
</LifecycleConfiguration>

[root@ceph-vimpy-zpo62y-node6 ~]#

5. Try object download for the objects transitioned to the cold class. 

[root@ceph-vimpy-zpo62y-node6 ~]# s3cmd info  s3://test-transit1/logs-obj-20M-3
s3://test-transit1/logs-obj-20M-3 (object):
   File size: 20971520
   Last mod:  Mon, 05 Feb 2024 07:36:16 GMT
   MIME type: application/octet-stream
   Storage:   cold                          <<<<<<<<<<<<<<<
   MD5 sum:   8f4e33f3dc3e414ff94e5fb6905cba8c
   SSE:       none
   Policy:    none
   CORS:      none
   ACL:       user1: FULL_CONTROL
   x-amz-meta-s3cmd-attrs: atime:1707114783/ctime:1707114565/gid:0/gname:root/md5:8f4e33f3dc3e414ff94e5fb6905cba8c/mode:33188/mtime:1707114565/uid:0/uname:root
[root@ceph-vimpy-zpo62y-node6 ~]#

[root@ceph-vimpy-zpo62y-node6 ~]# s3cmd get s3://test-transit1/logs-obj-20M-3
download: 's3://test-transit1/logs-obj-20M-3' -> './logs-obj-20M-3'  [1 of 1]
 20971520 of 20971520   100% in    0s   232.76 MB/s  done
WARNING: MD5 signatures do not match: computed=2c2cb58ecd02171724877444eebd85d1, received=8f4e33f3dc3e414ff94e5fb6905cba8c


Actual results:

Object download for multipart and encrypted object fails with MD5 signature mismatch error when the object was transitioned to a cold storage class

Expected results:

Download should not fail

Additional info:

--- Additional comment from Vidushi Mishra on 2024-02-05 11:04:51 UTC ---

This issue is also observed in 5.3 and 6.1 versions.
Comment 13 errata-xmlrpc 2025-04-28 05:29:34 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Red Hat Ceph Storage 6.1 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2025:4238
Comment 14 Red Hat Bugzilla 2025-08-27 04:25:05 UTC 
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days