Bug 2269342 - [6.1z backport] [rgw-multipart/encryption]: Object download fails with "MD5 signatures do not match" if transitioned to other storage class
Summary: [6.1z backport] [rgw-multipart/encryption]: Object download fails with "MD5 s...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Ceph Storage
Classification: Red Hat Storage
Component: RGW
Version: 6.1
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: 6.1z9
Assignee: Matt Benjamin (redhat)
QA Contact: Vidushi Mishra
URL:
Whiteboard:
Depends On: 2262741 2262766
Blocks: 2266648
TreeView+ depends on / blocked
 
Reported: 2024-03-13 09:00 UTC by Bipin Kunal
Modified: 2025-08-27 04:25 UTC (History)
9 users (show)

Fixed In Version: ceph-17.2.6-274.el9cp
Doc Type: No Doc Update
Doc Text:
Clone Of: 2262766
Environment:
Last Closed: 2025-04-28 05:29:34 UTC
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHCEPH-8579 0 None None None 2024-03-20 13:47:50 UTC
Red Hat Product Errata RHSA-2025:4238 0 None None None 2025-04-28 05:29:42 UTC

Description Bipin Kunal 2024-03-13 09:00:47 UTC
+++ This bug was initially created as a clone of Bug #2262766 +++

+++ This bug was initially created as a clone of Bug #2262741 +++

Description of problem:


On downloading a multipart and encrypted object that was transitioned via LC policy to another storage class, we observed an MD5 signature mismatch error 

[root@ceph-vimpy-zpo62y-node6 ~]# s3cmd info  s3://test-transit1/logs-obj-20M-3
s3://test-transit1/logs-obj-20M-3 (object):
   File size: 20971520
   Last mod:  Mon, 05 Feb 2024 07:36:16 GMT
   MIME type: application/octet-stream
   Storage:   cold                          <<<<<<<<<<<<<<<
   MD5 sum:   8f4e33f3dc3e414ff94e5fb6905cba8c
   SSE:       none
   Policy:    none
   CORS:      none
   ACL:       user1: FULL_CONTROL
   x-amz-meta-s3cmd-attrs: atime:1707114783/ctime:1707114565/gid:0/gname:root/md5:8f4e33f3dc3e414ff94e5fb6905cba8c/mode:33188/mtime:1707114565/uid:0/uname:root
[root@ceph-vimpy-zpo62y-node6 ~]#

[root@ceph-vimpy-zpo62y-node6 ~]# s3cmd get s3://test-transit1/logs-obj-20M-3
download: 's3://test-transit1/logs-obj-20M-3' -> './logs-obj-20M-3'  [1 of 1]
 20971520 of 20971520   100% in    0s   232.76 MB/s  done
WARNING: MD5 signatures do not match: computed=2c2cb58ecd02171724877444eebd85d1, received=8f4e33f3dc3e414ff94e5fb6905cba8c

Version-Release number of selected component (if applicable):

ceph version 18.2.1-10.el9cp

How reproducible:
Always

Steps to Reproduce:
1. Create a ceph cluster with rgw configured.
2. Enable default encryption and restart rgw
[root@ceph-vimpy-zpo62y-node6 ~]# ceph config dump | grep rgw_crypt_default_encryption_key
client.rgw.rgw.1                                       dev       rgw_crypt_default_encryption_key       4YSmvJtBv0aZ7geVgAsdpRnLBEwWSWlMIGnRS8a9TSA=

3. create a bucket 'test-transit1' and perform multipart upload of size  20M objects.

4. Apply an LC rule to transition to a cold storage class in 3 days 

[root@ceph-vimpy-zpo62y-node6 ~]# s3cmd getlifecycle s3://test-transit1
<?xml version="1.0" ?>
<LifecycleConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
	<Rule>
		<ID>example-id</ID>
		<Filter>
			<Prefix>logs</Prefix>
		</Filter>
		<Status>Enabled</Status>
		<Transition>
			<Days>3</Days>
			<StorageClass>cold</StorageClass>
		</Transition>
	</Rule>
</LifecycleConfiguration>

[root@ceph-vimpy-zpo62y-node6 ~]#

5. Try object download for the objects transitioned to the cold class. 

[root@ceph-vimpy-zpo62y-node6 ~]# s3cmd info  s3://test-transit1/logs-obj-20M-3
s3://test-transit1/logs-obj-20M-3 (object):
   File size: 20971520
   Last mod:  Mon, 05 Feb 2024 07:36:16 GMT
   MIME type: application/octet-stream
   Storage:   cold                          <<<<<<<<<<<<<<<
   MD5 sum:   8f4e33f3dc3e414ff94e5fb6905cba8c
   SSE:       none
   Policy:    none
   CORS:      none
   ACL:       user1: FULL_CONTROL
   x-amz-meta-s3cmd-attrs: atime:1707114783/ctime:1707114565/gid:0/gname:root/md5:8f4e33f3dc3e414ff94e5fb6905cba8c/mode:33188/mtime:1707114565/uid:0/uname:root
[root@ceph-vimpy-zpo62y-node6 ~]#

[root@ceph-vimpy-zpo62y-node6 ~]# s3cmd get s3://test-transit1/logs-obj-20M-3
download: 's3://test-transit1/logs-obj-20M-3' -> './logs-obj-20M-3'  [1 of 1]
 20971520 of 20971520   100% in    0s   232.76 MB/s  done
WARNING: MD5 signatures do not match: computed=2c2cb58ecd02171724877444eebd85d1, received=8f4e33f3dc3e414ff94e5fb6905cba8c


Actual results:

Object download for multipart and encrypted object fails with MD5 signature mismatch error when the object was transitioned to a cold storage class

Expected results:

Download should not fail

Additional info:

--- Additional comment from Vidushi Mishra on 2024-02-05 11:04:51 UTC ---

This issue is also observed in 5.3 and 6.1 versions.

Comment 13 errata-xmlrpc 2025-04-28 05:29:34 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Red Hat Ceph Storage 6.1 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2025:4238

Comment 14 Red Hat Bugzilla 2025-08-27 04:25:05 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days


Note You need to log in before you can comment on or make changes to this bug.