This bug was initially created as a copy of Bug #2262741 Description of problem: On downloading a multipart and encrypted object that was transitioned via LC policy to another storage class, we observed an MD5 signature mismatch error [root@ceph-vimpy-zpo62y-node6 ~]# s3cmd info s3://test-transit1/logs-obj-20M-3 s3://test-transit1/logs-obj-20M-3 (object): File size: 20971520 Last mod: Mon, 05 Feb 2024 07:36:16 GMT MIME type: application/octet-stream Storage: cold <<<<<<<<<<<<<<< MD5 sum: 8f4e33f3dc3e414ff94e5fb6905cba8c SSE: none Policy: none CORS: none ACL: user1: FULL_CONTROL x-amz-meta-s3cmd-attrs: atime:1707114783/ctime:1707114565/gid:0/gname:root/md5:8f4e33f3dc3e414ff94e5fb6905cba8c/mode:33188/mtime:1707114565/uid:0/uname:root [root@ceph-vimpy-zpo62y-node6 ~]# [root@ceph-vimpy-zpo62y-node6 ~]# s3cmd get s3://test-transit1/logs-obj-20M-3 download: 's3://test-transit1/logs-obj-20M-3' -> './logs-obj-20M-3' [1 of 1] 20971520 of 20971520 100% in 0s 232.76 MB/s done WARNING: MD5 signatures do not match: computed=2c2cb58ecd02171724877444eebd85d1, received=8f4e33f3dc3e414ff94e5fb6905cba8c Version-Release number of selected component (if applicable): ceph version 18.2.1-10.el9cp How reproducible: Always Steps to Reproduce: 1. Create a ceph cluster with rgw configured. 2. Enable default encryption and restart rgw [root@ceph-vimpy-zpo62y-node6 ~]# ceph config dump | grep rgw_crypt_default_encryption_key client.rgw.rgw.1 dev rgw_crypt_default_encryption_key 4YSmvJtBv0aZ7geVgAsdpRnLBEwWSWlMIGnRS8a9TSA= 3. create a bucket 'test-transit1' and perform multipart upload of size 20M objects. 4. Apply an LC rule to transition to a cold storage class in 3 days [root@ceph-vimpy-zpo62y-node6 ~]# s3cmd getlifecycle s3://test-transit1 <?xml version="1.0" ?> <LifecycleConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/"> <Rule> <ID>example-id</ID> <Filter> <Prefix>logs</Prefix> </Filter> <Status>Enabled</Status> <Transition> <Days>3</Days> <StorageClass>cold</StorageClass> </Transition> </Rule> </LifecycleConfiguration> [root@ceph-vimpy-zpo62y-node6 ~]# 5. Try object download for the objects transitioned to the cold class. [root@ceph-vimpy-zpo62y-node6 ~]# s3cmd info s3://test-transit1/logs-obj-20M-3 s3://test-transit1/logs-obj-20M-3 (object): File size: 20971520 Last mod: Mon, 05 Feb 2024 07:36:16 GMT MIME type: application/octet-stream Storage: cold <<<<<<<<<<<<<<< MD5 sum: 8f4e33f3dc3e414ff94e5fb6905cba8c SSE: none Policy: none CORS: none ACL: user1: FULL_CONTROL x-amz-meta-s3cmd-attrs: atime:1707114783/ctime:1707114565/gid:0/gname:root/md5:8f4e33f3dc3e414ff94e5fb6905cba8c/mode:33188/mtime:1707114565/uid:0/uname:root [root@ceph-vimpy-zpo62y-node6 ~]# [root@ceph-vimpy-zpo62y-node6 ~]# s3cmd get s3://test-transit1/logs-obj-20M-3 download: 's3://test-transit1/logs-obj-20M-3' -> './logs-obj-20M-3' [1 of 1] 20971520 of 20971520 100% in 0s 232.76 MB/s done WARNING: MD5 signatures do not match: computed=2c2cb58ecd02171724877444eebd85d1, received=8f4e33f3dc3e414ff94e5fb6905cba8c Actual results: Object download for multipart and encrypted object fails with MD5 signature mismatch error when the object was transitioned to a cold storage class Expected results: Download should not fail Additional info: manual test logs on 5.3: https://docs.google.com/document/d/1K5zRHfLEC1LfwSpsLxFO28WKh3NyFWdum1s6jeOEHkU/edit?usp=sharing