Bug 1902685

Summary: Too strict Content-Length header check refuses valid upload requests
Product: [Red Hat Storage] Red Hat OpenShift Container Storage Reporter: Michal Minar <miminar>
Component: Multi-Cloud Object GatewayAssignee: Jacky Albo <jalbo>
Status: CLOSED ERRATA QA Contact: Ben Eli <belimele>
Severity: high Docs Contact:
Priority: high    
Version: 4.5CC: belimele, ebenahar, etamir, jalbo, muagarwa, nbecker, ocs-bugs
Target Milestone: ---Keywords: AutomationBackLog
Target Release: OCS 4.7.0   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: v4.7.0-229.ci Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1915336 1917252 (view as bug list) Environment:
Last Closed: 2021-05-19 09:16:33 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1915336, 1917252    
Attachments:
Description Flags
logs from SDI's dlog backup pod
none
logs from noobaa-endpoint
none
logs from a successful SDI's dlog backup pod none

Description Michal Minar 2020-11-30 12:17:18 UTC 
Created attachment 1734808 [details]
logs from SDI's dlog backup pod

Description of problem:
  SAP backup solution for SAP Data Intelligence dlog component fails to push NooBaa S3 endpoint due to a strict Content-Length check.

Version-Release number of selected component (if applicable):
  4.5.2

How reproducible:
  Always
Steps to Reproduce:
1.  Deploy SAP Data Intelligence 3.1 and enable backups.
2.  Wait for the backup job to fail.

Actual results:
  Job fails (logs attached)

Expected results:
  backup succeeds

Additional info:
- The backup works AWS
- According to the following, Content-Length header is not required:
    - https://docs.aws.amazon.com/AmazonS3/latest/API/API_CopyObject.html
    - https://docs.aws.amazon.com/AmazonS3/latest/API/API_UploadPartCopy.html

Server side (NooBaa endpoint) error:
    2020-11-27T13:10:42.939062081Z Nov-27 13:10:42.938 [Endpoint/14] [ERROR] core.endpoint.s3.s3_rest:: S3 ERROR <?xml version="1.0" encoding="UTF-8"?><Error><Code>MissingContentLength</Code><Message>You must provide the Content-Length HTTP header.</Message><Resource>/sdi-checkpoint-store-6f3991ac-d410-424f-b588-004e3765339e/hxtwild/1606435972/vora/store/2/720575940379379772.100416/26?X-Amz-Algorithm=AWS4-HMAC-SHA256&amp;X-Amz-Credential=uxEosx1TU6sPSzAwbdJj%2F20201127%2F%2Fs3%2Faws4_request&amp;X-Amz-Date=20201127T131042Z&amp;X-Amz-Expires=180&amp;X-Amz-SignedHeaders=host%3Bx-amz-copy-source&amp;X-Amz-Signature=c0a6432f46d7dfacff6161b677bd9b87bbad918fd47947f11256bec9ed2daa76</Resource><RequestId>ki0aacjd-73j83o-ml1</RequestId></Error> PUT /sdi-checkpoint-store-6f3991ac-d410-424f-b588-004e3765339e/hxtwild/1606435972/vora/store/2/720575940379379772.100416/26?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=uxEosx1TU6sPSzAwbdJj%2F20201127%2F%2Fs3%2Faws4_request&X-Amz-Date=20201127T131042Z&X-Amz-Expires=180&X-Amz-SignedHeaders=host%3Bx-amz-copy-source&X-Amz-Signature=c0a6432f46d7dfacff6161b677bd9b87bbad918fd47947f11256bec9ed2daa76 {"host":"s3.openshift-storage.svc.cluster.local","accept":"*/*","x-amz-copy-source":"sdi-checkpoint-store-6f3991ac-d410-424f-b588-004e3765339e/checkpoints-hxtwild/store/2/720575940379379772.100416/26"} Error: You must provide the Content-Length HTTP header.
Comment 2 Michal Minar 2020-11-30 12:18:35 UTC 
Created attachment 1734809 [details]
logs from noobaa-endpoint
Comment 9 Michal Minar 2021-01-14 11:00:41 UTC 
Would it be possible to backport to 4.6.z, please?
Comment 21 Michal Minar 2021-02-05 19:23:42 UTC 
Created attachment 1755294 [details]
logs from a successful SDI's dlog backup pod

against ocs-operator.v4.7.0-241.ci
Comment 27 errata-xmlrpc 2021-05-19 09:16:33 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: Red Hat OpenShift Container Storage 4.7.0 security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:2041