Bug 1496847 - Failed CompleteMultipartUpload request does not release lock
Summary: Failed CompleteMultipartUpload request does not release lock
Alias: None
Product: Red Hat Ceph Storage
Classification: Red Hat
Component: RGW
Version: 3.0
Hardware: Unspecified
OS: Unspecified
Target Milestone: rc
: 2.4
Assignee: Matt Benjamin (redhat)
QA Contact: ceph-qe-bugs
Depends On: 1497853
TreeView+ depends on / blocked
Reported: 2017-09-28 15:42 UTC by Casey Bodley
Modified: 2017-10-17 18:12 UTC (History)
11 users (show)

Fixed In Version: RHEL: ceph-10.2.7-44.el7cp Ubuntu: ceph_10.2.7-44redhat1
Doc Type: No Doc Update
Doc Text:
Clone Of:
: 1497853 (view as bug list)
Last Closed: 2017-10-17 18:12:51 UTC
Target Upstream Version:

Attachments (Terms of Use)

System ID Private Priority Status Summary Last Updated
Ceph Project Bug Tracker 21596 0 None None None 2017-09-28 15:42:22 UTC
Red Hat Product Errata RHBA-2017:2903 0 normal SHIPPED_LIVE Red Hat Ceph Storage 2.4 enhancement and bug fix update 2017-10-17 22:12:30 UTC

Description Casey Bodley 2017-09-28 15:42:23 UTC
Description of problem:

A rados lock is used to prevent racing CompleteMultipartUpload requests on the same object. But if we fail the request (due to invalid arguments, missing parts, etc), the lock is not released.

This causes later CompleteMultipartUpload requests on that object to fail with '500 Internal Error' and log:

> RGWCompleteMultipart::execute() failed to acquire lock

Version-Release number of selected component (if applicable):


How reproducible:

After a failed CompleteMultipartUpload request, all later CompleteMultipartUpload requests on that object will fail with '500 Internal Error' for a duration of rgw_mp_lock_max_time (default: 10 minutes)

Steps to Reproduce:
1. Create an object
2. Issue a CompleteMultipartUpload on that object with an invalid upload id
3. Reissue the same invalid CompleteMultipartUpload request

Actual results:

The first request fails with 404 NoSuchUpload, and the second request fails with this 500 InternalError:

botocore.exceptions.ClientError: An error occurred (InternalError) when calling the CompleteMultipartUpload operation (reached max retries: 4): This multipart completion is already in progress

Expected results:

Both requests should fail with 404 NoSuchUpload.

Additional info:

Comment 8 Christina Meno 2017-10-02 20:54:20 UTC
When can we expect the patch downstream?

Comment 21 errata-xmlrpc 2017-10-17 18:12:51 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.