Bug 1530801 - Swift post via RGW ends up with random 500 errors
Summary: Swift post via RGW ends up with random 500 errors
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Ceph Storage
Classification: Red Hat Storage
Component: RGW
Version: 3.0
Hardware: x86_64
OS: Linux
high
high
Target Milestone: z1
: 3.0
Assignee: Adam C. Emerson
QA Contact: Tejas
URL:
Whiteboard:
Depends On:
Blocks: 1491739
TreeView+ depends on / blocked
 
Reported: 2018-01-03 19:42 UTC by Ken Dreyer (Red Hat)
Modified: 2021-12-10 15:51 UTC (History)
15 users (show)

Fixed In Version: RHEL: ceph-12.2.1-41.el7cp Ubuntu: ceph_12.2.1-43redhat1xenial
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1491739
Environment:
Last Closed: 2018-03-08 15:52:40 UTC
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Ceph Project Bug Tracker 22517 0 None None None 2018-01-03 19:42:51 UTC
Github ceph ceph pull 18954 0 None closed rgw: Potential fix for possible 500 on POST 2020-10-20 10:50:41 UTC
Github ceph ceph pull 19581 0 None closed Cache on the barrelhead 2020-10-20 10:50:41 UTC
Github ceph ceph pull 19601 0 None closed rgw: retry CORS put/delete operations on ECANCELLED 2020-10-20 10:50:41 UTC
Red Hat Issue Tracker RHCEPH-2620 0 None None None 2021-12-10 15:51:49 UTC
Red Hat Product Errata RHBA-2018:0474 0 normal SHIPPED_LIVE Red Hat Ceph Storage 3.0 bug fix update 2018-03-08 20:51:53 UTC

Comment 2 Adam C. Emerson 2018-01-05 22:20:50 UTC 
For testing purposes, we are working on the assumption that this bug is caused by cache invalidation notifies failing to propagate to all machines under heavy load

Thus, as with the other bug (as discussed before with me and Eric) run the cluster under heavy enough load with multiple gateways until the log starts showing failures to propagate cache. Have the gateways update bucket metadata. Eventually you should be able to provoke the behavior where one node has failed to have its cache updated and responds with a 500 error.

The fix should make that not happen
Comment 10 errata-xmlrpc 2018-03-08 15:52:40 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:0474
Note You need to log in before you can comment on or make changes to this bug.