Bug 2021993

Summary:	rgw: deleting and purging a bucket can get stuck in an endless loop 4.3
Product:	[Red Hat Storage] Red Hat Ceph Storage	Reporter:	J. Eric Ivancich <ivancich>
Component:	RGW	Assignee:	J. Eric Ivancich <ivancich>
Status:	CLOSED ERRATA	QA Contact:	Vidushi Mishra <vimishra>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	4.2	CC:	cbodley, ceph-eng-bugs, ceph-qe-bugs, kbader, mbenjamin, tchandra, tserlin
Target Milestone:	---
Target Release:	4.3
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	ceph-14.2.22-19.el8cp, ceph-14.2.22-19.el7cp	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2022-05-05 07:54:04 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description J. Eric Ivancich 2021-11-10 14:20:33 UTC

This bug was initially created as a copy of Bug #2021601

I am copying this bug because: 



This bug was initially created as a copy of Bug #2021600

I am copying this bug because: 

It appears in multiple downstream releases.

Description of problem:

This stems from upstream tracker: https://tracker.ceph.com/issues/52662 .

If all unfinished multipart upload ".meta" index entries cannot be retrieved in one go, the marker is not advanced and the listing will be stuck. This happens during the phase of bucket deletion/purge where it tries to abort all incomplete multipart uploads.


Version-Release number of selected component (if applicable):


How reproducible:

Very reproducible.


Steps to Reproduce:
1. Load up the bucket index with many thousands of entries in the _multipart_ namespace representing parts of incomplete multipart uploads. Add two entries that are in that namespace spread apart in the lexically (so there are thousands of entries in between) and end with ".meta" each representing one incomplete multipart upload.
2. Issue `radosgw-admin bucket rm --bucket=<bucket> --purge-objects --bypass-gc`

Actual results:

Command will never finished. If --debug-rgw=20 added, one will see the same bucket index entries re-visited multiple times in the log. Bucket will not be removed.

Expected results:

Command to complete, bucket removed.

Additional info:

Comment 1 J. Eric Ivancich 2021-11-10 16:18:17 UTC

Cherry-picked to add commit cf4a316d0073309a8a8755e7c875fd1ad050102d on ceph-4.3-rhel-patches.

Comment 4 J. Eric Ivancich 2021-11-11 19:27:25 UTC

See comments 7 and 8 on https://bugzilla.redhat.com/show_bug.cgi?id=2021601 for a reproducer.

Comment 16 errata-xmlrpc 2022-05-05 07:54:04 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: Red Hat Ceph Storage 4.3 Security and Bug Fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:1716