Bug 2267715 - [RFE] QoLife improvements for "/usr/bin/rgw-restore-bucket-index" tool
Summary: [RFE] QoLife improvements for "/usr/bin/rgw-restore-bucket-index" tool
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Ceph Storage
Classification: Red Hat Storage
Component: RGW
Version: 7.1
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
: 7.1
Assignee: J. Eric Ivancich
QA Contact: Vidushi Mishra
Akash Raj
URL:
Whiteboard:
Depends On:
Blocks: 2267614 2298578 2298579
TreeView+ depends on / blocked
 
Reported: 2024-03-04 16:00 UTC by Tomas Petr
Modified: 2024-12-18 04:25 UTC (History)
9 users (show)

Fixed In Version: ceph-18.2.1-70.el9cp
Doc Type: Enhancement
Doc Text:
.Users can now place temporary files into a directory using the `-t` command-line option Previously, the `/usr/bin/rgw-restore-bucket-index` tool just used `/tmp`, and that directory sometimes did not have enough free space to hold all the temporary files. With this enhancement, the user can specify a directory into which the temporary files can be placed using the `-t` command-line option and will be notified if they run out of space, thereby knowing what adjustments to make to re-run the tool. Also, users can periodically check if the tool's temporary files have exhausted the available space on the file system where the temporary files are present.
Clone Of:
Environment:
Last Closed: 2024-06-13 14:28:36 UTC
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github ceph ceph pull 55885 0 None open rgw: make rgw-restore-bucket-index more robust 2024-03-04 16:06:00 UTC
Red Hat Issue Tracker RHCEPH-8424 0 None None None 2024-03-04 16:03:33 UTC
Red Hat Product Errata RHSA-2024:3925 0 None None None 2024-06-13 14:28:43 UTC

Description Tomas Petr 2024-03-04 16:00:34 UTC 
Description of problem:
Based on the experience from the field the "/usr/bin/rgw-restore-bucket-index" tool stores temporary objects in /tmp directory, these objects consumes space based on the size of the bucket that is being recovered, for large buckets this can eventually lead to "/usr/bin/rgw-restore-bucket-index" tool failure due to running out of space, but the error message does not suggest that is the case, but rather fails with 
"ln: failed to access '/tmp/rgwrbi-object-list.XXX': No such file or directory"


With that information, would it make sense to consider 2 improvements of the "/usr/bin/rgw-restore-bucket-index" tool:
1. add option for custom path where the files will be stored
2. add detection of running out of space

Version-Release number of selected component (if applicable):
RHCS 5, 6, 7

How reproducible:
always

Steps to Reproduce:
1. have 4GB large /tmp
2. attempt to recover bucket with 17M objects


Actual results:


Expected results:


Additional info:
Comment 1 J. Eric Ivancich 2024-03-07 02:48:43 UTC 
Should we target this at 7.1? Any other versions?
Comment 2 J. Eric Ivancich 2024-03-12 17:20:40 UTC 
We need to pull the commits from 2 upstream PRs:

https://github.com/ceph/ceph/pull/55885
https://github.com/ceph/ceph/pull/56133
Comment 3 Tomas Petr 2024-03-14 08:33:36 UTC 
(In reply to J. Eric Ivancich from comment #1)
> Should we target this at 7.1? Any other versions?

Hi Eric,
it would be good to have it in all active versions so also RHCS 5, 6 and 7.
With known impact and possible workaround (increase the /tmp size if possible) we could skip RHCS 5, but it would be nice to have it in next RHCS 6 version or RHCS 6.2 (not sure about what is the next planned version).
Comment 4 J. Eric Ivancich 2024-03-18 18:34:56 UTC 
So I'm retargeting this BZ to 7.1 and we can clone it for the RHCS 6.*.
Comment 5 J. Eric Ivancich 2024-03-18 18:50:15 UTC 
Two commits added to ceph-7.1-rhel-patches.
Comment 12 errata-xmlrpc 2024-06-13 14:28:36 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Critical: Red Hat Ceph Storage 7.1 security, enhancements, and bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2024:3925
Comment 13 Red Hat Bugzilla 2024-12-18 04:25:03 UTC 
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days
Note You need to log in before you can comment on or make changes to this bug.