Bug 1463969 - bucket check --fix fails with (90) Message too long
bucket check --fix fails with (90) Message too long
Status: CLOSED ERRATA
Product: Red Hat Ceph Storage
Classification: Red Hat
Component: RGW (Show other bugs)
2.3
x86_64 Linux
high Severity unspecified
: rc
: 2.4
Assigned To: Orit Wasserman
ceph-qe-bugs
Bara Ancincova
:
Depends On:
Blocks: 1473436 1479701
  Show dependency treegraph
 
Reported: 2017-06-22 03:45 EDT by Orit Wasserman
Modified: 2017-10-18 14:13 EDT (History)
13 users (show)

See Also:
Fixed In Version: RHEL: ceph-10.2.7-33.el7cp Ubuntu: ceph_10.2.7-34redhat1
Doc Type: Bug Fix
Doc Text:
.Repairing bucket indexes works as expected Previously, the `cls` method of the Ceph Object Gateway that is used for repairing bucket indexes failed when its output result was too large. Consequently, affected bucket index objects could not be repaired using the `bucket check --fix` command, and the command failed with the "(90) Message too long" error. This update introduces a paging mechanism that ensures that bucket indexes can be repaired as expected.
Story Points: ---
Clone Of:
Environment:
Last Closed: 2017-10-17 14:12:51 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
bucket.py -- create a bucket with some objects (1.08 KB, text/plain)
2017-09-25 12:33 EDT, Warren
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
Ceph Project Bug Tracker 20772 None None None 2017-07-26 03:55 EDT

  None (edit)
Description Orit Wasserman 2017-06-22 03:45:57 EDT
Description of problem:
After failure when running bucket rm --purge-objects --bypass-gc  --inconsistent-index. Trying to fix the bucket index resulted in (90) Message too long error from the OSD.

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1. Create a large bucket 300K objects
2. run bucket rm --purge-objects --bypass-gc  --inconsistent-index and stop it in the middle creating a corrupted bucket index
3. try fixing the bucket index by running bucket check --fix

Actual results:
Error 90) Message too long

Expected results:
success

Additional info:
Comment 20 Warren 2017-09-25 12:33 EDT
Created attachment 1330617 [details]
bucket.py -- create a bucket with some objects

Test script used to create files to be deleted.
Comment 21 Warren 2017-09-25 12:38:15 EDT
Using the attached script, 300K files were created in a bucket named foo.

'time radosgw-admin bucket rm --purge-objects --bypass-gc --inconsistent-index --bucket=foo' was run.  This took about 90 minutes.

This process was repeated, and the rm operation was halted after 60 minutes.

radosgw-admin bucket check --fix ran to completion

repeated runs of halted rm operations showed that the number of objects decreased.  Eventually, all files and the bucket were deleted.

At no time did the "Error 90) Message too long" error appear.
Comment 25 Warren 2017-10-12 21:34:15 EDT
This has been verified on Ubuntu as well.
Comment 30 errata-xmlrpc 2017-10-17 14:12:51 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2903

Note You need to log in before you can comment on or make changes to this bug.