Bug 1462011 - changes for enabling rocksdb for filestore in RHCS 2.y
changes for enabling rocksdb for filestore in RHCS 2.y
Product: Red Hat Ceph Storage
Classification: Red Hat
Component: RADOS (Show other bugs)
x86_64 Linux
high Severity high
: rc
: 2.4
Assigned To: Josh Durgin
Bara Ancincova
: 1480720 (view as bug list)
Depends On:
Blocks: 1479701
  Show dependency treegraph
Reported: 2017-06-15 18:15 EDT by Vikhyat Umrao
Modified: 2017-10-27 12:56 EDT (History)
14 users (show)

See Also:
Fixed In Version: RHEL: ceph-10.2.7-48.el7cp Ubuntu: ceph_10.2.7-48redhat1
Doc Type: Enhancement
Doc Text:
.`RocksDB` is enabled as an option to replace `levelDB` This update enables an option to use the `RocksDB` back end for the `omap` database as opposed to `levelDB`. `RocksDB` uses the multi-threading mechanism in compaction so that it better handles the situation when the `omap` directories become very large (more than 40 G). `LevelDB` compaction takes a lot of time in such a situation and causes OSD daemons to time out. For details about conversion from `levelDB` to `RocksDB`, see the link:https://access.redhat.com/solutions/3210951[Ceph - Steps to convert OSD omap backend from leveldb to rocksdb] solution on Red Hat Customer Portal.
Story Points: ---
Clone Of:
: 1462012 1500100 (view as bug list)
Last Closed: 2017-10-17 14:12:51 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

External Trackers
Tracker ID Priority Status Summary Last Updated
Ceph Project Bug Tracker 20324 None None None 2017-06-15 18:29 EDT
Red Hat Knowledge Base (Solution) 2971581 None None None 2017-10-24 11:21 EDT
Red Hat Knowledge Base (Solution) 3210951 None None None 2017-10-09 18:14 EDT

  None (edit)
Comment 3 Vikhyat Umrao 2017-09-29 10:57:25 EDT
jewel rocksdb backport: https://github.com/ceph/ceph/pull/18010
Comment 4 Josh Durgin 2017-10-02 11:41:33 EDT
Doing some final scale testing this week before pulling this in.
Comment 27 Warren 2017-10-12 21:29:52 EDT
I am marking this verified because it works on a newly installed system.

However on an osd after I performed the operations in step 10, I then tried to run the new compact command for ceph (https://bugzilla.redhat.com/show_bug.cgi?id=1483224).  I got:

[root@magna007 ceph-5]# ceph daemon osd.5 compact
admin_socket: exception getting command descriptions: [Errno 2] No such file or directory
admin_socket: exception getting command descriptions: [Errno 2] No such file or directory

I am not sure if this is a problem or not.  I am also updating bz 1483224 as well.
Comment 28 Warren 2017-10-12 23:36:17 EDT
This does not seem to work on Ubuntu.  After running the steps in Comment 10:

-1 8.13950 root default                                        
-2 2.71317     host magna038                                   
 0 0.90439         osd.0          up  1.00000          1.00000 
 3 0.90439         osd.3          up  1.00000          1.00000 
 6 0.90439         osd.6          up  1.00000          1.00000 
-3 2.71317     host magna096                                   
 1 0.90439         osd.1          up  1.00000          1.00000 
 4 0.90439         osd.4          up  1.00000          1.00000 
 7 0.90439         osd.7          up  1.00000          1.00000 
-4 2.71317     host magna024                                   
 2 0.90439         osd.2          up  1.00000          1.00000 
 5 0.90439         osd.5        down        0          1.00000 
 8 0.90439         osd.8          up  1.00000          1.00000
Comment 31 Warren 2017-10-16 16:19:35 EDT
Passed on both Ubuntu and Rhel.  I ran into a slight gotcha in that I ran some of the commands as root rather than the ceph user.  Josh helped me a bit and things worked after we changed permissions and ran as the correct user.  This comment also applies to bugzilla #1483224
Comment 36 errata-xmlrpc 2017-10-17 14:12:51 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

Comment 38 Vikhyat Umrao 2017-10-24 11:21:48 EDT
*** Bug 1480720 has been marked as a duplicate of this bug. ***

Note You need to log in before you can comment on or make changes to this bug.