Bug 1388943

(In reply to jquinn from comment #0)
> Description of problem:The Admin Guide references having the journal on a
> separate drive, but does not provide the steps to configure the OSD's this
> way.  Reference Section 6.3.2 and 6.3.3, both ansible and CLI should have
> the steps listed.
> 

https://access.redhat.com/webassets/avalon/d/Red_Hat_Ceph_Storage-2-Administration_Guide-en-US/Red_Hat_Ceph_Storage-2-Administration_Guide-en-US.pdf

> 
> Version-Release number of selected component (if applicable):RHCS 2.0
> 
> 
> How reproducible:
> 
> 
> Steps to Reproduce:
> 1.
> 2.
> 3.
> 
> Actual results:
> 
> 
> Expected results:Would like to have the steps added to the Documentation.  I
> have not located the steps on any internal documents. 
> 
> 
> Additional info:

Added link to the Admin Guide

Hi Bara, 

Sorry for the delay, I must have missed that variable in the ansible guide.  That looks good, i'll have to run through it at some point to test everything out.  

As for the CLI, when I went through this in my lab I followed the below guide from ceph.com to create the osd's with seperate journal.  These would most likely be the steps that we should have in the docs on the customer portal as well. 

http://docs.ceph.com/docs/jewel/rados/deployment/ceph-deploy-osd/

Let me know if you need further information on creating the OSD's from CLI

Thanks, 
Joe

Hi Bara,

I tried same steps mentioned in the doc but couldn't add a new OSD. I got error message '** ERROR: unable to open OSD superblock on /var/lib/ceph/osd/ceph-3: (13) Permission denied' on logs, when it failed to start osd service. Apart from this, doc needs some small changes as listed below. Can you please have a look ?


A. In step 2 of subsection 'Before you Start'

1) If convention of command template is followed, a new file will be created in /etc file in the name 'ceph' and keyring/config file gets copied to the file.
 
# scp root@vm1:/etc/ceph/ceph.conf /etc/ceph
root@vm1's password: 
ceph.conf                                                                                    100%  970     1.0KB/s   00:00  
# ls -l /etc/ceph
-rw-r--r--. 1 root root 970 Feb 13 10:18 /etc/ceph

2) Using example commands will result in error as copied below

# ls -l /etc |grep ceph
# scp root@vm1:/etc/ceph/ceph.client.admin.keyring /etc/ceph/
root@vm1's password: 
/etc/ceph/: Is a directory


So, I think an extra command to create ceph directory in /etc has to be given before command given for copying keyring.

[root@vm5 ~]# mkdir /etc/ceph
[root@vm5 ~]# scp root@vm1:/etc/ceph/ceph.client.admin.keyring /etc/ceph
root@vm1's password:
ceph.client.admin.keyring                                                                    100%   63     0.1KB/s   00:00   
[root@vm5 ~]# ls /etc/ceph
ceph.client.admin.keyring   


B. In subsection Initializing the OSD Data and Journal Directory and Registering the OSD Authentication Key, command given to Create the journal on the journal disk (step iii of step 2) needs to be changed

Given -

ceph-osd -i <osd-id> —mkjournal
For example:
# ceph-osd -i 4 -mkjournal

Appropriate version -

ceph-osd -i <osd-id> --mkjournal
For example:
# ceph-osd -i 4 --mkjournal


C. Step 5 of subsection Preparing the OSD Data and Journal Drives needs a small change, which is being tracked under BZ 1379188


Regards,
Vasishta

Hi Bara,

'root' is the owner of the log and SELinux was enabled.
Just before trying to start OSD, I tried changing the defaultlabel as you mentioned, that didn't work.
Just for the information, I'm trying on VMs. 

Regards,
Vasishta

Hi Bara,

Sorry for being late. Yes, updating owner permission was the thing I had done wrong by getting confused with owner name and cluster user name.Sorry.

Following is a list of of steps where '--cluster <cluster_name>' option is needed if user is going to use a cluster of custom cluster name. Either this argument can be added in both syntax and command or a note can be given as it has been done for step 1 of subsection initializing-the-osd-data-directory-and-registering-the-osd-authentication-key or the note can be generalized. 


1) Step 3 of subsection installing-ceph-osd-and-creating-a-new-osd-instance.
2) Step iii of step 2 in initializing-the-osd-data-directory-and-registering-the-osd-authentication-key.
3) Step 3 of initializing-the-osd-data-directory-and-registering-the-osd-authentication-key.
4) Syntax and Example of adding-the-new-osd-node-to-the-crush-map

Please let me know if there are any concerns or issues.

Regards,
Vasishta