Bug 1463787 - Ceph data and journal disk declaration enhancement in osds.yml or host_vars files
Ceph data and journal disk declaration enhancement in osds.yml or host_vars f...
Status: ASSIGNED
Product: Red Hat Ceph Storage
Classification: Red Hat
Component: Ceph-Ansible (Show other bugs)
3.0
All Linux
medium Severity medium
: rc
: 3.1
Assigned To: leseb
ceph-qe-bugs
: FutureFeature
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2017-06-21 14:31 EDT by Vikhyat Umrao
Modified: 2017-09-21 19:10 EDT (History)
10 users (show)

See Also:
Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Vikhyat Umrao 2017-06-21 14:31:28 EDT
Description of problem:
[RFE] Ceph data and journal disk declaration enhancement in osds.yml or host_vars files

For now, we do not have like this. It is all in one section, as given here: https://access.redhat.com/documentation/en-us/red_hat_ceph_storage/2/html-single/installation_guide_for_red_hat_enterprise_linux/#configuring_ceph_osd_settings

devices:
<osd data disk1>
<osd data disk2>
<osd data disk3>
<osd data disk4>
[...]
<osd data diskN>

raw_journal_devices:
<osd journal disk1>
<osd journal disk1>
<osd journal disk2>
<osd journal disk2>
[....]
<osd journal diskN>

So osd data disk1 and 2 would be in osd journal disk 1 partition 1 and 2 and osd data disk3 and 4 would be in osd journal disk 2 partitions 1 and 2. Right now we identify mapping like this only.

This causes issues when we remove OSDs and change the mapping with journals. 
It would be great if we can define OSD data and journal devices in the same section as the ceph-disk list does.

For example, maybe we can have like this:

devices:

devices:
<osd data disk1>
  <osd journal disk1>
<osd data disk2>
  <osd journal disk1>
<osd data disk3>
  <osd journal disk2>
<osd data disk4>
  <osd journal disk2>
[...]
<osd data diskN>
  <osd journal diskN>

If possible it would be better if we can specify journal partition number instead of repeating same disk name again and again for that many numbers of OSD journal in that journal disk.

devices:
<osd data disk1>
  <osd journal disk1 partition 1>
<osd data disk2>
  <osd journal disk1 partition 2>
<osd data disk3>
  <osd journal disk2 partition 1>
<osd data disk4>
  <osd journal disk2 partition 2>
[...]
<osd data diskN>
  <osd journal diskN partition N>


Version-Release number of selected component (if applicable):
Red Hat Ceph Storage 2.3
Comment 1 Tupper Cole 2017-06-22 16:32:03 EDT
For context- currently osds and journals are laid out like so:

devices:
  - /dev/sdc
  - /dev/sdd
  - /dev/sde
  - /dev/sdf
  - /dev/sdg
  - /dev/sdh

raw_journal_devices:
  - /dev/sdi
  - /dev/sdi
  - /dev/sdi
  - /dev/sdj
  - /dev/sdj
  - /dev/sdj


Which works fine so long as the devices and journals line up ok. In the case of a disk out, like:

devices:
  - /dev/sdc
#  - /dev/sdd
  - /dev/sde
  - /dev/sdf
  - /dev/sdg
  - /dev/sdh

running the playbook fails like:

2017-06-22 19:27:27,478 p=28798 u=a_ansible |  fatal: [osd-03.ceph-ci.cloudlab.jpmchase.net]: FAILED! => {"failed": true, "msg": "The conditional check 'item.0.get(\"skipped\") or item.0.get(\"rc\", 0) != 0' failed. The error was: error while evaluating conditional (item.0.get(\"skipped\") or item.0.get(\"rc\", 0) != 0): 'None' has no attribute 'get'\n\nThe error appears to have been in '/usr/share/ceph-ansible/roles/ceph-osd/tasks/scenarios/raw_multi_journal.yml': line 10, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: prepare filestore osd disk(s) with a dedicated journal device\n  ^ here\n"}

Wouldn't it make more sense to:

A)handle the array better so it doesn't fail

and 

B)use similar syntax to ceph-deploy? 

devices:
  - /dev/sd{c..e}:/dev/sdi
  - /dev/sd{f..h}:/dev/sdj
Comment 6 seb 2017-07-07 10:44:26 EDT
Not possible with Ansible, this can potentially be solved by this: https://github.com/ceph/ceph-ansible/pull/1558

We can not commit to this for 3.0.

Note You need to log in before you can comment on or make changes to this bug.