Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
This project is now read‑only. Starting Monday, February 2, please use https://ibm-ceph.atlassian.net/ for all bug tracking management.

Bug 1591791

Summary: ceph-ansible 3.1 deployment does not allow lvm based osd_scenario configured for bluestore.
Product: [Red Hat Storage] Red Hat Ceph Storage Reporter: Deepthi Dharwar <ddharwar>
Component: Ceph-AnsibleAssignee: Sébastien Han <shan>
Status: CLOSED NEXTRELEASE QA Contact: ceph-qe-bugs <ceph-qe-bugs>
Severity: high Docs Contact:
Priority: unspecified    
Version: 3.1CC: acalhoun, aschoen, bengland, ceph-eng-bugs, ddharwar, gmeno, nthomas, sankarshan
Target Milestone: rc   
Target Release: 3.*   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-06-15 20:07:12 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Deepthi Dharwar 2018-06-15 14:54:51 UTC
Description of problem:
RHCS 3.1 has a check in roles/ceph-osd/tasks/check_mandatory_vars.yml which prevents having "lvm" osd_scenario configured for Bluestore.

msg: "the lvm osd_scenario currently only works for filestore, not bluestore"

RHCS 3.1 is backed to Luminous, which supports Bluestore with lvm scenarios.
This check should not be there.


Version-Release number of selected component (if applicable):
3.1

How reproducible:
Run ceph-ansible with lvm osd_scenario and Bluestore enabled.
It fails when deploying OSD.


Actual results: It fails to deploy ceph OSD.


Expected results:
It should run as RHCS 3.1 is backed to Luminous. Removing this check from check-mandatory_vars.yml works well.

Additional info:

Comment 3 Sébastien Han 2018-06-15 20:07:12 UTC
This is fixed in 3.1, with https://github.com/ceph/ceph-ansible/commit/d3b427e16990f9ebcde7575aae367fd7dfe36a8d#diff-34d2eea5f7de9a9e89c1e66b15b4cd0a

I'm closing this as this wasn't really a bug.
Thanks.

Comment 4 acalhoun 2018-06-15 20:32:33 UTC
After commenting out this piece of ansible code, ceph-ansible was still unable to install ceph. 

it appears as that if one sets osd scenario to LVM ceph-ansible assumes the object store is filestore.

Has ceph-ansible been update to address the below error as part of the "fix"?

Software version used:

RHCEPH-3.0-RHEL-7-20180604.ci.0-x86_64-dvd.iso from http://download-node-02.eng.bos.redhat.com/composes/auto/ceph-3.0-rhel-7/RHCEPH-3.0-RHEL-7-20180604.ci.0/compose/OSD/x86_64/iso/


Error from logs and config below:


 TASK [ceph-osd : use ceph-volume to create filestore osds with dedicated journals] ***
fatal: [b06-h02-1029u.rdu.openstack.engineering.redhat.com]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'journal'\n\nThe error appears to have been in '/usr/share/ceph-ansible/roles/ceph-osd/tasks/scenarios/lvm.yml': line 2, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n---\n- name: use ceph-volume to create filestore osds with dedicated journals\n  ^ here\n\nexception type: <class 'ansible.errors.AnsibleUndefinedVariable'>\nexception: 'dict object' has no attribute 'journal'"}
fatal: [b06-h03-1029u.rdu.openstack.engineering.redhat.com]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'journal'\n\nThe error appears to have been in '/usr/share/ceph-ansible/roles/ceph-osd/tasks/scenarios/lvm.yml': line 2, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n---\n- name: use ceph-volume to create filestore osds with dedicated journals\n  ^ here\n\nexception type: <class 'ansible.errors.AnsibleUndefinedVariable'>\nexception: 'dict object' has no attribute 'journal'"}
fatal: [b06-h05-1029u.rdu.openstack.engineering.redhat.com]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'journal'\n\nThe error appears to have been in '/usr/share/ceph-ansible/roles/ceph-osd/tasks/scenarios/lvm.yml': line 2, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n---\n- name: use ceph-volume to create filestore osds with dedicated journals\n  ^ here\n\nexception type: <class 'ansible.errors.AnsibleUndefinedVariable'>\nexception: 'dict object' has no attribute 'journal'"}
fatal: [b06-h06-1029u.rdu.openstack.engineering.redhat.com]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'journal'\n\nThe error appears to have been in '/usr/share/ceph-ansible/roles/ceph-osd/tasks/scenarios/lvm.yml': line 2, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n---\n- name: use ceph-volume to create filestore osds with dedicated journals\n  ^ here\n\nexception type: <class 'ansible.errors.AnsibleUndefinedVariable'>\nexception: 'dict object' has no attribute 'journal'"}


all.yml
---
dummy:
fetch_directory: /ceph-ansible-keys
ceph_rhcs_iso_install: true
ceph_origin: repository
ceph_repository: rhcs
ceph_rhcs_version: 3
valid_ceph_repository_type:
  - iso
ceph_rhcs_iso_path: $HOME/automated_ceph_test/staging_area/rhcs_latest/$ceph_iso_file
monitor_interface: enp94s0f0
journal_size: 5120 # OSD journal size in MB
public_network: 172.16.76.0/16
cluster_network: 172.17.76.0/16
osd_objectstore: bluestore
ceph_conf_overrides:
  mon:
    mon_allow_pool_delete: true
  osd:
    bluestore_cache_size: 46170898432
    bluestore_cache_kv_max: 214748364800
    bluestore_cache_kv_ratio: 0.7
    bluestore_cache_meta_ratio: 0.1

osd.yml
---
dummy:
osd_scenario: lvm

lvm_volumes:
    - data: data-lv1
      data_vg: vg_nvme0n1

    - data: data-lv2
      data_vg: vg_nvme0n1

    - data: data-lv1
      data_vg: vg_nvme1n1

    - data: data-lv2
      data_vg: vg_nvme1n1

    - data: data-lv1
      data_vg: vg_nvme2n1

    - data: data-lv2
      data_vg: vg_nvme2n1

    - data: data-lv1
      data_vg: vg_nvme3n1

    - data: data-lv2
      data_vg: vg_nvme3n1