Bug 2151189

Summary:	[cephadm] DriveGroup can't handle multiple crush_device_classes
Product:	[Red Hat Storage] Red Hat Ceph Storage	Reporter:	Francesco Pantano <fpantano>
Component:	Cephadm	Assignee:	Adam King <adking>
Status:	CLOSED ERRATA	QA Contact:	Manisha Saini <msaini>
Severity:	medium	Docs Contact:	Akash Raj <akraj>
Priority:	unspecified
Version:	5.0	CC:	adking, akraj, alfrgarc, cephqe-warriors, gfidente, jlabarre, johfulto, ltoscano, vereddy
Target Milestone:	---
Target Release:	6.1
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	ceph-17.2.6-5.el9cp	Doc Type:	Enhancement
Doc Text:	.`crush_device_class` can now be specified per path in an OSD specification With this release, to allow users more flexibility with `crush_device_class` settings when deploying OSDs through Cephadm, `crush_device_class`, you can specify per path inside an OSD specification. It is also supported to provide these per-path `crush_device_classes` along with a service-wide `crush_device_class` for the OSD service. In cases of service-wide `crush_device_class`, the setting is considered as default, and the path-specified settings take priority. .Example ---- service_type: osd service_id: osd_using_paths placement: hosts: - Node01 - Node02 crush_device_class: hdd spec: data_devices: paths: - path: /dev/sdb crush_device_class: ssd - path: /dev/sdc crush_device_class: nvme - /dev/sdd db_devices: paths: - /dev/sde wal_devices: paths: - /dev/sdf ----	Story Points:	---
Clone Of:		Environment:
Last Closed:	2023-06-15 09:16:25 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	2180567
Bug Blocks:	2071977, 2192813

Description Francesco Pantano 2022-12-06 10:02:39 UTC

With ceph-ansible is possible to have the following OSD definition:

CephAnsibleDisksConfig:
  lvm_volumes:
    - data: '/dev/vdx'
      crush_device_class: 'ssd'
    - data: '/dev/vdz'
      crush_device_class: 'hdd'

which allows to test in the OpenStack CI both crush hierarchy and the rules associated to the defined pools.
With cephadm, --crush_device_class is a global option for ceph-volume, which prepares and activates disks in batch mode.

We should extend the DriveGroup "paths" definition within the OSDspec to allow something like:

data_devices:
  paths:
    - data: /dev/ceph_vg/ceph_lv_data
      crush_device_class: ssd
    - data: /dev/ceph_vg/ceph_lv_data2
      crush_device_class: hdd
    - data: /dev/ceph_vg/ceph_lv_data3
      crush_device_class: hdd

and make ceph-volume able to prepare single osds with an associated `crush_device_class`

Comment 4 John Fulton 2023-01-09 13:56:37 UTC

To verify this bug with "internal ceph" in OSP17, create an osd_spec.yaml file containing:

data_devices:
  paths:
    - data: /dev/ceph_vg/ceph_lv_data
      crush_device_class: ssd
    - data: /dev/ceph_vg/ceph_lv_data2
      crush_device_class: hdd
    - data: /dev/ceph_vg/ceph_lv_data3
      crush_device_class: hdd

and when you run `openstack overcloud ceph deploy` pass it with `--osd-spec osd_spec.yaml` as described here:

  https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/features/deployed_ceph.html#overriding-which-disks-should-be-osds

Comment 13 Manisha Saini 2023-04-26 20:15:54 UTC

Based on comment#12, Moving this BZ to verified state.

Comment 17 errata-xmlrpc 2023-06-15 09:16:25 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: Red Hat Ceph Storage 6.1 security and bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2023:3623