Describe the issue ================== Ceph deployment via cephadm as explained in the text is performing both bootstrap and cluster configuration in one step, which may be useful in some cases, but in the text it's done in both highly non optimal and confusing way. As a consequence, the procedure in the text requires additional relocation of ceph services. Such step should not be necessary. Describe the task you were trying to accomplish =============================================== Deploy a stretched ceph cluster for MetroDR. Suggestions for improvement =========================== Better align with and reference to ceph docs. Distinguish bootstrap and cluster setup as 2 separate steps. Moreover we can use service configuration yaml file to allocate ceph services in the cluster so that it matches the table 3.1 (physical server locations and Ceph component layout for Red Hat Ceph Storage cluster deployment) and whole chapter "3.6. Relocating the monitors and managers to an appropriate location" could be dropped as it won't be necessary. Document URL ============ https://access.redhat.com/documentation/en-us/red_hat_openshift_data_foundation/4.10/html/configuring_openshift_data_foundation_for_metro-dr_with_advanced_cluster_management/requirements-for-deploying-rhcs-stretch-cluster_rhodf Chapter/Section Number and Title ================================ 3.5. Cluster bootstrapping with Cephadm 3.6. Relocating the monitors and managers to an appropriate location Product Version =============== ODF 4.10 Environment Details =================== MetroDR with ACM: deployment with stretched RHCS cluster. Any other versions of this document that also needs this update =============================================================== This text was introduced in 4.10, and should be fixed at least in 4.11. Additional information ====================== An example of service configuration yaml which can be used to setup cluster after bootstrap is done: ``` service_type: host hostname: osd-0.mbukatov-ceph01.qe.example.com addr: 10.1.161.15 labels: - osd - mon --- service_type: host hostname: osd-1.mbukatov-ceph01.qe.example.com addr: 10.1.160.255 labels: - osd - mon --- service_type: host hostname: osd-2.mbukatov-ceph01.qe.example.com addr: 10.1.161.36 labels: - osd - mds - rgw --- service_type: host hostname: osd-3.mbukatov-ceph01.qe.example.com addr: 10.1.161.37 labels: - osd - mon --- service_type: host hostname: osd-4.mbukatov-ceph01.qe.example.com addr: 10.1.161.40 labels: - osd - mon --- service_type: host hostname: osd-5.mbukatov-ceph01.qe.example.com addr: 10.1.161.39 labels: - osd - mds - rgw --- service_type: host hostname: arbiter.mbukatov-ceph01.qe.example.com addr: 10.1.161.33 labels: - mon --- service_type: osd service_id: default_drive_group data_devices: limit: 2 placement: label: "osd" --- service_type: mon placement: label: "mon" --- service_type: mds service_id: fs_name placement: label: "mds" ``` Note that labels are only one option how to allocate ceph services in a cluster, but here it seems to match nicely with the table 3.1. We can always reference ceph docs stating that there are other options.
During today's discussion among Daniel, Olive and me, I agreed that separating bootstrap and cluster setup as 2 steps won't be necessary as long as we point out that we are actually doing these 2 operations at once, referencing related RHCS docs. This way, the reader would still easily see the relation between the setup in MetroDR text and ceph docs, while we keep the MetroDR clear and short.
As noted in comment 2, we actually decided to keep combined bootstrap and deployment, but we also wanted to mention this decision in the text which I don't see there. Few sentences in the beginning of section 3.5 would be enough.
Re-targeting the verification part
Looking at the preview build (commit 4e13663): I see that the beginning of section "4.5.2. Cluster bootstrapping and service deployment with Cephadm" explains the bootstrap and service deployment well. >>> VERIFIED