Bug 2139886
| Summary: | [RFE] Adding the crush_location label to monitors with Cephadm. Required for Stretched RHCS | ||
|---|---|---|---|
| Product: | [Red Hat Storage] Red Hat Ceph Storage | Reporter: | daniel parkes <dparkes> |
| Component: | Cephadm | Assignee: | Adam King <adking> |
| Status: | CLOSED DUPLICATE | QA Contact: | Manasa <mgowri> |
| Severity: | high | Docs Contact: | Anjana Suparna Sriram <asriram> |
| Priority: | unspecified | ||
| Version: | 6.0 | CC: | cephqe-warriors, pdhiran, vereddy |
| Target Milestone: | --- | Keywords: | FutureFeature |
| Target Release: | 6.1 | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2022-11-07 03:34:11 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Please specify the severity of this bug. Severity is defined here: https://bugzilla.redhat.com/page.cgi?id=fields.html#bug_severity. *** This bug has been marked as a duplicate of bug 2016889 *** |
Description of problem: When deploying RHCS in stretched mode, the monitors need to have a crush location label to identify the site there are in, for example: $ ceph mon dump | grep location 0: [v2:10.70.56.157:3300/0,v1:10.70.56.157:6789/0] mon.ceph-0; crush_location {datacenter=DC1} 1: [v2:10.70.56.125:3300/0,v1:10.70.56.125:6789/0] mon.ceph-1; crush_location {datacenter=DC1} 2: [v2:10.70.56.175:3300/0,v1:10.70.56.175:6789/0] mon.ceph-3; crush_location {datacenter=DC2} 3: [v2:10.70.56.228:3300/0,v1:10.70.56.228:6789/0] mon.ceph-4; crush_location {datacenter=DC2} 4: [v2:10.70.56.111:3300/0,v1:10.70.56.111:6789/0] mon.ceph-6; crush_location {datacenter=DC3} We currently add these labels using ceph commands as a day 2 operation: $ ceph mon set_location ceph1 datacenter=DC1 Also, when replacing monitors with Ceph in stretched mode, you need to have the location label in place, or the Monitor will refuse to start: $ ceph orch daemon add mon ceph03 In the monitor logs, you get the following entries: cluster [INF] ceph03 attempted to join from ceph03 [v2:192.168.1.207:3300/0,v1:192.168.1.207:6789/0]; but lacks a crush_location for stretch mode We are currently working around this by updating the monmap with the new monitor: $ ceph mon add ceph03 192.168.1.207 datacenter=DC3 $ ceph orch daemon add mon ceph03 But this is not ideal, and it would be great if we could pass the crush_location label to the monitor with cephadm. I would mention that monitor crush_location is a label the monitors use. It's not related to the OSD crush map, so I would plan to add this feature for the monitors independently from the location spec we use for the service_type: host, as they could be completely different values. Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info: