Description of problem:

When deploying RHCS in stretched mode, the monitors need to have a crush location label to identify the site there are in, for example:

$ ceph mon dump | grep location
0: [v2:10.70.56.157:3300/0,v1:10.70.56.157:6789/0] mon.ceph-0; crush_location {datacenter=DC1}
1: [v2:10.70.56.125:3300/0,v1:10.70.56.125:6789/0] mon.ceph-1; crush_location {datacenter=DC1}
2: [v2:10.70.56.175:3300/0,v1:10.70.56.175:6789/0] mon.ceph-3; crush_location {datacenter=DC2}
3: [v2:10.70.56.228:3300/0,v1:10.70.56.228:6789/0] mon.ceph-4; crush_location {datacenter=DC2}
4: [v2:10.70.56.111:3300/0,v1:10.70.56.111:6789/0] mon.ceph-6; crush_location {datacenter=DC3}

We currently add these labels using ceph commands as a day 2 operation:

$ ceph mon set_location ceph1 datacenter=DC1

Also, when replacing monitors with Ceph in stretched mode, you need to have the location label in place, or the Monitor will refuse to start:

$ ceph orch daemon add mon ceph03

In the monitor logs, you get the following entries:

cluster [INF] ceph03 attempted to join from ceph03 [v2:192.168.1.207:3300/0,v1:192.168.1.207:6789/0]; but lacks a crush_location for stretch mode

We are currently working around this by updating the monmap with the new monitor:

$ ceph mon add ceph03 192.168.1.207 datacenter=DC3
$ ceph orch daemon add mon ceph03

But this is not ideal, and it would be great if we could pass the crush_location label to the monitor with cephadm.

I would mention that monitor crush_location is a label the monitors use. It's not related to the OSD crush map, so I would plan to add this feature for the monitors independently from the location spec we use for the service_type: host, as they could be completely different values.


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info: