Description of problem:

When configuring a multisite deployment, it is quite common practice to dedicate specific RGW services to the client IO, and other RGW services to take care of the multisite replication.

By default all RGWs participate in multisite replication, there are two steps needed to remove and RGW for participating in the multisite replication sync.

1. Set the ceph parameter on the RGW client:  ceph config set ${KEY_ID} rgw_run_sync_thread false , when false, prevents this object store''s gateways from transmitting multisite replication data

2. The previous parameter only tells the RGW not to SEND replication data, but it can keep receiving, to also avoid receiving we need to remove the RGWs from the zonegroup and zone replication endpoints.

From Cephadm point of view, to provide a way to configure this in the spec file it would be enough to address point 1, to add a new parameter in the spec file, something like the rook example: 

"disableMultisiteSyncTraffic: false"

There is also an example in rook around the same topic: https://github.com/rook/rook/pull/12327/files#diff-471e0bbca5b3e456a7a2fca6611d7c235e584dc0a297a00d40743ec571ddb393R211

For the second point I think we could just need to document, that the rgws with sync disable should be removed from the zone, zonegroup endpoints, like: https://github.com/rook/rook/pull/12327/files#diff-35a70e8ee5bd35588349525fa34d27ca5f1ceea83477d0275cb4d889b6d6a0cdR6001.