Bug 2083126

Description of problem
======================

When one tries to use ceph toolbox pod on an external cluster, the ceph tools
won't connect to the ceph cluster since cephx keys haven't been configured.

See also: BZ 2066906 - ceph command failed with error: RADOS permission error on consumer cluster

It seems reasonable to me that ODF should handle setup of authentication (cephx
key pairs) for ceph toolbox to work. This could include extensions in cluster
importer script, and in ocs operator deploying the toolbox.

Version-Release number of selected component
============================================

ODF 4.10

How reproducible
================

100%

Steps to Reproduce
==================

1. Install OCP cluster

2. Install ODF operator and create storage system importing already existing
   ceph cluster (via OCP Console web ui)

3. Deploy ceph toolbox pod:

```
$ oc patch ocsinitialization ocsinit -n openshift-storage --type json --patch '[{ "op": "replace", "path": "/spec/enableCephTools", "value": true }]'
```

4. Try to use the toolbox:

```
$ oc get pods -n openshift-storage | grep ceph-tools
$ oc rsh -n openshift-storage rook-ceph-tools-foo-bar bash
[root@compute-0 /]# ceph -s
```

Actual results
==============

```
[root@compute-0 /]# ceph -s
[errno 1] RADOS permission error (error connecting to the cluster)
```

Expected results
================

Ceph tools can authenticate the the cluster out of the box:

```
[root@compute-0 /]# ceph -s
  cluster:
    id:     6f73f37c-863f-4851-94ca-ea7fbd5eb044
    health: HEALTH_OK

  services:
    mon: 3 daemons, quorum a,b,c (age 3d)
    mgr: a(active, since 3d)
    mds: rook-ceph-cephfilesystem:1 {0=rook-ceph-cephfilesystem-a=up:active} 1 up:standby-replay
    osd: 3 osds: 3 up (since 3d), 3 in (since 3d)

  data:
    pools:   4 pools, 128 pgs
    objects: 22 objects, 2.2 KiB
    usage:   12 GiB used, 285 GiB / 297 GiB avail
    pgs:     128 active+clean

  io:
    client:   767 B/s rd, 1 op/s rd, 0 op/s wr
```

Workaround
==========

To use ceph toolbox with external ceph cluster, one have to configure cephx
authentication between ceph toolbox pod and the ceph cluster manually. See:

- https://bugzilla.redhat.com/show_bug.cgi?id=2066906#c3
- https://docs.ceph.com/en/latest/rados/operations/user-management/