Created attachment 1789117 [details]
See 2021-06-06 12:11:47,842

Description of problem:
switch-to-containerised fails on RHCS-4 when the following is not set in all.yml:
ceph_docker_registry: "registry.redhat.io"
ceph_docker_registry_auth: true
ceph_docker_registry_username:
ceph_docker_registry_password:

But it does not fail until after the non-containerized mon service have been removed.
This results in the cluster missing a monitor and the playbook fails on subsequent runs because it can't find the removed mon service

Version-Release number of selected component (if applicable):
ceph-ansible.noarch                  4.0.41-1.el7cp          @rhel-7-server-rhceph-4-tools-rpms


How reproducible:
I deploye RHCS-3, non-containerized, upgraded to RHCS-4 non-containerized, followed by running infrastructure-playbooks/switch-from-non-containerized-to-containerized-ceph-daemons.yml without adding the above-mentioned ceph_docker_registry variables.

Steps to Reproduce:
1. Install RHCS-3 non-containerized
2. Upgrade to RHCS-4
3. Convert to containerized, by running infrastructure-playbooks/switch-from-non-containerized-to-containerized-ceph-daemons.yml with -limit rhceph01 # rhceph01 first monitor

Actual results:
mon on rhceph01 is removed and cluster is left with 2 functioning mon's and health_warn

Expected results:
Early fail of playbook, with a message pointing out that registry.redhat.io requires said variables to be set, while keeping the cluster HEALTH_OK

Additional info:
Look at line: 2021-06-06 12:11:47,842 in the attached ansible.log