Description of problem:

Not able to migrate from ceph filestore to bluestore while performing an upgrade from OSP13 to OSP16.1.

We have applied the patch[1] per discussion in upgrade hackfest, performed the stack update[2] and ran "external-upgrade" with the "ceph_fstobs" command[3]. All the steps are getting executed successfully but it doesn't seem playbook[3] is honoring current stored "osd_objectstore" value.


[1] https://review.opendev.org/#/c/733812
[2] openstack overcloud deploy --stack-only --templates /usr/share/openstack-tripleo-heat-templates --stack msufiyan -e .... -e ...
[3] openstack overcloud external-upgrade run --stack msufiyan --tags ceph_fstobs -e ceph_ansible_limit=computehci-0 | tee oc-fstobs.lo
[4] /usr/share/ceph-ansible/infrastructure-playbooks/filestore-to-bluestore.yml:

~~~
(undercloud) [stack@undercloud-0 upgrade_commands]$ openstack stack environment show  msufiyan|grep -i osd_objectstore
    osd_objectstore: filestore
~~~

~~~
2020-07-03 15:39:07,106 p=889157 u=root n=ansible | ok: [computehci-0 -> 192.168.24.35] => (item=controller-2)
2020-07-03 15:39:07,283 p=889157 u=root n=ansible | TASK [set_fact current_objectstore] ********************************************
2020-07-03 15:39:07,283 p=889157 u=root n=ansible | task path: /usr/share/ceph-ansible/infrastructure-playbooks/filestore-to-bluestore.yml:24
2020-07-03 15:39:07,284 p=889157 u=root n=ansible | Friday 03 July 2020  15:39:07 +0000 (0:00:07.210)       0:00:07.500 ***********
2020-07-03 15:39:07,304 p=889157 u=root n=ansible | ok: [computehci-0] => {"ansible_facts": {"current_objectstore": "bluestore"}, "changed": false}
2020-07-03 15:39:07,354 p=889157 u=root n=ansible | TASK [warn user about osd already using bluestore] *****************************
2020-07-03 15:39:07,354 p=889157 u=root n=ansible | task path: /usr/share/ceph-ansible/infrastructure-playbooks/filestore-to-bluestore.yml:28
2020-07-03 15:39:07,354 p=889157 u=root n=ansible | Friday 03 July 2020  15:39:07 +0000 (0:00:00.070)       0:00:07.570 ***********
2020-07-03 15:39:07,375 p=889157 u=root n=ansible | ok: [computehci-0] => {
    "msg": "WARNING: computehci-0 is already using bluestore. Skipping all tasks."   <<<<----- Execution is getting skipped here.
}
~~~


Version-Release number of selected component (if applicable):
(undercloud) [stack@undercloud-0 upgrade_commands]$ rpm -qa | grep -i ceph-ansible
ceph-ansible-4.0.23-1.el8cp.noarch