Bug 1967964

It seems there is the possibility that after the upgrade to RHCS4 the cluster remains in HEALTH_WARN state with:

  crush map has legacy tunables (require firefly, min is hammer)

This is because in nautilus the default value for mon_crush_min_required_version has been changed from firefly to hammer, which means the cluster will issue a health warning if the CRUSH tunables are older than hammer. The upstream guide [1] says thas:

"""
If your CRUSH tunables are older than Hammer, Ceph will now issue a health warning. If you see a health alert to that effect, you can revert this change with:

ceph config set mon mon_crush_min_required_version firefly

If Ceph does not complain, however, then we recommend you also switch any existing CRUSH buckets to straw2, which was added back in the Hammer release. If you have any ‘straw’ buckets, this will result in a modest amount of data movement, but generally nothing too severe.:

ceph osd getcrushmap -o backup-crushmap
ceph osd crush set-all-straw-buckets-to-straw2

If there are problems, you can easily revert with:

ceph osd setcrushmap -i backup-crushmap

Moving to ‘straw2’ buckets will unlock a few recent features, like the crush-compat balancer mode added back in Luminous.
"""

There is generally a small (but non-zero) amount of data that will move around by making the switch to hammer tunables; for more information, see Tunables.

1. https://docs.ceph.com/en/latest/releases/nautilus/#v14-2-1-nautilus