The fix for `coreos-assembler:rhcos-4.6` landed in https://github.com/coreos/coreos-assembler/pull/2733 I've confirmed the latest machine-os-content inages for all arches that were pushed to Quay are in v2s2 format.
nightly builds from March 9 2022 or later should have the proper docker v2s2 format
This was addressed in 4.6.56 however the errata automation failed to close this bug as it should have.