Description of problem: We've noticed that when doing an update on a heat stack in the Overcloud, changing the user_data user_data[1] property of a OS::Nova::Server, the server get re-created. And that's in- line with expectations. However, what we've noticed is that Heat will _first_ create a new VM and only then destroy the old one. And during this process, we are in risk of running out of CPU cores. Currently, the user_data_update_policy property can be set to either REPLACE (default) or IGNORE. What we would like to see is to have a third option REBUILD which will cause the server to rebuild, similar to what happens when the image or flavor is changed. This will then not require additional computing resources during the stack update. [1] https://docs.openstack.org/heat/queens/template_guide/openstack.html#OS::Nova::Server-prop-user_data
This sounds entirely reasonable, and according to https://docs.openstack.org/nova/latest/contributor/evacuate-vs-rebuild.html#high-level should work fine starting with microversion 2.57. In fact, I'd go so far as to say that if we were starting today this should be the default, and that changes of the key_name should also initiate a rebuild instead of replacement. I don't know how we go about getting there without breaking backwards-compat expectations though.
In regards to user_data, I envision creating REBUILD as a (non-default) option for user_data_update_policy, this shouldn't break compatibility, should it? Regarding key_name, how about addgin key_name_update_policy ? Or am I missing something?
Fixing flags and moving the target to 16.2 as this is a new feature and behavior that will probably not be backported.
Moving to 17 as it didn't made the deadline.