Description of problem: Since bz 1508689 (CVE-2017-16239), we call to the scheduler when doing a rebuild with a new image. If the scheduler refuses a rebuild because a filter forbids the new image on the instance's host (for example, IsolatedHostsFilter), at first there was no indication of this in the API (bz 1536150). Currently, with the fix for bz 1536150 merged, the instance goes to ERROR to indicate the refused rebuild. However, by rebuilding again with the same "forbidden" image it is possible to circumvent scheduler restrictions. Version-Release number of selected component (if applicable): OSP 10 through 12 How reproducible ================ Always. Steps to Reproduce ================== 1. Configure IsolatedHostsFilter: [filter_scheduler] enabled_filters = [...],IsolatedHostsFilter isolated_images = 41d3e5ca-14cf-436c-9413-4826b5c8bdb1 isolated_hosts = ubuntu restrict_isolated_hosts_to_isolated_images = true 2. Have two images, one isolated and one not: $ openstack image list 8d0581a5-ed9d-4b98-a766-a41efbc99929 | centos | active 41d3e5ca-14cf-436c-9413-4826b5c8bdb1 | cirros-0.3.5-x86_64-disk | active cirros is the isolated one 3. Have only one hypervisor (the isolated one): $ openstack hypervisor list ubuntu | QEMU | 192.168.100.194 | up 4. To confirm, boot a centos (non-isolated) image, expecting it to be refused by the scheduler: $ openstack server create \ --image 8d0581a5-ed9d-4b98-a766-a41efbc99929 \ --flavor \ m1.nano centos-test-expect-fail $ openstack server list centos-test-expect-fail | ERROR | | centos | m1.nano 5. Boot a cirros (isolated) image: $ openstack server create \ --image 41d3e5ca-14cf-436c-9413-4826b5c8bdb1 \ --flavor m1.nano \ cirros-test-expect-success $ openstack server list cirros-test-expect-success | ACTIVE | [...] | cirros-0.3.5-x86_64-disk | m1.nano 6. Rebuild the cirros instance with centos: $ nova --debug rebuild cirros-test-expect-success centos DEBUG (session:722) POST call to compute for http://192.168.100.194/compute/v2.1/servers/d9d98bf7-623e-4587-b82c-06f36abf59cb/action used request id req-c234346a-6e05-47cf-a0cd-45f89d11e15d 7. Observe the rebuild being refused in the conductor: WARNING nova.conductor.manager [None req-c234346a-6e05-47cf-a0cd-45f89d11e15d demo admin] [instance: d9d98bf7-623e-4587-b82c-06f36abf59cb] No valid host found for rebuild: NoValidHost_Remote: No valid host was found. There are not enough hosts available. 8. Observe the instance going to ERROR, but still showing the new centos image : $ nova show cirros-test-expect-success [...] status | ERROR image | centos (8d0581a5-ed9d-4b98-a766-a41efbc99929) [...] 9. Rebuild again with the same centos image: $ nova rebuild cirros-test-expect-success centos 10. The rebuild goes through. Expected results ================ The rebuild keeps being refused. Additional info =============== We only set the scheduler hint if the new imageref is different from the old imageref [1], this is a legit bug. [1] https://github.com/openstack/nova/blob/master/nova/compute/api.py#L3055-L3073
Here is the fix: https://review.openstack.org/#/c/538961/ There is a patch with a recreate functional test underneath that in the series.
Looks like the stable backport merged.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2018:2332