Description of problem: Regression of BZ #1533161, Setting disk_allocation_ratio=3.0, live migration fails to initiate and give a No valid host was found error Topology: 1 controller, 2 compute How reproducible: always Steps to Reproduce: 1. Deploy RHOS 15 with mentioned topology and disk_allocation_ratio=3.0 set in nova.conf 2. qemu-img resize cirros-0.3.5-x86_64-disk.img +10G 3. glance image-create --name CirrOS --disk-format qcow2 --container-format bare --file cirros-0.3.5-x86_64-disk.img 4. Spin up instances 5. Live block migrate one VM to another compute host such that its disk_available_least becomes negative Actual results: [root@controller-0 nova]# grep disk_allocation_ratio /var/lib/config-data/puppet-generated/nova/etc/nova/nova.conf disk_allocation_ratio=3.0 (overcloud) [stack@undercloud-0 ~]$ qemu-img resize cirros-0.3.5-x86_64-disk.img +10G Image resized. (overcloud) [stack@undercloud-0 ~]$ (overcloud) [stack@undercloud-0 ~]$ glance image-create --name CirrOS --disk-format qcow2 --container-format bare --file cirros-0.3.5-x86_64-disk.img +------------------+----------------------------------------------------------------------------------+ | Property | Value | +------------------+----------------------------------------------------------------------------------+ | checksum | f5f3fc17345acea89c0b68db0e29d888 | | container_format | bare | | created_at | 2019-06-03T21:30:43Z | | direct_url | swift+config://ref1/glance/6bf8b4be-a712-45b9-919e-324a9ea3c7c2 | | disk_format | qcow2 | | id | 6bf8b4be-a712-45b9-919e-324a9ea3c7c2 | | min_disk | 0 | | min_ram | 0 | | name | CirrOS | | os_hash_algo | sha512 | | os_hash_value | c71dc7c9c4de43f73c677fcacc898a7a65336a10a6c512121c6d2d80f3b8054038e484062f3fd8fa | | | 9567a3004844e274633d5b91478cc807cd54f886e3e017ef | | os_hidden | False | | owner | 1a933d33afb3490698135a8f45aee6cd | | protected | False | | size | 13303976 | | status | active | | tags | [] | | updated_at | 2019-06-03T21:30:45Z | | virtual_size | Not available | | visibility | shared | +------------------+----------------------------------------------------------------------------------+ (overcloud) [stack@undercloud-0 ~]$ openstack server list --long +--------------------------------------+------------+--------+------------+-------------+-------------------------+------------+--------------------------------------+-------------+--------------------------------------+-------------------+-----------------------+------------+ | ID | Name | Status | Task State | Power State | Networks | Image Name | Image ID | Flavor Name | Flavor ID | Availability Zone | Host | Properties | +--------------------------------------+------------+--------+------------+-------------+-------------------------+------------+--------------------------------------+-------------+--------------------------------------+-------------------+-----------------------+------------+ | 23cab51d-08d6-4dad-b735-dd36a3ea58d2 | test-28565 | ACTIVE | None | Running | private=192.168.100.213 | CirrOS | 6bf8b4be-a712-45b9-919e-324a9ea3c7c2 | m1.custom | 0fc173b4-9ba1-4251-bcf2-659123fe6ad8 | nova | compute-1.localdomain | | | 68c794e6-0fec-4f04-a0e3-1e11ab36f6db | test-21988 | ACTIVE | None | Running | private=192.168.100.24 | CirrOS | 6bf8b4be-a712-45b9-919e-324a9ea3c7c2 | m1.tiny | d3922670-e1fc-4a77-9d20-7dc73e35dc42 | nova | compute-0.localdomain | | +--------------------------------------+------------+--------+------------+-------------+-------------------------+------------+--------------------------------------+-------------+--------------------------------------+-------------------+-----------------------+------------+ (overcloud) [stack@undercloud-0 ~]$ nova hypervisor-show 1a521cfe-63aa-471a-ab06-070c181a6296 +---------------------------+------------------------------------------+ | Property | Value | +---------------------------+------------------------------------------+ | cpu_info_arch | x86_64 | | cpu_info_features | ["mce", "hle", "pse36", "ss", "avx", | | | "rdrand", "lm", "ssbd", "3dnowprefetch", | | | "erms", "xsave", "hypervisor", | | | "lahf_lm", "rdtscp", "apic", "de", | | | "pae", "abm", "tsc_adjust", "xsaveopt", | | | "fma", "vme", "pcid", "adx", "msr", | | | "fxsr", "mmx", "pge", "cx16", "tsc- | | | deadline", "vmx", "mca", "popcnt", | | | "clflush", "rtm", "sse2", "syscall", | | | "sep", "cx8", "ssse3", "pse", "tsc", | | | "pat", "sse4.2", "rdseed", "movbe", | | | "sse4.1", "f16c", "bmi2", "pni", "cmov", | | | "nx", "spec-ctrl", "pclmuldq", "bmi1", | | | "avx2", "sse", "pdpe1gb", "arat", | | | "smap", "aes", "mtrr", "fsgsbase", | | | "x2apic", "fpu", "osxsave", "stibp", | | | "smep", "invpcid"] | | cpu_info_model | Broadwell-IBRS | | cpu_info_topology_cells | 1 | | cpu_info_topology_cores | 1 | | cpu_info_topology_sockets | 3 | | cpu_info_topology_threads | 1 | | cpu_info_vendor | Intel | | current_workload | 0 | | disk_available_least | 1 | | free_disk_gb | 7 | | free_ram_mb | 14810 | | host_ip | 172.17.1.140 | | hypervisor_hostname | compute-1.localdomain | | hypervisor_type | QEMU | | hypervisor_version | 3001000 | | id | 1a521cfe-63aa-471a-ab06-070c181a6296 | | local_gb | 19 | | local_gb_used | 12 | | memory_mb | 19418 | | memory_mb_used | 4608 | | running_vms | 1 | | service_disabled_reason | None | | service_host | compute-1.localdomain | | service_id | 25e1eccc-de9d-494f-b822-e35e98b66fe1 | | state | up | | status | enabled | | vcpus | 3 | | vcpus_used | 1 | +---------------------------+------------------------------------------+ (overcloud) [stack@undercloud-0 ~]$ nova --debug live-migration --block-migrate test-21988 compute-1.localdomain DEBUG (extension:189) found extension EntryPoint.parse('v1password = swiftclient.authv1:PasswordLoader') DEBUG (extension:189) found extension EntryPoint.parse('token_endpoint = openstackclient.api.auth_plugin:TokenEndpoint') DEBUG (extension:189) found extension EntryPoint.parse('noauth = cinderclient.contrib.noauth:CinderNoAuthLoader') DEBUG (extension:189) found extension EntryPoint.parse('admin_token = keystoneauth1.loading._plugins.admin_token:AdminToken') DEBUG (extension:189) found extension EntryPoint.parse('none = keystoneauth1.loading._plugins.noauth:NoAuth') DEBUG (extension:189) found extension EntryPoint.parse('password = keystoneauth1.loading._plugins.identity.generic:Password') DEBUG (extension:189) found extension EntryPoint.parse('token = keystoneauth1.loading._plugins.identity.generic:Token') DEBUG (extension:189) found extension EntryPoint.parse('v2password = keystoneauth1.loading._plugins.identity.v2:Password') DEBUG (extension:189) found extension EntryPoint.parse('v2token = keystoneauth1.loading._plugins.identity.v2:Token') DEBUG (extension:189) found extension EntryPoint.parse('v3adfspassword = keystoneauth1.extras._saml2._loading:ADFSPassword') DEBUG (extension:189) found extension EntryPoint.parse('v3applicationcredential = keystoneauth1.loading._plugins.identity.v3:ApplicationCredential') DEBUG (extension:189) found extension EntryPoint.parse('v3fedkerb = keystoneauth1.extras.kerberos._loading:MappedKerberos') DEBUG (extension:189) found extension EntryPoint.parse('v3kerberos = keystoneauth1.extras.kerberos._loading:Kerberos') DEBUG (extension:189) found extension EntryPoint.parse('v3oauth1 = keystoneauth1.extras.oauth1._loading:V3OAuth1') DEBUG (extension:189) found extension EntryPoint.parse('v3oidcaccesstoken = keystoneauth1.loading._plugins.identity.v3:OpenIDConnectAccessToken') DEBUG (extension:189) found extension EntryPoint.parse('v3oidcauthcode = keystoneauth1.loading._plugins.identity.v3:OpenIDConnectAuthorizationCode') DEBUG (extension:189) found extension EntryPoint.parse('v3oidcclientcredentials = keystoneauth1.loading._plugins.identity.v3:OpenIDConnectClientCredentials') DEBUG (extension:189) found extension EntryPoint.parse('v3oidcpassword = keystoneauth1.loading._plugins.identity.v3:OpenIDConnectPassword') DEBUG (extension:189) found extension EntryPoint.parse('v3password = keystoneauth1.loading._plugins.identity.v3:Password') DEBUG (extension:189) found extension EntryPoint.parse('v3samlpassword = keystoneauth1.extras._saml2._loading:Saml2Password') DEBUG (extension:189) found extension EntryPoint.parse('v3token = keystoneauth1.loading._plugins.identity.v3:Token') DEBUG (extension:189) found extension EntryPoint.parse('v3tokenlessauth = keystoneauth1.loading._plugins.identity.v3:TokenlessAuth') DEBUG (extension:189) found extension EntryPoint.parse('v3totp = keystoneauth1.loading._plugins.identity.v3:TOTP') DEBUG (extension:189) found extension EntryPoint.parse('gnocchi-basic = gnocchiclient.auth:GnocchiBasicLoader') DEBUG (extension:189) found extension EntryPoint.parse('gnocchi-noauth = gnocchiclient.auth:GnocchiNoAuthLoader') DEBUG (extension:189) found extension EntryPoint.parse('aodh-noauth = aodhclient.noauth:AodhNoAuthLoader') DEBUG (session:464) REQ: curl -g -i -X GET http://10.0.0.133:5000 -H "Accept: application/json" -H "User-Agent: nova keystoneauth1/3.13.1 python-requests/2.20.0 CPython/3.6.8" DEBUG (connectionpool:205) Starting new HTTP connection (1): 10.0.0.133:5000 DEBUG (connectionpool:393) http://10.0.0.133:5000 "GET / HTTP/1.1" 300 265 DEBUG (session:495) RESP: [300] Content-Length: 265 Content-Type: application/json Date: Tue, 04 Jun 2019 18:30:46 GMT Location: http://10.0.0.133:5000/v3/ Server: Apache Vary: X-Auth-Token x-openstack-request-id: req-11e54a73-b2d8-435d-87d1-075613aa1b43 DEBUG (session:527) RESP BODY: {"versions": {"values": [{"id": "v3.12", "status": "stable", "updated": "2019-01-22T00:00:00Z", "links": [{"rel": "self", "href": "http://10.0.0.133:5000/v3/"}], "media-types": [{"base": "application/json", "type": "application/vnd.openstack.identity-v3+json"}]}]}} DEBUG (session:874) GET call to http://10.0.0.133:5000/ used request id req-11e54a73-b2d8-435d-87d1-075613aa1b43 DEBUG (base:176) Making authentication request to http://10.0.0.133:5000/v3/auth/tokens DEBUG (connectionpool:393) http://10.0.0.133:5000 "POST /v3/auth/tokens HTTP/1.1" 201 8436 DEBUG (base:181) {"token": {"methods": ["password"], "user": {"domain": {"id": "default", "name": "Default"}, "id": "605f6dd1d0624acc801bc464c911b811", "name": "admin", "password_expires_at": null}, "audit_ids": ["Vuu_UG5WQz2LWz7fJGn7_g"], "expires_at": "2019-06-04T19:30:46.000000Z", "issued_at": "2019-06-04T18:30:46.000000Z", "project": {"domain": {"id": "default", "name": "Default"}, "id": "1a933d33afb3490698135a8f45aee6cd", "name": "admin"}, "is_domain": false, "roles": [{"id": "4463a3aa4f674d54b62746ba56603e46", "name": "admin"}, {"id": "43b9b410c44144c39de02d2afeeaa6e4", "name": "member"}, {"id": "04fc76f4114f4ab98990f1e3bbb93842", "name": "_member_"}, {"id": "8ca1a80e421d439785b6977f14f5089e", "name": "reader"}], "catalog": [{"endpoints": [{"id": "5f9bc45a10784525934735f9fa1edddc", "interface": "admin", "region_id": "regionOne", "url": "http://172.17.1.130:8778/placement", "region": "regionOne"}, {"id": "906abfa950df4af5b2aa1bc9468b7e5b", "interface": "public", "region_id": "regionOne", "url": "http://10.0.0.133:8778/placement", "region": "regionOne"}, {"id": "9d49dec0db3548d397671fa7a8a3026c", "interface": "internal", "region_id": "regionOne", "url": "http://172.17.1.130:8778/placement", "region": "regionOne"}], "id": "026dafa0cded48a6bcf2175dc8746969", "type": "placement", "name": "placement"}, {"endpoints": [{"id": "036ba9b1c0cb4de5b38bde007e0c90c8", "interface": "internal", "region_id": "regionOne", "url": "http://172.17.1.130:8042", "region": "regionOne"}, {"id": "3650464d70c4478984f570bb6b0a8c05", "interface": "public", "region_id": "regionOne", "url": "http://10.0.0.133:8042", "region": "regionOne"}, {"id": "37175bbf244548e5b448511e05df9414", "interface": "admin", "region_id": "regionOne", "url": "http://172.17.1.130:8042", "region": "regionOne"}], "id": "03b5addae41741249a32d745cec76152", "type": "alarming", "name": "aodh"}, {"endpoints": [{"id": "01d3b2d8960c40e095f2e14e096f96c2", "interface": "admin", "region_id": "regionOne", "url": "http://172.17.1.130:8776/v3/1a933d33afb3490698135a8f45aee6cd", "region": "regionOne"}, {"id": "3d8048cb82074226be4defc99a5f69af", "interface": "public", "region_id": "regionOne", "url": "http://10.0.0.133:8776/v3/1a933d33afb3490698135a8f45aee6cd", "region": "regionOne"}, {"id": "b624b39011ba4f4092fb97d73d2f257e", "interface": "internal", "region_id": "regionOne", "url": "http://172.17.1.130:8776/v3/1a933d33afb3490698135a8f45aee6cd", "region": "regionOne"}], "id": "15f9ccac13564b14b94e29905c53f5bc", "type": "volumev3", "name": "cinderv3"}, {"endpoints": [{"id": "27c3ef5076be4408933815340c5fee87", "interface": "admin", "region_id": "regionOne", "url": "http://172.17.3.80:8080", "region": "regionOne"}, {"id": "46eafc1b9326491c9fa95223123a1cc6", "interface": "public", "region_id": "regionOne", "url": "http://10.0.0.133:8080/v1/AUTH_1a933d33afb3490698135a8f45aee6cd", "region": "regionOne"}, {"id": "ec1a92cd25534a6b900e1db75361e8a1", "interface": "internal", "region_id": "regionOne", "url": "http://172.17.3.80:8080/v1/AUTH_1a933d33afb3490698135a8f45aee6cd", "region": "regionOne"}], "id": "1a344285f2ef4954a71ac3a1d95617ff", "type": "object-store", "name": "swift"}, {"endpoints": [{"id": "039fbeed1a1c4d708678f049f4f3db45", "interface": "public", "region_id": "regionOne", "url": "http://10.0.0.133:8004/v1/1a933d33afb3490698135a8f45aee6cd", "region": "regionOne"}, {"id": "31afac7817354f0eaa68412d6f5393d4", "interface": "internal", "region_id": "regionOne", "url": "http://172.17.1.130:8004/v1/1a933d33afb3490698135a8f45aee6cd", "region": "regionOne"}, {"id": "4607848b3c164f87ac1453321219e4ac", "interface": "admin", "region_id": "regionOne", "url": "http://172.17.1.130:8004/v1/1a933d33afb3490698135a8f45aee6cd", "region": "regionOne"}], "id": "2e4a207cf5484fdbb83bc9480b370084", "type": "orchestration", "name": "heat"}, {"endpoints": [{"id": "71fee02838954299aa591ae80738eb00", "interface": "internal", "region_id": "regionOne", "url": "http://172.17.1.130:8977", "region": "regionOne"}, {"id": "82677d91aa304762853949da878d4f91", "interface": "public", "region_id": "regionOne", "url": "http://10.0.0.133:8977", "region": "regionOne"}, {"id": "a8b5adba59f64fbc9d2358130fa8665c", "interface": "admin", "region_id": "regionOne", "url": "http://172.17.1.130:8977", "region": "regionOne"}], "id": "47262b1b02bd46d7afad6564f79851bd", "type": "event", "name": "panko"}, {"endpoints": [{"id": "3d422ee489f74be390b0a9a7a8d08d62", "interface": "admin", "region_id": "regionOne", "url": "http://172.17.1.130:8774/v2.1", "region": "regionOne"}, {"id": "4c61acf88a4d4102bb72491a90a4b70b", "interface": "public", "region_id": "regionOne", "url": "http://10.0.0.133:8774/v2.1", "region": "regionOne"}, {"id": "731eb0982697447b84eabce3473344c5", "interface": "internal", "region_id": "regionOne", "url": "http://172.17.1.130:8774/v2.1", "region": "regionOne"}], "id": "4b24452d48374418a0b0346831d7005d", "type": "compute", "name": "nova"}, {"endpoints": [{"id": "3614fde11c574bedb645f0c6f93f72b3", "interface": "public", "region_id": "regionOne", "url": "http://10.0.0.133:8041", "region": "regionOne"}, {"id": "c21e1797253a4e7583d6a036495fff54", "interface": "admin", "region_id": "regionOne", "url": "http://172.17.1.130:8041", "region": "regionOne"}, {"id": "f47bc6302215431a88e322cf2bfcbb9b", "interface": "internal", "region_id": "regionOne", "url": "http://172.17.1.130:8041", "region": "regionOne"}], "id": "8a036f0e5be549d18c625593e5ff7391", "type": "metric", "name": "gnocchi"}, {"endpoints": [{"id": "1eaca2e7165249f6bd4ad7801a91bcc0", "interface": "public", "region_id": "regionOne", "url": "http://10.0.0.133:9696", "region": "regionOne"}, {"id": "2038742c2bae4a3db0d078d72887a13a", "interface": "admin", "region_id": "regionOne", "url": "http://172.17.1.130:9696", "region": "regionOne"}, {"id": "48506dda87a64287b2c0a41af6b85ac9", "interface": "internal", "region_id": "regionOne", "url": "http://172.17.1.130:9696", "region": "regionOne"}], "id": "92c8e4a6f9774352bb2600d6f30d018c", "type": "network", "name": "neutron"}, {"endpoints": [{"id": "3ed6f1fee6774c0aaca4c78e8a26ca2f", "interface": "admin", "region_id": "regionOne", "url": "http://172.17.1.130:8776/v2/1a933d33afb3490698135a8f45aee6cd", "region": "regionOne"}, {"id": "67ef6f74479d4cd9bbf96bcc4f4b9b1a", "interface": "internal", "region_id": "regionOne", "url": "http://172.17.1.130:8776/v2/1a933d33afb3490698135a8f45aee6cd", "region": "regionOne"}, {"id": "ab61267b8be24a4faa45803a30e8d39a", "interface": "public", "region_id": "regionOne", "url": "http://10.0.0.133:8776/v2/1a933d33afb3490698135a8f45aee6cd", "region": "regionOne"}], "id": "a24307d404fb4265b5cbfe50899aab64", "type": "volumev2", "name": "cinderv2"}, {"endpoints": [], "id": "ae5fb0b127f74e4db9c4fcf1f43f61df", "type": "metering", "name": "ceilometer"}, {"endpoints": [{"id": "25e0be73ee13472eab605b39be8ac8c5", "interface": "admin", "region_id": "regionOne", "url": "http://172.17.1.130:8000/v1", "region": "regionOne"}, {"id": "6f70b99738db468ba7df091b15e3d0c0", "interface": "internal", "region_id": "regionOne", "url": "http://172.17.1.130:8000/v1", "region": "regionOne"}, {"id": "e358b038c82b431aae75b3aad56ada72", "interface": "public", "region_id": "regionOne", "url": "http://10.0.0.133:8000/v1", "region": "regionOne"}], "id": "bcc0336755b84cf09dca6e1a110d844f", "type": "cloudformation", "name": "heat-cfn"}, {"endpoints": [{"id": "158f7c61e1254bc4b04704f01d18bad9", "interface": "admin", "region_id": "regionOne", "url": "http://192.168.24.11:35357", "region": "regionOne"}, {"id": "aea90b0053e34575bef02f3049ddd6ac", "interface": "internal", "region_id": "regionOne", "url": "http://172.17.1.130:5000", "region": "regionOne"}, {"id": "bbf7ee88017e440682ebf59a169cd1c2", "interface": "public", "region_id": "regionOne", "url": "http://10.0.0.133:5000", "region": "regionOne"}], "id": "c45be027d0ee401daa6d55868b4dc582", "type": "identity", "name": "keystone"}, {"endpoints": [{"id": "0e1d60911c8243deb3dd0390eae5fca8", "interface": "internal", "region_id": "regionOne", "url": "http://172.17.1.130:9292", "region": "regionOne"}, {"id": "417ed6d071504d5b8d533fddc9fad759", "interface": "admin", "region_id": "regionOne", "url": "http://172.17.1.130:9292", "region": "regionOne"}, {"id": "77420aaf27ad46ada70be56baeff3451", "interface": "public", "region_id": "regionOne", "url": "http://10.0.0.133:9292", "region": "regionOne"}], "id": "d01a48e044d04209acf55c7a31ccff35", "type": "image", "name": "glance"}]}} REQ: curl -g -i -X GET http://10.0.0.133:8774/v2.1 -H "Accept: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}1c75f94710f38e53c0e34ae639fdc4cff20aa2d7c7192f96378fcd18180234fc" DEBUG (session:464) REQ: curl -g -i -X GET http://10.0.0.133:8774/v2.1 -H "Accept: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}1c75f94710f38e53c0e34ae639fdc4cff20aa2d7c7192f96378fcd18180234fc" DEBUG (connectionpool:205) Starting new HTTP connection (1): 10.0.0.133:8774 DEBUG (connectionpool:393) http://10.0.0.133:8774 "GET /v2.1 HTTP/1.1" 302 None RESP: [302] Content-Type: text/plain; charset=utf8 Date: Tue, 04 Jun 2019 18:30:46 GMT Location: http://10.0.0.133:8774/v2.1/ Server: Apache Transfer-Encoding: chunked x-compute-request-id: req-c9a7b53c-fdf4-4c66-b227-3f2103a8601e x-openstack-request-id: req-c9a7b53c-fdf4-4c66-b227-3f2103a8601e DEBUG (session:495) RESP: [302] Content-Type: text/plain; charset=utf8 Date: Tue, 04 Jun 2019 18:30:46 GMT Location: http://10.0.0.133:8774/v2.1/ Server: Apache Transfer-Encoding: chunked x-compute-request-id: req-c9a7b53c-fdf4-4c66-b227-3f2103a8601e x-openstack-request-id: req-c9a7b53c-fdf4-4c66-b227-3f2103a8601e RESP BODY: Omitted, Content-Type is set to text/plain; charset=utf8. Only application/json responses have their bodies logged. DEBUG (session:527) RESP BODY: Omitted, Content-Type is set to text/plain; charset=utf8. Only application/json responses have their bodies logged. DEBUG (connectionpool:393) http://10.0.0.133:8774 "GET /v2.1/ HTTP/1.1" 200 None RESP: [200] Content-Encoding: gzip Content-Type: application/json Date: Tue, 04 Jun 2019 18:30:47 GMT OpenStack-API-Version: compute 2.1 Server: Apache Transfer-Encoding: chunked Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version,Accept-Encoding X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-18e11988-fb55-4d56-b008-686b3fe63799 x-openstack-request-id: req-18e11988-fb55-4d56-b008-686b3fe63799 DEBUG (session:495) RESP: [200] Content-Encoding: gzip Content-Type: application/json Date: Tue, 04 Jun 2019 18:30:47 GMT OpenStack-API-Version: compute 2.1 Server: Apache Transfer-Encoding: chunked Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version,Accept-Encoding X-OpenStack-Nova-API-Version: 2.1 x-compute-request-id: req-18e11988-fb55-4d56-b008-686b3fe63799 x-openstack-request-id: req-18e11988-fb55-4d56-b008-686b3fe63799 RESP BODY: {"version": {"id": "v2.1", "status": "CURRENT", "version": "2.72", "min_version": "2.1", "updated": "2013-07-23T11:33:21Z", "links": [{"rel": "self", "href": "http://10.0.0.133:8774/v2.1/"}, {"rel": "describedby", "type": "text/html", "href": "http://docs.openstack.org/"}], "media-types": [{"base": "application/json", "type": "application/vnd.openstack.compute+json;version=2.1"}]}} DEBUG (session:527) RESP BODY: {"version": {"id": "v2.1", "status": "CURRENT", "version": "2.72", "min_version": "2.1", "updated": "2013-07-23T11:33:21Z", "links": [{"rel": "self", "href": "http://10.0.0.133:8774/v2.1/"}, {"rel": "describedby", "type": "text/html", "href": "http://docs.openstack.org/"}], "media-types": [{"base": "application/json", "type": "application/vnd.openstack.compute+json;version=2.1"}]}} GET call to compute for http://10.0.0.133:8774/v2.1/ used request id req-18e11988-fb55-4d56-b008-686b3fe63799 DEBUG (session:865) GET call to compute for http://10.0.0.133:8774/v2.1/ used request id req-18e11988-fb55-4d56-b008-686b3fe63799 DEBUG (extension:189) found extension EntryPoint.parse('v1password = swiftclient.authv1:PasswordLoader') DEBUG (extension:189) found extension EntryPoint.parse('token_endpoint = openstackclient.api.auth_plugin:TokenEndpoint') DEBUG (extension:189) found extension EntryPoint.parse('noauth = cinderclient.contrib.noauth:CinderNoAuthLoader') DEBUG (extension:189) found extension EntryPoint.parse('admin_token = keystoneauth1.loading._plugins.admin_token:AdminToken') DEBUG (extension:189) found extension EntryPoint.parse('none = keystoneauth1.loading._plugins.noauth:NoAuth') DEBUG (extension:189) found extension EntryPoint.parse('password = keystoneauth1.loading._plugins.identity.generic:Password') DEBUG (extension:189) found extension EntryPoint.parse('token = keystoneauth1.loading._plugins.identity.generic:Token') DEBUG (extension:189) found extension EntryPoint.parse('v2password = keystoneauth1.loading._plugins.identity.v2:Password') DEBUG (extension:189) found extension EntryPoint.parse('v2token = keystoneauth1.loading._plugins.identity.v2:Token') DEBUG (extension:189) found extension EntryPoint.parse('v3adfspassword = keystoneauth1.extras._saml2._loading:ADFSPassword') DEBUG (extension:189) found extension EntryPoint.parse('v3applicationcredential = keystoneauth1.loading._plugins.identity.v3:ApplicationCredential') DEBUG (extension:189) found extension EntryPoint.parse('v3fedkerb = keystoneauth1.extras.kerberos._loading:MappedKerberos') DEBUG (extension:189) found extension EntryPoint.parse('v3kerberos = keystoneauth1.extras.kerberos._loading:Kerberos') DEBUG (extension:189) found extension EntryPoint.parse('v3oauth1 = keystoneauth1.extras.oauth1._loading:V3OAuth1') DEBUG (extension:189) found extension EntryPoint.parse('v3oidcaccesstoken = keystoneauth1.loading._plugins.identity.v3:OpenIDConnectAccessToken') DEBUG (extension:189) found extension EntryPoint.parse('v3oidcauthcode = keystoneauth1.loading._plugins.identity.v3:OpenIDConnectAuthorizationCode') DEBUG (extension:189) found extension EntryPoint.parse('v3oidcclientcredentials = keystoneauth1.loading._plugins.identity.v3:OpenIDConnectClientCredentials') DEBUG (extension:189) found extension EntryPoint.parse('v3oidcpassword = keystoneauth1.loading._plugins.identity.v3:OpenIDConnectPassword') DEBUG (extension:189) found extension EntryPoint.parse('v3password = keystoneauth1.loading._plugins.identity.v3:Password') DEBUG (extension:189) found extension EntryPoint.parse('v3samlpassword = keystoneauth1.extras._saml2._loading:Saml2Password') DEBUG (extension:189) found extension EntryPoint.parse('v3token = keystoneauth1.loading._plugins.identity.v3:Token') DEBUG (extension:189) found extension EntryPoint.parse('v3tokenlessauth = keystoneauth1.loading._plugins.identity.v3:TokenlessAuth') DEBUG (extension:189) found extension EntryPoint.parse('v3totp = keystoneauth1.loading._plugins.identity.v3:TOTP') DEBUG (extension:189) found extension EntryPoint.parse('gnocchi-basic = gnocchiclient.auth:GnocchiBasicLoader') DEBUG (extension:189) found extension EntryPoint.parse('gnocchi-noauth = gnocchiclient.auth:GnocchiNoAuthLoader') DEBUG (extension:189) found extension EntryPoint.parse('aodh-noauth = aodhclient.noauth:AodhNoAuthLoader') DEBUG (session:464) REQ: curl -g -i -X GET http://10.0.0.133:8774/v2.1/servers?name=test-21988 -H "Accept: application/json" -H "OpenStack-API-Version: compute 2.72" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}1c75f94710f38e53c0e34ae639fdc4cff20aa2d7c7192f96378fcd18180234fc" -H "X-OpenStack-Nova-API-Version: 2.72" DEBUG (connectionpool:393) http://10.0.0.133:8774 "GET /v2.1/servers?name=test-21988 HTTP/1.1" 200 None DEBUG (session:495) RESP: [200] Content-Encoding: gzip Content-Type: application/json Date: Tue, 04 Jun 2019 18:30:47 GMT OpenStack-API-Version: compute 2.72 Server: Apache Transfer-Encoding: chunked Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version,Accept-Encoding X-OpenStack-Nova-API-Version: 2.72 x-compute-request-id: req-b94b2f69-0ae4-44fa-b4f7-04f2b496fb00 x-openstack-request-id: req-b94b2f69-0ae4-44fa-b4f7-04f2b496fb00 DEBUG (session:527) RESP BODY: {"servers": [{"id": "68c794e6-0fec-4f04-a0e3-1e11ab36f6db", "name": "test-21988", "links": [{"rel": "self", "href": "http://10.0.0.133:8774/v2.1/servers/68c794e6-0fec-4f04-a0e3-1e11ab36f6db"}, {"rel": "bookmark", "href": "http://10.0.0.133:8774/servers/68c794e6-0fec-4f04-a0e3-1e11ab36f6db"}]}]} DEBUG (session:865) GET call to compute for http://10.0.0.133:8774/v2.1/servers?name=test-21988 used request id req-b94b2f69-0ae4-44fa-b4f7-04f2b496fb00 DEBUG (session:464) REQ: curl -g -i -X GET http://10.0.0.133:8774/v2.1/servers/68c794e6-0fec-4f04-a0e3-1e11ab36f6db -H "Accept: application/json" -H "OpenStack-API-Version: compute 2.72" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}1c75f94710f38e53c0e34ae639fdc4cff20aa2d7c7192f96378fcd18180234fc" -H "X-OpenStack-Nova-API-Version: 2.72" DEBUG (connectionpool:393) http://10.0.0.133:8774 "GET /v2.1/servers/68c794e6-0fec-4f04-a0e3-1e11ab36f6db HTTP/1.1" 200 None DEBUG (session:495) RESP: [200] Content-Encoding: gzip Content-Type: application/json Date: Tue, 04 Jun 2019 18:30:47 GMT OpenStack-API-Version: compute 2.72 Server: Apache Transfer-Encoding: chunked Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version,Accept-Encoding X-OpenStack-Nova-API-Version: 2.72 x-compute-request-id: req-2fda8c35-f4ea-43bd-99b0-8acb4ad711c8 x-openstack-request-id: req-2fda8c35-f4ea-43bd-99b0-8acb4ad711c8 DEBUG (session:527) RESP BODY: {"server": {"id": "68c794e6-0fec-4f04-a0e3-1e11ab36f6db", "name": "test-21988", "status": "ACTIVE", "tenant_id": "1a933d33afb3490698135a8f45aee6cd", "user_id": "605f6dd1d0624acc801bc464c911b811", "metadata": {}, "hostId": "c96174fffd319833ffaf86f1b8866ab2a3c527d0b904a561a99dcc4c", "image": {"id": "6bf8b4be-a712-45b9-919e-324a9ea3c7c2", "links": [{"rel": "bookmark", "href": "http://10.0.0.133:8774/images/6bf8b4be-a712-45b9-919e-324a9ea3c7c2"}]}, "flavor": {"vcpus": 1, "ram": 512, "disk": 15, "ephemeral": 0, "swap": 0, "original_name": "m1.tiny", "extra_specs": {}}, "created": "2019-06-03T21:34:19Z", "updated": "2019-06-04T15:04:18Z", "addresses": {"private": [{"version": 4, "addr": "192.168.100.24", "OS-EXT-IPS:type": "fixed", "OS-EXT-IPS-MAC:mac_addr": "fa:16:3e:32:ef:09"}]}, "accessIPv4": "", "accessIPv6": "", "links": [{"rel": "self", "href": "http://10.0.0.133:8774/v2.1/servers/68c794e6-0fec-4f04-a0e3-1e11ab36f6db"}, {"rel": "bookmark", "href": "http://10.0.0.133:8774/servers/68c794e6-0fec-4f04-a0e3-1e11ab36f6db"}], "OS-DCF:diskConfig": "MANUAL", "progress": 0, "OS-EXT-AZ:availability_zone": "nova", "config_drive": "", "key_name": null, "OS-SRV-USG:launched_at": "2019-06-03T21:34:32.000000", "OS-SRV-USG:terminated_at": null, "security_groups": [{"name": "default"}], "OS-EXT-SRV-ATTR:host": "compute-0.localdomain", "OS-EXT-SRV-ATTR:instance_name": "instance-00000001", "OS-EXT-SRV-ATTR:hypervisor_hostname": "compute-0.localdomain", "OS-EXT-SRV-ATTR:reservation_id": "r-tor15uhj", "OS-EXT-SRV-ATTR:launch_index": 0, "OS-EXT-SRV-ATTR:hostname": "test-21988", "OS-EXT-SRV-ATTR:kernel_id": "", "OS-EXT-SRV-ATTR:ramdisk_id": "", "OS-EXT-SRV-ATTR:root_device_name": "/dev/vda", "OS-EXT-SRV-ATTR:user_data": null, "OS-EXT-STS:task_state": null, "OS-EXT-STS:vm_state": "active", "OS-EXT-STS:power_state": 1, "os-extended-volumes:volumes_attached": [], "host_status": "UP", "locked": false, "description": "test-21988", "tags": [], "trusted_image_certificates": null, "server_groups": []}} DEBUG (session:865) GET call to compute for http://10.0.0.133:8774/v2.1/servers/68c794e6-0fec-4f04-a0e3-1e11ab36f6db used request id req-2fda8c35-f4ea-43bd-99b0-8acb4ad711c8 DEBUG (session:464) REQ: curl -g -i -X POST http://10.0.0.133:8774/v2.1/servers/68c794e6-0fec-4f04-a0e3-1e11ab36f6db/action -H "Accept: application/json" -H "Content-Type: application/json" -H "OpenStack-API-Version: compute 2.72" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}1c75f94710f38e53c0e34ae639fdc4cff20aa2d7c7192f96378fcd18180234fc" -H "X-OpenStack-Nova-API-Version: 2.72" -d '{"os-migrateLive": {"host": "compute-1.localdomain", "block_migration": true}}' ^[[ADEBUG (connectionpool:393) http://10.0.0.133:8774 "POST /v2.1/servers/68c794e6-0fec-4f04-a0e3-1e11ab36f6db/action HTTP/1.1" 202 0 DEBUG (session:495) RESP: [202] Content-Length: 0 Content-Type: application/json Date: Tue, 04 Jun 2019 18:30:48 GMT OpenStack-API-Version: compute 2.72 Server: Apache Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version X-OpenStack-Nova-API-Version: 2.72 x-compute-request-id: req-65d94222-1be1-46d6-bbe4-af0d1bbab2aa x-openstack-request-id: req-65d94222-1be1-46d6-bbe4-af0d1bbab2aa DEBUG (session:865) POST call to compute for http://10.0.0.133:8774/v2.1/servers/68c794e6-0fec-4f04-a0e3-1e11ab36f6db/action used request id req-65d94222-1be1-46d6-bbe4-af0d1bbab2aa (overcloud) [stack@undercloud-0 ~]$ openstack server list --long +--------------------------------------+------------+--------+------------+-------------+-------------------------+------------+--------------------------------------+-------------+--------------------------------------+-------------------+-----------------------+------------+ | ID | Name | Status | Task State | Power State | Networks | Image Name | Image ID | Flavor Name | Flavor ID | Availability Zone | Host | Properties | +--------------------------------------+------------+--------+------------+-------------+-------------------------+------------+--------------------------------------+-------------+--------------------------------------+-------------------+-----------------------+------------+ | 23cab51d-08d6-4dad-b735-dd36a3ea58d2 | test-28565 | ACTIVE | None | Running | private=192.168.100.213 | CirrOS | 6bf8b4be-a712-45b9-919e-324a9ea3c7c2 | m1.custom | 0fc173b4-9ba1-4251-bcf2-659123fe6ad8 | nova | compute-1.localdomain | | | 68c794e6-0fec-4f04-a0e3-1e11ab36f6db | test-21988 | ACTIVE | None | Running | private=192.168.100.24 | CirrOS | 6bf8b4be-a712-45b9-919e-324a9ea3c7c2 | m1.tiny | d3922670-e1fc-4a77-9d20-7dc73e35dc42 | nova | compute-0.localdomain | | +--------------------------------------+------------+--------+------------+-------------+-------------------------+------------+--------------------------------------+-------------+--------------------------------------+-------------------+-----------------------+------------+ nova-manage.log: 2019-06-04 18:30:52.295 19 WARNING nova.scheduler.utils [req-65d94222-1be1-46d6-bbe4-af0d1bbab2aa 605f6dd1d0624acc801bc464c911b811 1a933d33afb3490698135a8f45aee6cd - default default] [instance: 68c794e6-0fec-4f04-a0e3-1e11ab36f6db] Setting instance to ACTIVE state.: nova.exception_Remote.NoValidHost_Remote: No valid host was found. 2019-06-04 18:30:52.464 19 ERROR oslo_messaging.rpc.server [req-65d94222-1be1-46d6-bbe4-af0d1bbab2aa 605f6dd1d0624acc801bc464c911b811 1a933d33afb3490698135a8f45aee6cd - default default] Exception during message handling: nova.exception_Remote.NoValidHost_Remote: No valid host was found. Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 229, in inner return func(*args, **kwargs) File "/usr/lib/python3.6/site-packages/nova/scheduler/manager.py", line 153, in select_destinations raise exception.NoValidHost(reason="") nova.exception.NoValidHost: No valid host was found. Expected results: Live migration passes Additional info:
Created attachment 1577247 [details] compure-1sosreport
Created attachment 1577248 [details] compute-0sosreport
Starting Queens the disk_allocation_ratio should not be set on the controllers. Right now we don't have a parameter to set the disk_allocation_ratio. This BZ tracks the implementation of a role parameter to set the disk_allocation_ratio on the computes.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:3148