Description of problem:


SOS report on: http://ikook.tlv.redhat.com/uploads/sosreport-RonnieRasouli-20180215031515.tar.xz

By running tempest automation tests, there are repeating errors of "Message: No valid host was found. , Code: 500"'

Looking at the openstack hypervisor list show the resources are empty.

+----------------------+-------+
| Field                | Value |
+----------------------+-------+
| count                | 0     |
| current_workload     | 0     |
| disk_available_least | 0     |
| free_disk_gb         | 0     |
| free_ram_mb          | 0     |
| local_gb             | 0     |
| local_gb_used        | 0     |
| memory_mb            | 0     |
| memory_mb_used       | 0     |
| running_vms          | 0     |
| vcpus                | 0     |
| vcpus_used           | 0     |
+----------------------+-------+

This test for example:
http://staging-jenkins2-qe-playground.usersys.redhat.com/view/DFG/view/cloud_apps/view/heat/job/DFG-cloud_apps-heat-13_director-rhel-virthost-3cont_1comp_3ceph-ipv4-vxlan-poc/1/testReport/heat_tempest_plugin.tests.functional.test_translation/TestTranslation/test_value_from_nested_stack/

Version-Release number of selected component (if applicable):

RHOS13 core_puddle: 2018-02-07.4

openstack-nova-api-17.0.0-0.20180123163703.27eadbc.el7ost.noarch
openstack-nova-conductor-17.0.0-0.20180123163703.27eadbc.el7ost.noarch
openstack-nova-compute-17.0.0-0.20180123163703.27eadbc.el7ost.noarch

How reproducible:
100%

Steps to Reproduce:
1. Deploy RHOS13 with 3 controllers, 3 CEPH, 1 Compute
2. run tempest Jenkins job for heat


Actual results:

12 failures related to nova valid host found 


Expected results:

provision VM without those errors 

Additional info:

 fault                               | {u'message': u'No valid host was found. There are not enough hosts available.', u'code': 500, u'details': u'  File "/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 1108, in schedule_and_build_instances\n    instance_uuids, return_alternates=True)\n  File "/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 714, in _schedule_instances\n    return_alternates=return_alternates)\n  File "/usr/lib/python2.7/site-packages/nova/scheduler/utils.py", line 729, in wrapped\n    return func(*args, **kwargs)\n  File "/usr/lib/python2.7/site-packages/nova/scheduler/client/__init__.py", line 53, in select_destinations\n    instance_uuids, return_objects, return_alternates)\n  File "/usr/lib/python2.7/site-packages/nova/scheduler/client/__init__.py", line 37, in __run_method\n    return getattr(self.instance, __name)(*args, **kwargs)\n  File "/usr/lib/python2.7/site-packages/nova/scheduler/client/query.py", line 42, in select_destinations\n    instance_uuids, return_objects, return_alternates)\n  File "/usr/lib/python2.7/site-packages/nova/scheduler/rpcapi.py", line 158, in select_destinations\n    return cctxt.call(ctxt, \'select_destinations\', **msg_args)\n  File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 174, in call\n    retry=self.retry)\n  File "/usr/lib/python2.7/site-packages/oslo_messaging/transport.py", line 131, in _send\n    timeout=timeout, retry=retry)\n  File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 559, in send\n    retry=retry)\n  File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 550, in _send\n    raise result\n', u'created': u'2018-02-15T08:33:42Z'} 

|