Bug 1487156

Summary: Creating a server fails with "No valid host was found"
Product: Red Hat OpenStack Reporter: Tomas Jamrisko <tjamrisk>
Component: openstack-novaAssignee: Eoghan Glynn <eglynn>
Status: CLOSED WORKSFORME QA Contact: Joe H. Rahme <jhakimra>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 12.0 (Pike)CC: berrange, dasmith, eglynn, kchamart, oblaut, sbauza, sferdjao, sgordon, srevivo, vromanso
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
N/A
Last Closed: 2017-09-04 06:43:37 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Tomas Jamrisko 2017-08-31 10:47:16 UTC
Description of problem:
Attempting to create a server in a deployment with 3 controllers using ODL as networking backed fails with the mentioned 

Version-Release number of selected component (if applicable):
16.0.0-0.20170818142923.716a4c8.el7ost.noarch

How reproducible:
100 %

Steps to Reproduce:
1. Get a virtual deployment of OSP12 with ODL and 3 controllers
2. Attempt to start a server

Actual results:
The newly created server ends in "ERROR" status
openstack server shows: 
u'message': u'No valid host was found. There are not enough hosts available.', u'code': 500, u'details': u'  File "/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 937, in schedule_and_build_instances
    instance_uuids)
  File "/usr/lib/python2.7/site-packages/nova/conductor/manager.py", line 625, in _schedule_instances
    request_spec, instance_uuids)
  File "/usr/lib/python2.7/site-packages/nova/scheduler/utils.py", line 505, in wrapped
    return func(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/nova/scheduler/client/__init__.py", line 52, in select_destinations
    instance_uuids)
  File "/usr/lib/python2.7/site-packages/nova/scheduler/client/__init__.py", line 37, in __run_method
    return getattr(self.instance, __name)(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/nova/scheduler/client/query.py", line 33, in select_destinations
    instance_uuids)
  File "/usr/lib/python2.7/site-packages/nova/scheduler/rpcapi.py", line 137, in select_destinations
    return cctxt.call(ctxt, \'select_destinations\', **msg_args)
  File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/client.py", line 169, in call
    retry=self.retry)
  File "/usr/lib/python2.7/site-packages/oslo_messaging/transport.py", line 123, in _send
    timeout=timeout, retry=retry)
  File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 578, in send
    retry=retry)
  File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 569, in _send
    raise result
', u'created': u'2017-08-31T10:30:34Z'} 

Expected results:
server starts 

Additional info:

The hypervisor on which it is executed has 128 GB RAM and 32 cores, computes are VMs, with 16 GB RAM and 2 cpus.

Comment 1 Tomas Jamrisko 2017-08-31 13:31:05 UTC
Just a quick note:

This is a weird one, it's possible it was caused by tempest changing something in the environment/it might have appeared in a cluster after a while that it was running/some packages got update as I am no longer able to reproduce this on a clean environment