Description of problem: FFU: nova api is unreacheable after running deploy_steps_playbook.yaml (overcloud) [stack@undercloud-0 ~]$ nova list ERROR (ClientException): Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <type 'exceptions.TypeError'> (HTTP 500) (Request-ID: req-180d6851-b14d-4499-bd44-5f17a97b7dc9) it looks like both the cell0 and placement databases are empty: [root@controller-0 heat-admin]# mysql -e 'show tables' nova_placement [root@controller-0 heat-admin]# mysql -e 'show tables' nova_cell0 [root@controller-0 heat-admin]# nova-api log: 2018-02-21 02:12:29.960 21 ERROR nova.context [req-b81326d7-06fc-4823-aed3-58041b867e7e eef382c2c4534957bcad96c64e83929a beb53df11b78484fa74a85739a13a57e - default default] Error gathering result from cell 00000000-0000-0000-0000-000000000000: ArgumentError: Could not parse rfc1738 URL from string 'nil' 2018-02-21 02:12:29.960 21 ERROR nova.context Traceback (most recent call last): 2018-02-21 02:12:29.960 21 ERROR nova.context File "/usr/lib/python2.7/site-packages/nova/context.py", line 433, in gather_result 2018-02-21 02:12:29.960 21 ERROR nova.context result = fn(cctxt, *args, **kwargs) 2018-02-21 02:12:29.960 21 ERROR nova.context File "/usr/lib/python2.7/site-packages/nova/compute/multi_cell_list.py", line 239, in do_query 2018-02-21 02:12:29.960 21 ERROR nova.context **kwargs) 2018-02-21 02:12:29.960 21 ERROR nova.context File "/usr/lib/python2.7/site-packages/nova/compute/instance_list.py", line 82, in get_by_filters 2018-02-21 02:12:29.960 21 ERROR nova.context **kwargs) 2018-02-21 02:12:29.960 21 ERROR nova.context File "/usr/lib/python2.7/site-packages/nova/db/api.py", line 802, in instance_get_all_by_filters_sort 2018-02-21 02:12:29.960 21 ERROR nova.context sort_dirs=sort_dirs) 2018-02-21 02:12:29.960 21 ERROR nova.context File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 168, in wrapper 2018-02-21 02:12:29.960 21 ERROR nova.context return f(*args, **kwargs) 2018-02-21 02:12:29.960 21 ERROR nova.context File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api.py", line 254, in wrapped 2018-02-21 02:12:29.960 21 ERROR nova.context with ctxt_mgr.reader.allow_async.using(context): 2018-02-21 02:12:29.960 21 ERROR nova.context File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ 2018-02-21 02:12:29.960 21 ERROR nova.context return self.gen.next() 2018-02-21 02:12:29.960 21 ERROR nova.context File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 1035, in _transaction_scope 2018-02-21 02:12:29.960 21 ERROR nova.context context=context) as resource: 2018-02-21 02:12:29.960 21 ERROR nova.context File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ 2018-02-21 02:12:29.960 21 ERROR nova.context return self.gen.next() 2018-02-21 02:12:29.960 21 ERROR nova.context File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 638, in _session 2018-02-21 02:12:29.960 21 ERROR nova.context bind=self.connection, mode=self.mode) 2018-02-21 02:12:29.960 21 ERROR nova.context File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 403, in _create_session 2018-02-21 02:12:29.960 21 ERROR nova.context self._start() 2018-02-21 02:12:29.960 21 ERROR nova.context File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 489, in _start 2018-02-21 02:12:29.960 21 ERROR nova.context engine_args, maker_args) 2018-02-21 02:12:29.960 21 ERROR nova.context File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 513, in _setup_for_connection 2018-02-21 02:12:29.960 21 ERROR nova.context sql_connection=sql_connection, **engine_kwargs) 2018-02-21 02:12:29.960 21 ERROR nova.context File "/usr/lib/python2.7/site-packages/debtcollector/renames.py", line 43, in decorator 2018-02-21 02:12:29.960 21 ERROR nova.context return wrapped(*args, **kwargs) 2018-02-21 02:12:29.960 21 ERROR nova.context File "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py", line 138, in create_engine 2018-02-21 02:12:29.960 21 ERROR nova.context url = sqlalchemy.engine.url.make_url(sql_connection) 2018-02-21 02:12:29.960 21 ERROR nova.context File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/url.py", line 205, in make_url 2018-02-21 02:12:29.960 21 ERROR nova.context return _parse_rfc1738_args(name_or_url) 2018-02-21 02:12:29.960 21 ERROR nova.context File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/url.py", line 254, in _parse_rfc1738_args 2018-02-21 02:12:29.960 21 ERROR nova.context "Could not parse rfc1738 URL from string '%s'" % name) 2018-02-21 02:12:29.960 21 ERROR nova.context ArgumentError: Could not parse rfc1738 URL from string 'nil' 2018-02-21 02:12:29.960 21 ERROR nova.context 2018-02-21 02:12:29.961 21 DEBUG oslo_concurrency.lockutils [req-b81326d7-06fc-4823-aed3-58041b867e7e eef382c2c4534957bcad96c64e83929a beb53df11b78484fa74a85739a13a57e - default default] Lock "09c02079-f255-49aa-9cfc-e996e912fd27" acquired by "nova.context.get_or_set_cached_cell_and_set_connections" :: waited 0.000s inner /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:273 2018-02-21 02:12:29.962 21 DEBUG oslo_concurrency.lockutils [req-b81326d7-06fc-4823-aed3-58041b867e7e eef382c2c4534957bcad96c64e83929a beb53df11b78484fa74a85739a13a57e - default default] Lock "09c02079-f255-49aa-9cfc-e996e912fd27" released by "nova.context.get_or_set_cached_cell_and_set_connections" :: held 0.000s inner /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:285 2018-02-21 02:12:29.995 21 ERROR nova.api.openstack.wsgi [req-b81326d7-06fc-4823-aed3-58041b867e7e eef382c2c4534957bcad96c64e83929a beb53df11b78484fa74a85739a13a57e - default default] Unexpected exception in API method: TypeError: 'object' object is not iterable 2018-02-21 02:12:29.995 21 ERROR nova.api.openstack.wsgi Traceback (most recent call last): 2018-02-21 02:12:29.995 21 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py", line 788, in wrapped 2018-02-21 02:12:29.995 21 ERROR nova.api.openstack.wsgi return f(*args, **kwargs) 2018-02-21 02:12:29.995 21 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/api/validation/__init__.py", line 192, in wrapper 2018-02-21 02:12:29.995 21 ERROR nova.api.openstack.wsgi return func(*args, **kwargs) 2018-02-21 02:12:29.995 21 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/api/validation/__init__.py", line 192, in wrapper 2018-02-21 02:12:29.995 21 ERROR nova.api.openstack.wsgi return func(*args, **kwargs) 2018-02-21 02:12:29.995 21 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/api/openstack/compute/servers.py", line 165, in detail 2018-02-21 02:12:29.995 21 ERROR nova.api.openstack.wsgi servers = self._get_servers(req, is_detail=True) 2018-02-21 02:12:29.995 21 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/api/openstack/compute/servers.py", line 308, in _get_servers 2018-02-21 02:12:29.995 21 ERROR nova.api.openstack.wsgi sort_keys=sort_keys, sort_dirs=sort_dirs) 2018-02-21 02:12:29.995 21 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 2419, in get_all 2018-02-21 02:12:29.995 21 ERROR nova.api.openstack.wsgi context, filters, limit, marker, fields, sort_keys, sort_dirs) 2018-02-21 02:12:29.995 21 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/compute/instance_list.py", line 107, in get_instance_objects_sorted 2018-02-21 02:12:29.995 21 ERROR nova.api.openstack.wsgi expected_attrs) 2018-02-21 02:12:29.995 21 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/objects/instance.py", line 1199, in _make_instance_list 2018-02-21 02:12:29.995 21 ERROR nova.api.openstack.wsgi for db_inst in db_inst_list: 2018-02-21 02:12:29.995 21 ERROR nova.api.openstack.wsgi File "/usr/lib/python2.7/site-packages/nova/compute/multi_cell_list.py", line 259, in get_records_sorted 2018-02-21 02:12:29.995 21 ERROR nova.api.openstack.wsgi for i in heapq.merge(*results.values()): 2018-02-21 02:12:29.995 21 ERROR nova.api.openstack.wsgi File "/usr/lib64/python2.7/heapq.py", line 372, in merge 2018-02-21 02:12:29.995 21 ERROR nova.api.openstack.wsgi for itnum, it in enumerate(map(iter, iterables)): 2018-02-21 02:12:29.995 21 ERROR nova.api.openstack.wsgi TypeError: 'object' object is not iterable 2018-02-21 02:12:29.995 21 ERROR nova.api.openstack.wsgi 2018-02-21 02:12:29.996 21 INFO nova.api.openstack.wsgi [req-b81326d7-06fc-4823-aed3-58041b867e7e eef382c2c4534957bcad96c64e83929a beb53df11b78484fa74a85739a13a57e - default default] HTTP exception thrown: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <type 'exceptions.TypeError'>
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2018:2086