Description of problem: The upgrade of the OSP10 undercloud to OSP11 fails because of a nova-api dbsync related problem: Version-Release number of selected component (if applicable): OSP11 How reproducible: Steps to Reproduce: 1. setup OSP11 repos (e.g. sudo rhos-release -P 11) 2. stop services 3. openstack undercloud upgrade Actual results: 2017-01-03 07:34:55,735 INFO: Notice: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]/returns: File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/migration.py", line 57, in db_sync 2017-01-03 07:34:55,735 INFO: Notice: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]/returns: repository, version) 2017-01-03 07:34:55,735 INFO: Notice: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]/returns: File "/usr/lib/python2.7/site-packages/migrate/versioning/api.py", line 186, in upgrade 2017-01-03 07:34:55,736 INFO: Notice: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]/returns: return _migrate(url, repository, version, upgrade=True, err=err, **opts) 2017-01-03 07:34:55,736 INFO: Notice: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]/returns: File "<string>", line 2, in _migrate 2017-01-03 07:34:55,736 INFO: Notice: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]/returns: File "/usr/lib/python2.7/site-packages/migrate/versioning/util/__init__.py", line 160, in with_engine 2017-01-03 07:34:55,736 INFO: Notice: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]/returns: return f(*a, **kw) 2017-01-03 07:34:55,737 INFO: Notice: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]/returns: File "/usr/lib/python2.7/site-packages/migrate/versioning/api.py", line 366, in _migrate 2017-01-03 07:34:55,737 INFO: Notice: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]/returns: schema.runchange(ver, change, changeset.step) 2017-01-03 07:34:55,737 INFO: Notice: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]/returns: File "/usr/lib/python2.7/site-packages/migrate/versioning/schema.py", line 93, in runchange 2017-01-03 07:34:55,737 INFO: Notice: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]/returns: change.run(self.engine, step) 2017-01-03 07:34:55,738 INFO: Notice: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]/returns: File "/usr/lib/python2.7/site-packages/migrate/versioning/script/py.py", line 148, in run 2017-01-03 07:34:55,738 INFO: Notice: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]/returns: script_func(engine) 2017-01-03 07:34:55,738 INFO: Notice: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]/returns: File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api_migrations/migrate_repo/versions/030_require_cell_setup.py", line 41, in upgrade 2017-01-03 07:34:55,738 INFO: Notice: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]/returns: raise exception.ValidationError(detail=msg) 2017-01-03 07:34:55,739 INFO: Notice: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]/returns: ValidationError: Cell mappings are not created, but required for Ocata. Please run nova-manage cell_v2 simple_cell_setup before continuing. Expected results: Additional info: Manually running the nova-manage cell_v2 simple_cell_setup as suggested isn't working in my environment. This bug is already reported upstream and there are reviews in place from Alex Schultz (adding the bug to external trackers). The reviews are actually against puppet-nova/tripleo/openstacklib as well as tripleo-heat-templates and instack-undercloud. I have set instack-undercloud as the component here for now and we will use this BZ to track the right things land where they are needed for the OSP11 undercloud upgrade
tried manually running the simple_cell_setup (both with and without the --transport-url) but I still get the error running the api_db sync: [stack@instack ~]$ sudo nova-manage cell_v2 simple_cell_setup --transport-url "rabbit://97ae6497ff5c3436313e8dd28bc1c008ce8f187d:f21294fb551cf49ab6aff3574cb84cda66486795.24.1//" Already setup, nothing to do. [stack@instack ~]$ sudo /usr/bin/nova-manage -v api_db sync Option "verbose" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future. An error has occurred: Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/nova/cmd/manage.py", line 1584, in main ret = fn(*fn_args, **fn_kwargs) File "/usr/lib/python2.7/site-packages/nova/cmd/manage.py", line 948, in sync return migration.db_sync(version, database='api') File "/usr/lib/python2.7/site-packages/nova/db/migration.py", line 26, in db_sync return IMPL.db_sync(version=version, database=database, context=context) File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/migration.py", line 57, in db_sync repository, version) File "/usr/lib/python2.7/site-packages/migrate/versioning/api.py", line 186, in upgrade return _migrate(url, repository, version, upgrade=True, err=err, **opts) File "<string>", line 2, in _migrate File "/usr/lib/python2.7/site-packages/migrate/versioning/util/__init__.py", line 160, in with_engine return f(*a, **kw) File "/usr/lib/python2.7/site-packages/migrate/versioning/api.py", line 366, in _migrate schema.runchange(ver, change, changeset.step) File "/usr/lib/python2.7/site-packages/migrate/versioning/schema.py", line 93, in runchange change.run(self.engine, step) File "/usr/lib/python2.7/site-packages/migrate/versioning/script/py.py", line 148, in run script_func(engine) File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api_migrations/migrate_repo/versions/030_require_cell_setup.py", line 41, in upgrade raise exception.ValidationError(detail=msg) ValidationError: Cell mappings are not created, but required for Ocata. Please run nova-manage cell_v2 simple_cell_setup before continuing.
Hi, putting this to POST. Marios I think it's safe to move it to closed, but waiting for your definitive input :) Thanks,
yeah been running for a while without any workarounds downstream OSP10 to OSP11 latest puddle should go ON_QA ... setting.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2017:1245
This issue happen again upgrading from OSP10z6 to OSP11 (FFU) The workaround is to run "nova-manage cell_v2 simple_cell_setup" manually and run the upgrade again Packages version after the update: openstack-tripleo-heat-templates-6.2.12-2.el7ost.noarch python-nova-15.1.0-3.el7ost.noarch openstack-nova-common-15.1.0-3.el7ost.noarch 2018-07-26 03:20:16,799 INFO: Notice: /Stage[main]/Nova::Deps/Anchor[nova::db::end]: Triggered 'refresh' from 2 events 2018-07-26 03:20:16,800 INFO: Notice: /Stage[main]/Nova::Deps/Anchor[nova::dbsync_api::begin]: Triggered 'refresh' from 1 events 2018-07-26 03:21:41,503 INFO: Notice: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]/returns: An error has occurred: 2018-07-26 03:21:41,503 INFO: Notice: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]/returns: Traceback (most recent call last): 2018-07-26 03:21:41,504 INFO: Notice: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]/returns: File "/usr/lib/python2.7/site-packages/nova/cmd/manage.py", line 1678, in main 2018-07-26 03:21:41,504 INFO: Notice: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]/returns: ret = fn(*fn_args, **fn_kwargs) 2018-07-26 03:21:41,504 INFO: Notice: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]/returns: File "/usr/lib/python2.7/site-packages/nova/cmd/manage.py", line 820, in sync 2018-07-26 03:21:41,504 INFO: Notice: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]/returns: return migration.db_sync(version, database='api') 2018-07-26 03:21:41,505 INFO: Notice: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]/returns: File "/usr/lib/python2.7/site-packages/nova/db/migration.py", line 26, in db_sync 2018-07-26 03:21:41,505 INFO: Notice: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]/returns: return IMPL.db_sync(version=version, database=database, context=context) 2018-07-26 03:21:41,505 INFO: Notice: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]/returns: File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/migration.py", line 57, in db_sync 2018-07-26 03:21:41,505 INFO: Notice: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]/returns: repository, version) 2018-07-26 03:21:41,506 INFO: Notice: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]/returns: File "/usr/lib/python2.7/site-packages/migrate/versioning/api.py", line 186, in upgrade 2018-07-26 03:21:41,506 INFO: Notice: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]/returns: return _migrate(url, repository, version, upgrade=True, err=err, **opts) 2018-07-26 03:21:41,506 INFO: Notice: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]/returns: File "<string>", line 2, in _migrate 2018-07-26 03:21:41,506 INFO: Notice: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]/returns: File "/usr/lib/python2.7/site-packages/migrate/versioning/util/__init__.py", line 160, in with_engine 2018-07-26 03:21:41,506 INFO: Notice: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]/returns: return f(*a, **kw) 2018-07-26 03:21:41,507 INFO: Notice: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]/returns: File "/usr/lib/python2.7/site-packages/migrate/versioning/api.py", line 366, in _migrate 2018-07-26 03:21:41,507 INFO: Notice: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]/returns: schema.runchange(ver, change, changeset.step) 2018-07-26 03:21:41,507 INFO: Notice: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]/returns: File "/usr/lib/python2.7/site-packages/migrate/versioning/schema.py", line 93, in runchange 2018-07-26 03:21:41,507 INFO: Notice: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]/returns: change.run(self.engine, step) 2018-07-26 03:21:41,508 INFO: Notice: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]/returns: File "/usr/lib/python2.7/site-packages/migrate/versioning/script/py.py", line 148, in run 2018-07-26 03:21:41,508 INFO: Notice: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]/returns: script_func(engine) 2018-07-26 03:21:41,508 INFO: Notice: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]/returns: File "/usr/lib/python2.7/site-packages/nova/db/sqlalchemy/api_migrations/migrate_repo/versions/030_require_cell_setup.py", line 44, in upgrade 2018-07-26 03:21:41,508 INFO: Notice: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]/returns: raise exception.ValidationError(detail=msg) 2018-07-26 03:21:41,509 INFO: Notice: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]/returns: ValidationError: Cell mappings are not created, but required for Ocata. Please run nova-manage cell_v2 simple_cell_setup before continuing. 2018-07-26 03:21:41,509 INFO: Error: /usr/bin/nova-manage api_db sync returned 1 instead of one of [0] 2018-07-26 03:21:41,509 INFO: Error: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]/returns: change from notrun to 0 failed: /usr/bin/nova-manage api_db sync returned 1 instead of one of [0] 2018-07-26 03:23:06,041 INFO: Error: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]: Failed to call refresh: /usr/bin/nova-manage api_db sync returned 1 instead of one of [0] 2018-07-26 03:23:06,041 INFO: Error: /Stage[main]/Nova::Db::Sync_api/Exec[nova-db-sync-api]: /usr/bin/nova-manage api_db sync returned 1 instead of one of [0] 2018-07-26 03:23:06,041 INFO: Notice: /Stage[main]/Nova::Deps/Anchor[nova::dbsync_api::end]: Dependency Exec[nova-db-sync-api] has failures: true 2018-07-26 03:23:06,042 INFO: Warning: /Stage[main]/Nova::Deps/Anchor[nova::dbsync_api::end]: Skipping because of failed dependencies