Description of problem: FFU fails on FFU upgrade run, neutron DB sync fails File \"/usr/lib/python2.7/site-packages/alembic/script/revision.py\", line 303, in get_revisions\n for rev_id in resolved_id)\n File \"/usr/lib/python2.7/site-packages/alembic/script/revision.py\", line 303, in <genexpr>\n for rev_id in resolved_id)\n File \"/usr/lib/python2.7/site-packages/alembic/script/revision.py\", line 358, in _revision_for_ident\n resolved_id)\nalembic.script.revision.ResolutionError: No such revision or branch '7db8cd315b95'", "stderr_lines": ["INFO [alembic.runtime.migration] Context impl MySQLImpl.", "INFO [alembic.runtime.migration] Will assume non-transactional DDL.", "INFO [alembic.runtime.migration] Context impl MySQLImpl." Continuing 1692838 Version-Release number of selected component (if applicable): core_puddle: 2019-05-10.1 How reproducible: 100% Steps to Reproduce: 1. deploy osp10 2. upgrade the undercloud FFU to osp13 3. Perform overcloud FFU upgrade Actual results: The FFU overcloud upgrade run fails on neutron DB sync step Expected results: no failures Additional info: TASK [Neutron db sync] ********************************************************* Tuesday 14 May 2019 11:50:24 -0400 (0:00:02.002) 0:19:44.385 *********** fatal: [controller-0]: FAILED! => {"changed": true, "cmd": ["neutron-db-manage", "upgrade", "head"], "delta": "0:00:01.420792", "end": "2019-05-14 15:50:25.825312", "msg": "non-zero return code", "rc": 1, "start": "2019-05-14 15:50:24.404520", "stderr": "INFO [alembic.runtime.migration] Context impl MySQLImpl.\nINFO [alembic.runtime.migration] Will assume non-transactional DDL.\nINFO [alembic.runtime.migration] Context impl MySQLImpl.\nINFO [alembic.runtime.migration] Will assume non-transactional DDL.\nINFO [alembic.runtime.migration] Running upgrade 5cd92597d11d -> 929c968efe70, add_pk_version_table\nINFO [alembic.runtime.migration] Running upgrade 929c968efe70 -> a9c43481023c, extend_pk_with_host_and_add_status_to_ml2_port_binding\nINFO [alembic.runtime.migration] Context impl MySQLImpl.\nINFO [alembic.runtime.migration] Will assume non-transactional DDL.\nTraceback (most recent call last):\n File \"/bin/neutron-db-manage\", line 10, in <module>\n sys.exit(main())\n File \"/usr/lib/python2.7/site-packages/neutron/db/migration/cli.py\", line 686, in main\n return_val |= bool(CONF.command.func(config, CONF.command.name))\n File \"/usr/lib/python2.7/site-packages/neutron/db/migration/cli.py\", line 205, in do_upgrade\n run_sanity_checks(config, revision)\n File \"/usr/lib/python2.7/site-packages/neutron/db/migration/cli.py\", line 670, in run_sanity_checks\n script_dir.run_env()\n File \"/usr/lib/python2.7/site-packages/alembic/script/base.py\", line 407, in run_env\n util.load_python_file(self.dir, 'env.py')\n File \"/usr/lib/python2.7/site-packages/alembic/util/pyfiles.py\", line 93, in load_python_file\n module = load_module_py(module_id, path)\n File \"/usr/lib/python2.7/site-packages/alembic/util/compat.py\", line 79, in load_module_py\n mod = imp.load_source(module_id, path, fp)\n File \"/usr/lib/python2.7/site-packages/networking_bigswitch/plugins/bigswitch/db/migration/alembic_migrations/env.py\", line 86, in <module>\n run_migrations_online()\n File \"/usr/lib/python2.7/site-packages/networking_bigswitch/plugins/bigswitch/db/migration/alembic_migrations/env.py\", line 77, in run_migrations_online\n context.run_migrations()\n File \"<string>\", line 8, in run_migrations\n File \"/usr/lib/python2.7/site-packages/alembic/runtime/environment.py\", line 797, in run_migrations\n self.get_context().run_migrations(**kw)\n File \"/usr/lib/python2.7/site-packages/alembic/runtime/migration.py\", line 303, in run_migrations\n for step in self._migrations_fn(heads, self):\n File \"/usr/lib/python2.7/site-packages/neutron/db/migration/cli.py\", line 661, in check_sanity\n revision, rev, implicit_base=True):\n File \"/usr/lib/python2.7/site-packages/alembic/script/revision.py\", line 641, in _iterate_revisions\n requested_lowers = self.get_revisions(lower)\n File \"/usr/lib/python2.7/site-packages/alembic/script/revision.py\", line 298, in get_revisions\n return sum([self.get_revisions(id_elem) for id_elem in id_], ())\n File \"/usr/lib/python2.7/site-packages/alembic/script/revision.py\", line 303, in get_revisions\n for rev_id in resolved_id)\n File \"/usr/lib/python2.7/site-packages/alembic/script/revision.py\", line 303, in <genexpr>\n for rev_id in resolved_id)\n File \"/usr/lib/python2.7/site-packages/alembic/script/revision.py\", line 358, in _revision_for_ident\n resolved_id)\nalembic.script.revision.ResolutionError: No such revision or branch '7db8cd315b95'", "stderr_lines": ["INFO [alembic.runtime.migration] Context impl MySQLImpl.", "INFO [alembic.runtime.migration] Will assume non-transactional DDL.", "INFO [alembic.runtime.migration] Context impl MySQLImpl.", "INFO [alembic.runtime.migration] Will assume non-transactional DDL.", "INFO [alembic.runtime.migration] Running upgrade 5cd92597d11d -> 929c968efe70, add_pk_version_table", "INFO [alembic.runtime.migration] Running upgrade 929c968efe70 -> a9c43481023c, extend_pk_with_host_and_add_status_to_ml2_port_binding", "INFO [alembic.runtime.migration] Context impl MySQLImpl.", "INFO [alembic.runtime.migration] Will assume non-transactional DDL.", "Traceback (most recent call last):", " File \"/bin/neutron-db-manage\", line 10, in <module>", " sys.exit(main())", " File \"/usr/lib/python2.7/site-packages/neutron/db/migration/cli.py\", line 686, in main", " return_val |= bool(CONF.command.func(config, CONF.command.name))", " File \"/usr/lib/python2.7/site-packages/neutron/db/migration/cli.py\", line 205, in do_upgrade", " run_sanity_checks(config, revision)", " File \"/usr/lib/python2.7/site-packages/neutron/db/migration/cli.py\", line 670, in run_sanity_checks", " script_dir.run_env()", " File \"/usr/lib/python2.7/site-packages/alembic/script/base.py\", line 407, in run_env", " util.load_python_file(self.dir, 'env.py')", " File \"/usr/lib/python2.7/site-packages/alembic/util/pyfiles.py\", line 93, in load_python_file", " module = load_module_py(module_id, path)", " File \"/usr/lib/python2.7/site-packages/alembic/util/compat.py\", line 79, in load_module_py", " mod = imp.load_source(module_id, path, fp)", " File \"/usr/lib/python2.7/site-packages/networking_bigswitch/plugins/bigswitch/db/migration/alembic_migrations/env.py\", line 86, in <module>", " run_migrations_online()", " File \"/usr/lib/python2.7/site-packages/networking_bigswitch/plugins/bigswitch/db/migration/alembic_migrations/env.py\", line 77, in run_migrations_online", " context.run_migrations()", " File \"<string>\", line 8, in run_migrations", " File \"/usr/lib/python2.7/site-packages/alembic/runtime/environment.py\", line 797, in run_migrations", " self.get_context().run_migrations(**kw)", " File \"/usr/lib/python2.7/site-packages/alembic/runtime/migration.py\", line 303, in run_migrations", " for step in self._migrations_fn(heads, self):", " File \"/usr/lib/python2.7/site-packages/neutron/db/migration/cli.py\", line 661, in check_sanity", " revision, rev, implicit_base=True):", " File \"/usr/lib/python2.7/site-packages/alembic/script/revision.py\", line 641, in _iterate_revisions", " requested_lowers = self.get_revisions(lower)", " File \"/usr/lib/python2.7/site-packages/alembic/script/revision.py\", line 298, in get_revisions", " return sum([self.get_revisions(id_elem) for id_elem in id_], ())", " File \"/usr/lib/python2.7/site-packages/alembic/script/revision.py\", line 303, in get_revisions", " for rev_id in resolved_id)", " File \"/usr/lib/python2.7/site-packages/alembic/script/revision.py\", line 303, in <genexpr>", " for rev_id in resolved_id)", " File \"/usr/lib/python2.7/site-packages/alembic/script/revision.py\", line 358, in _revision_for_ident", " resolved_id)", "alembic.script.revision.ResolutionError: No such revision or branch '7db8cd315b95'"], "stdout": "Running upgrade for neutron ...\nOK", "stdout_lines": ["Running upgrade for neutron ...", "OK"]}
The migration 7db8cd315b95 comes from the python-networking-bigswitch package. * OSP 10 - existed previously * OSP 11 - now included in python-networking-bigswitch-10.46.4-6.el7ost; see https://bugzilla.redhat.com/show_bug.cgi?id=1692838 * OSP 12 - now included in python-networking-bigswitch-11.47.2-7.el7ost; see https://bugzilla.redhat.com/show_bug.cgi?id=1702767 * OSP 13 - existed previously Can you try the FFU with the listed packages included at their respective phases? If so then the 7db8cd315b95 error should be gone.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:1721