Created attachment 1583942 [details] controller update log Description of problem: overcloud update fails with this error: 2019-06-24 03:39:45 | fatal: [controller-2]: FAILED! => {"changed": true, "cmd": "nova-manage cell_v2 list_cells --verbose | grep 'mysql+pymysql://nova:UCRUMjFVXPNQuGgNeUbRAwuTU.1.29/nova_cell0?read_default_group=tripleo&read_default_file=/etc/my.cnf.d/tripleo.cnf' || nova-manage cell_v2 update_cell --cell_uuid 00000000-0000-0000-0000-000000000000 --transport-url 'none:///' --database_connection='mysql+pymysql://nova:UCRUMjFVXPNQuGgNeUbRAwuTU.1.29/nova_cell0?read_default_group=tripleo&read_default_file=/etc/my.cnf.d/tripleo.cnf'", "delta": "0:00:08.022272", "end": "2019-06-24 07:39:43.993213", "msg": "non-zero return code", "rc": 1, "start": "2019-06-24 07:39:35.970941", "stderr": "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py:332: NotSupportedWarning: Configuration option(s) ['use_tpool'] not supported\n exception.NotSupportedWarning\n/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py:332: NotSupportedWarning: Configuration option(s) ['use_tpool'] not supported\n exception.NotSupportedWarning", "stderr_lines": ["/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py:332: NotSupportedWarning: Configuration option(s) ['use_tpool'] not supported", " exception.NotSupportedWarning", "/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py:332: NotSupportedWarning: Configuration option(s) ['use_tpool'] not supported", " exception.NotSupportedWarning"], "stdout": "An error has occurred:\nTraceback (most recent call last):\n File \"/usr/lib/python2.7/site-packages/nova/cmd/manage.py\", line 1654, in main\n ret = fn(*fn_args, **fn_kwargs)\n File \"/usr/lib/python2.7/site-packages/nova/cmd/manage.py\", line 1499, in update_cell\n cell_mapping = objects.CellMapping.get_by_uuid(ctxt, cell_uuid)\n File \"/usr/lib/python2.7/site-packages/oslo_versionedobjects/base.py\", line 184, in wrapper\n result = fn(cls, context, *args, **kwargs)\n File \"/usr/lib/python2.7/site-packages/nova/objects/cell_mapping.py\", line 65, in get_by_uuid\n db_mapping = cls._get_by_uuid_from_db(context, uuid)\n File \"/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py\", line 987, in wrapper\n with self._transaction_scope(context):\n File \"/usr/lib64/python2.7/contextlib.py\", line 17, in __enter__\n return self.gen.next()\n File \"/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py\", line 1037, in _transaction_scope\n context=context) as resource:\n File \"/usr/lib64/python2.7/contextlib.py\", line 17, in __enter__\n return self.gen.next()\n File \"/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py\", line 640, in _session\n bind=self.connection, mode=self.mode)\n File \"/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py\", line 404, in _create_session\n self._start()\n File \"/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py\", line 491, in _start\n engine_args, maker_args)\n File \"/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py\", line 513, in _setup_for_connection\n \"No sql_connection parameter is established\")\nCantStartEngineError: No sql_connection parameter is established", "stdout_lines": ["An error has occurred:", "Traceback (most recent call last):", " File \"/usr/lib/python2.7/site-packages/nova/cmd/manage.py\", line 1654, in main", " ret = fn(*fn_args, **fn_kwargs)", " File \"/usr/lUpdate failed with: Ansible failed, check log at /var/lib/mistral/8bd5ee37-71f5-4ed3-b84c-ac854c0e6235/ansible.log. 2019-06-24 03:39:45 | ib/python2.7/site-packages/nova/cmd/manage.py\", line 1499, in update_cell", " cell_mapping = objects.CellMapping.get_by_uuid(ctxt, cell_uuid)", " File \"/usr/lib/python2.7/site-packages/oslo_versionedobjects/base.py\", line 184, in wrapper", " result = fn(cls, context, *args, **kwargs)", " File \"/usr/lib/python2.7/site-packages/nova/objects/cell_mapping.py\", line 65, in get_by_uuid", " db_mapping = cls._get_by_uuid_from_db(context, uuid)", " File \"/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py\", line 987, in wrapper", " with self._transaction_scope(context):", " File \"/usr/lib64/python2.7/contextlib.py\", line 17, in __enter__", " return self.gen.next()", " File \"/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py\", line 1037, in _transaction_scope", " context=context) as resource:", " File \"/usr/lib64/python2.7/contextlib.py\", line 17, in __enter__", " return self.gen.next()", " File \"/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py\", line 640, in _session", " bind=self.connection, mode=self.mode)", " File \"/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py\", line 404, in _create_session", " self._start()", " File \"/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py\", line 491, in _start", " engine_args, maker_args)", " File \"/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py\", line 513, in _setup_for_connection", " \"No sql_connection parameter is established\")", "CantStartEngineError: No sql_connection parameter is established"]} Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. install osp13 z5 2. update to passed_phase 2 - currently 2019-06-20.1 3. Actual results: Expected results: Additional info: controller update log is attached
more logs can be found in this url: https://rhos-qe-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/OSPD-Customized-Update-virt/43/artifact/
The same issue happened when running minor update from z4 up to passed_phase2 / 2019-06-20.1 The job https://rhos-ci-staging-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/DFG-edge-day2-13-rhel-virthost-3cont-spine-2leaf-2comp-update/1/
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:1738