Hide Forgot
[rhel-osp-director][upgrade to 8.0][negative]: unable to resume 'openstack undercloud upgrade' in case power-outage occurred during the process. Environment ( after upgrade to 8.0): ------------ instack-undercloud-2.2.7-4.el7ost.noarch instack-0.0.8-2.el7ost.noarch python-heatclient-1.0.0-1.el7ost.noarch openstack-heat-engine-5.0.1-5.el7ost.noarch openstack-heat-templates-0-0.8.20150605git.el7ost.noarch openstack-tripleo-heat-templates-0.8.14-7.el7ost.noarch puppet-3.6.2-2.el7.noarch openstack-tripleo-puppet-elements-0.0.5-1.el7ost.noarch openstack-puppet-modules-7.0.17-1.el7ost.noarch Steps: ------- (1) deploy undercloud 7.3GA (2) update to 8.0 repos : sudo rhos-release -P 8-director (3) sudo yum update -y (4) openstack undercloud upgrade (5) While undercloud upgrade running shutdown the undercloud ( power-off!) (6) Power-on the undercloud and attempt to resume and complete the undercloud upgrade. Results: --------- (1)The command fails with error (see view) . (2)re-try running over the upgrade command will end-up with the same error (no workaround was found) The openstack undercloud upgrade "view": ---------------------------------------- INFO: 2016-04-15 14:00:40,131 -- ############### End stdout/stderr logging ############### ERROR: 2016-04-15 14:00:40,131 -- Hook FAILED. ERROR: 2016-04-15 14:00:40,132 -- Failed running command ['dib-run-parts', u'/tmp/tmpi4uWYL/install.d'] File "/usr/lib/python2.7/site-packages/instack/main.py", line 163, in main em.run() File "/usr/lib/python2.7/site-packages/instack/runner.py", line 79, in run self.run_hook(hook) File "/usr/lib/python2.7/site-packages/instack/runner.py", line 172, in run_hook raise Exception("Failed running command %s" % command) ERROR: 2016-04-15 14:00:40,132 -- None Traceback (most recent call last): File "<string>", line 1, in <module> File "/usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py", line 808, in install _run_instack(instack_env) File "/usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py", line 692, in _run_instack _run_live_command(args, instack_env, 'instack') File "/usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py", line 370, in _run_live_command raise RuntimeError('%s failed. See log for details.' % name) RuntimeError: instack failed. See log for details. Command 'instack-install-undercloud' returned non-zero exit status 1
Intermediate issue. the recovery ability is probably depended on which exact moment the power-outage occurred during the upgrade. the openstack undercloud upgrade command - runs yum action, which in some cases will possible to tweak by remove/re-install the specific packages and proceed to successful undercloud upgrade. In this specific case, we couldn't found a workaround that will allow to proceed and complete the upgrade.
Created attachment 1147750 [details] Full-install-unercloud.log
Resolution of this situation is to restore undercloud.