Bug 1572259

Summary: openstack undercloud upgrade fails when overcloud has Failed state
Product: Red Hat OpenStack Reporter: Sergii Golovatiuk <sgolovat>
Component: rhosp-directorAssignee: RHOS Maint <rhos-maint>
Status: CLOSED DUPLICATE QA Contact: Amit Ugol <augol>
Severity: high Docs Contact:
Priority: unspecified    
Version: 13.0 (Queens)CC: dbecker, mburns, morazi
Target Milestone: ---Keywords: Upgrades
Target Release: ---   
Hardware: All   
OS: All   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-04-26 14:28:33 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Sergii Golovatiuk 2018-04-26 14:25:29 UTC
Description of problem:

openstack undercloud upgrade

fails when overcloud stack was in Failed state with the following error

How reproducible:
All the time

Steps to Reproduce:
Install RHOSP12
Install overcloud
make it in failed state
run: openstack undercloud upgrade

Actual results:

#############
TypeError: 'NoneType' object has no attribute '__getitem__'


2018-04-26 10:14:57,328 DEBUG: An exception occurred
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py", line 2336, in install
    _post_config(instack_env, upgrade)
  File "/usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py", line 2042, in _post_config
    _migrate_to_convergence(heat)
  File "/usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py", line 1982, in _migrate_to_convergence
    _run_command(args, name='heat-manage')
  File "/usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py", line 642, in _run_command
    env=env).decode('utf-8')
  File "/usr/lib64/python2.7/subprocess.py", line 575, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
CalledProcessError: Command '['sudo', '-E', 'heat-manage', 'migrate_convergence_1', u'1e0047ea-0462-43c9-a7a3-aea13aa49789']' returned non-zero exit status 1
2018-04-26 10:14:57,329 ERROR:
#############################################################################
Undercloud upgrade failed.

Reason: Command '['sudo', '-E', 'heat-manage', 'migrate_convergence_1', u'1e0047ea-0462-43c9-a7a3-aea13aa49789']' returned non-zero exit status 1

See the previous output for details about what went wrong.  The full install
log can be found at /home/stack/.instack/install-undercloud.log.

#############################################################################

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py", line 2336, in install
    _post_config(instack_env, upgrade)
  File "/usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py", line 2042, in _post_config
    _migrate_to_convergence(heat)
  File "/usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py", line 1982, in _migrate_to_convergence
    _run_command(args, name='heat-manage')
  File "/usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py", line 642, in _run_command
    env=env).decode('utf-8')
  File "/usr/lib64/python2.7/subprocess.py", line 575, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command '['sudo', '-E', 'heat-manage', 'migrate_convergence_1', u'1e0047ea-0462-43c9-a7a3-aea13aa49789']' returned non-zero exit status 1
Command 'instack-upgrade-undercloud' returned non-zero exit status 1
###############


undercloud upgrade failed.

Expected results:
Detect the state of overcloud and not to run upgrade. Ask operator to fix overcloud stack.

Additional info:

Comment 1 Sergii Golovatiuk 2018-04-26 14:28:33 UTC

*** This bug has been marked as a duplicate of bug 1572257 ***