Bug 1391686 - Undercloud Upgrade OSP 9->10 fails
Summary: Undercloud Upgrade OSP 9->10 fails
Keywords:
Status: CLOSED DUPLICATE of bug 1372040
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: instack-undercloud
Version: 10.0 (Newton)
Hardware: x86_64
OS: Linux
high
high
Target Milestone: ---
: ---
Assignee: James Slagle
QA Contact: Arik Chernetsky
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-11-03 18:52 UTC by Chris Janiszewski
Modified: 2016-12-29 16:54 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-12-02 07:55:32 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
sos report from failed upgrade (15.93 MB, application/x-xz)
2016-11-03 18:52 UTC, Chris Janiszewski
no flags Details
console dump from failed upgrade (94.52 KB, text/plain)
2016-11-03 18:54 UTC, Chris Janiszewski
no flags Details

Description Chris Janiszewski 2016-11-03 18:52:43 UTC
Created attachment 1217118 [details]
sos report from failed upgrade

Description of problem:
Undercloud Upgrade OSP 9->10 fails 

Version-Release number of selected component (if applicable):
OSP 9 to OSP 10 Beta


How reproducible:
Deploy OSP 9 and then upgrade to OSP 10 using:
openstack undercloud upgrade
 

Steps to Reproduce:
1. Deploy OSP9
2. Upgrade to OSP10 with -> openstack undercloud upgrade
3.

Actual results:

2016-11-03 09:05:56 - Notice: Finished catalog run in 949.12 seconds
+ rc=6
+ set -e
+ echo 'puppet apply exited with exit code 6'
puppet apply exited with exit code 6
+ '[' 6 '!=' 2 -a 6 '!=' 0 ']'
+ exit 6
[2016-11-03 09:06:00,373] (os-refresh-config) [ERROR] during configure phase. [Command '['dib-run-parts', '/usr/libexec/os-refresh-config/configure.d']' returned non-zero exit status 1]

[2016-11-03 09:06:00,373] (os-refresh-config) [ERROR] Aborting...
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py", line 1047, in install
    _run_orc(instack_env)
  File "/usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py", line 904, in _run_orc
    _run_live_command(args, instack_env, 'os-refresh-config')
  File "/usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py", line 508, in _run_live_command
    raise RuntimeError('%s failed. See log for details.' % name)
RuntimeError: os-refresh-config failed. See log for details.
Command 'instack-install-undercloud' returned non-zero exit status 1


Expected results:
Successful upgrade 


Additional info:
Please see attached logs from the system. It looks OpenStack-nova-compute and nova-scheduler had a hard time starting up, but they are being up now after the error

Comment 1 Chris Janiszewski 2016-11-03 18:54:18 UTC
Created attachment 1217119 [details]
console dump from failed upgrade

Comment 2 Chris Janiszewski 2016-11-03 19:04:42 UTC
I just re-ran the 'openstack undercloud upgrade', without making any changes and the upgrade finished successfully - it looks like there is a timing issue with how long we wait for services to come back online ?

Comment 3 Sofer Athlan-Guyot 2016-12-02 07:55:32 UTC
Hi,

this is a known behaviour.  The current way to upgrade the undercloud is to run

sudo systemctl stop 'openstack-*'
sudo systemctl stop 'neutron-*'

before the undercloud upgrade.  Services are restarted during the upgrade.

I'm marking this bug as a duplicate of https://bugzilla.redhat.com/show_bug.cgi?id=1372040.  Check this one for details.

*** This bug has been marked as a duplicate of bug 1372040 ***


Note You need to log in before you can comment on or make changes to this bug.