1391730 – overcloud upgrade failed OSP 9->10 - no dhcp and rebuilding

Bug 1391730 - overcloud upgrade failed OSP 9->10 - no dhcp and rebuilding

Summary: overcloud upgrade failed OSP 9->10 - no dhcp and rebuilding

Keywords:
Status:	CLOSED WORKSFORME
Alias:	None
Product:	Red Hat OpenStack
Classification:	Red Hat
Component:	instack-undercloud
Sub Component:
Version:	10.0 (Newton)
Hardware:	x86_64
OS:	Linux
Priority:	high
Severity:	high
Target Milestone:	---
Target Release:	---
Assignee:	James Slagle
QA Contact:	Arik Chernetsky
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2016-11-03 20:50 UTC by Chris Janiszewski
Modified:	2017-02-10 09:24 UTC (History)
CC List:	5 users (show)
Fixed In Version:
Doc Type:	If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:	2017-02-10 09:24:25 UTC
Target Upstream Version:

Attachments	(Terms of Use)
sosreport, templates, screenshots and console dump (19.30 MB, application/x-gzip) 2016-11-03 20:50 UTC, Chris Janiszewski	no flags	Details
View All Add an attachment (proposed patch, testcase, etc.)

Description Chris Janiszewski 2016-11-03 20:50:29 UTC

Created attachment 1217152 [details]
sosreport, templates, screenshots and console dump

Description of problem:
overcloud upgrade failed OSP 9->10 
after updating images in glance and trying to run overcloud deploy, the overcloud seems to be rebuilding instead of upgrading but there is no dhcp responses serving pxe boot. Please see logs for details

Version-Release number of selected component (if applicable):
OSP 9 -> 10

How reproducible:
upgrade overcloud from 9 -> 10

Steps to Reproduce:
Followed steps from https://access.redhat.com/documentation/en/red-hat-openstack-platform/9/paged/upgrading-red-hat-openstack-platform/chapter-3-director-based-environments-performing-upgrades-to-major-versions 
with obvious changes that would cover OSP 10
1. uploaded and updated images in glance for osp 10
2. updated a templates to osp 10
3. re-deployed overcloud with the same parameters 

Actual results:
2016-11-03 15:13:55 [NovaCompute]: UPDATE_FAILED Error: resources.NovaCompute: Rebuilding server failed, status 'ERROR'
2016-11-03 15:13:55 [CephStorage]: UPDATE_FAILED Error: resources.CephStorage: Rebuilding server failed, status 'ERROR'
2016-11-03 15:13:55 [overcloud-Compute-552rctjcui55-2-tperv57mqzpj]: UPDATE_FAILED Error: resources.NovaCompute: Rebuilding server failed, status 'ERROR'
2016-11-03 15:13:55 [overcloud-CephStorage-mz7xfjsphasa-2-suk44lwoq24f]: UPDATE_FAILED Error: resources.CephStorage: Rebuilding server failed, status 'ERROR'
2016-11-03 15:13:55 [CephStorage]: UPDATE_FAILED Error: resources.CephStorage: Rebuilding server failed, status 'ERROR'
2016-11-03 15:13:55 [NovaCompute]: UPDATE_FAILED Error: resources.NovaCompute: Rebuilding server failed, status 'ERROR'
2016-11-03 15:13:55 [overcloud-CephStorage-mz7xfjsphasa-0-z53tnuwmpobh]: UPDATE_FAILED Error: resources.CephStorage: Rebuilding server failed, status 'ERROR'
2016-11-03 15:13:55 [overcloud-Compute-552rctjcui55-1-joohaycslt2j]: UPDATE_FAILED Error: resources.NovaCompute: Rebuilding server failed, status 'ERROR'
2016-11-03 15:13:55 [CephStorage]: UPDATE_FAILED Error: resources.CephStorage: Rebuilding server failed, status 'ERROR'
2016-11-03 15:13:55 [overcloud-CephStorage-mz7xfjsphasa-1-jcpehymtncr7]: UPDATE_FAILED Error: resources.CephStorage: Rebuilding server failed, status 'ERROR'
2016-11-03 15:13:56 [0]: UPDATE_FAILED resources[0]: Error: resources.CephStorage: Rebuilding server failed, status 'ERROR'
2016-11-03 15:13:56 [NovaCompute]: UPDATE_FAILED Error: resources.NovaCompute: Rebuilding server failed, status 'ERROR'
2016-11-03 15:13:56 [1]: UPDATE_FAILED UPDATE aborted
2016-11-03 15:13:56 [overcloud-Compute-552rctjcui55-0-ee737ackqzoo]: UPDATE_FAILED Error: resources.NovaCompute: Rebuilding server failed, status 'ERROR'
2016-11-03 15:13:56 [2]: UPDATE_FAILED UPDATE aborted
2016-11-03 15:13:56 [overcloud-CephStorage-mz7xfjsphasa]: UPDATE_FAILED resources[0]: Error: resources.CephStorage: Rebuilding server failed, status 'ERROR'
2016-11-03 15:13:56 [0]: UPDATE_FAILED resources[0]: Error: resources.NovaCompute: Rebuilding server failed, status 'ERROR'
2016-11-03 15:13:56 [2]: UPDATE_FAILED UPDATE aborted
2016-11-03 15:13:56 [1]: UPDATE_FAILED UPDATE aborted
2016-11-03 15:13:57 [overcloud-Compute-552rctjcui55]: UPDATE_FAILED resources[0]: Error: resources.NovaCompute: Rebuilding server failed, status 'ERROR'
2016-11-03 15:13:57 [Compute]: UPDATE_FAILED resources.Compute: resources[0]: Error: resources.NovaCompute: Rebuilding server failed, status 'ERROR'
2016-11-03 15:13:58 [Controller]: UPDATE_FAILED UPDATE aborted
2016-11-03 15:13:58 [1]: UPDATE_FAILED UPDATE aborted
2016-11-03 15:13:58 [CephStorage]: UPDATE_FAILED UPDATE aborted
2016-11-03 15:13:58 [overcloud]: UPDATE_FAILED resources.Compute: resources[0]: Error: resources.NovaCompute: Rebuilding server failed, status 'ERROR'
2016-11-03 15:13:58 [2]: UPDATE_FAILED UPDATE aborted
2016-11-03 15:13:59 [0]: UPDATE_FAILED UPDATE aborted
2016-11-03 15:13:59 [overcloud-Controller-234d35efwqoe]: UPDATE_FAILED Operation cancelled
2016-11-03 15:14:00 [Controller]: UPDATE_FAILED UPDATE aborted
2016-11-03 15:14:00 [overcloud-Controller-234d35efwqoe-0-dtgz2hw6ccso]: UPDATE_FAILED Operation cancelled
2016-11-03 15:14:00 [Controller]: UPDATE_FAILED UPDATE aborted
2016-11-03 15:14:00 [Controller]: UPDATE_FAILED UPDATE aborted
2016-11-03 15:14:00 [overcloud-Controller-234d35efwqoe-2-ovpd5hgpljow]: UPDATE_FAILED Operation cancelled
2016-11-03 15:14:00 [overcloud-Controller-234d35efwqoe-1-2eum5ftnt3q3]: UPDATE_FAILED Operation cancelled
Stack overcloud UPDATE_FAILED
Heat Stack update failed.
./deploy.sh: line 6: --ntp-server: command not found


Expected results:
overcloud upgraded successfully 

Additional info:
Attaching sosreport, templates, screenshots and console dump

Comment 1 Sofer Athlan-Guyot 2017-02-10 09:24:25 UTC

Hi Chris,

I'm closing this one as old.  If it's still an issue don't hesitate to re-open the bz.

Regards,

Note You need to log in before you can comment on or make changes to this bug.