Bug 1599764

Summary:	FFU: Launching an instance post upgrade when Nova services are running on a different role than controllers fails
Product:	Red Hat OpenStack	Reporter:	Marius Cornea <mcornea>
Component:	openstack-tripleo-heat-templates	Assignee:	Lukas Bezdicka <lbezdick>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Ronnie Rasouli <rrasouli>
Severity:	urgent	Docs Contact:
Priority:	urgent
Version:	13.0 (Queens)	CC:	amodi, augol, ccamacho, dasmith, dbecker, eglynn, emacchi, emilien, jfrancoa, jhakimra, jschluet, jstransk, kchamart, lbezdick, lyarwood, mburns, mcornea, morazi, nova-maint, owalsh, sbauza, sgordon, slinaber, srevivo, vromanso
Target Milestone:	z7	Keywords:	TestOnly, Triaged, ZStream
Target Release:	13.0 (Queens)
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	openstack-tripleo-heat-templates-8.3.1-34.el7ost	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2019-09-04 10:44:43 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1609966, 1651136, 1723176
Bug Blocks:

Description Marius Cornea 2018-07-10 14:10:49 UTC

Description of problem:

FFU: Launching an instance post upgrade when Nova services are running on a different role than controllers fails

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1. Deploy OSP10 with 3 controllers + 2 serviceapi + 2 computes + 3 ceph nodes; controllers are running pacemaker managed services only and serviceapi nodes are running systemd managed services(including all nova services)
 
2. Upgrade the environment via FFU procedure

3. Once upgrade completes successful launch an instance on the overcloud

Actual results:
Instance fails to start.

Expected results:
Instance start successfully. 

Additional info:
Attaching links to job and artifacts including logs.

Comment 8 Joanne O'Flynn 2018-08-13 10:06:00 UTC

This bug is marked for inclusion in the errata but does not currently contain draft documentation text. To ensure the timely release of this advisory please provide draft documentation text for this bug as soon as possible.

If you do not think this bug requires errata documentation, set the requires_doc_text flag to "-".


To add draft documentation text:

* Select the documentation type from the "Doc Type" drop down field.

* A template will be provided in the "Doc Text" field based on the "Doc Type" value selected. Enter draft text in the "Doc Text" field.

Comment 17 Jiri Stransky 2018-09-06 13:25:25 UTC

We were able to reproduce authentication issues even without FFU. After some debugging, it seems that depending on the service composition/ordering in roles data, sometimes Nova can end up with:

"nova::keystone::authtoken::auth_url": "http://192.168.24.10:35357"

while all other services have:

"swift::proxy::authtoken::auth_url": "http://172.17.1.10:5000"

This looks like a missing backport.

Comment 18 Jiri Stransky 2018-09-06 13:26:11 UTC

Backport posted: https://review.openstack.org/#/c/600444/

Comment 20 Carlos Camacho 2018-09-07 13:30:34 UTC

The fix is on the gate.

Comment 21 Jiri Stransky 2018-09-07 13:35:10 UTC

Master gate though :) I'll be backporting to Rocky when it lands to master.

Comment 23 Jiri Stransky 2018-10-16 10:05:05 UTC

Included in upstream t-h-t 8.0.7 so this must have gone in with the rebase, setting to MODIFIED.

Comment 26 Marius Cornea 2018-11-02 17:21:56 UTC

The issue is still present, I'm wondering if this is not related to https://bugzilla.redhat.com/1626140

Comment 27 Steve Linabery 2018-11-06 22:34:06 UTC

(In reply to Marius Cornea from comment #26)
> The issue is still present, I'm wondering if this is not related to
> https://bugzilla.redhat.com/1626140

Is there any workaround for this issue? Should this be considered a blocker for shipping osp13 next z-stream update? Thanks.

Comment 28 Marius Cornea 2018-11-06 23:00:31 UTC

(In reply to Steve Linabery from comment #27)
> (In reply to Marius Cornea from comment #26)
> > The issue is still present, I'm wondering if this is not related to
> > https://bugzilla.redhat.com/1626140
> 
> Is there any workaround for this issue? Should this be considered a blocker
> for shipping osp13 next z-stream update? Thanks.

There is no workaround for this issue. Given that this scenario has never worked properly I don't think it should be considered a blocker.

Comment 42 Lon Hohberger 2019-09-04 10:44:43 UTC

According to our records, this should be resolved by openstack-tripleo-heat-templates-8.3.1-79.el7ost.  This build is available now.