Bug 1599764 - FFU: Launching an instance post upgrade when Nova services are running on a different role than controllers fails
Summary: FFU: Launching an instance post upgrade when Nova services are running on a d...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates
Version: 13.0 (Queens)
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: z7
: 13.0 (Queens)
Assignee: Lukas Bezdicka
QA Contact: Ronnie Rasouli
URL:
Whiteboard:
Depends On: 1609966 1651136 1723176
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-07-10 14:10 UTC by Marius Cornea
Modified: 2019-09-04 10:44 UTC (History)
25 users (show)

Fixed In Version: openstack-tripleo-heat-templates-8.3.1-34.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-09-04 10:44:43 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1767139 0 None None None 2018-07-10 14:13:15 UTC
Launchpad 1791093 0 None None None 2018-09-06 14:25:31 UTC
Launchpad 1828590 0 None None None 2019-05-10 14:48:14 UTC
OpenStack gerrit 566161 0 None stable/queens: MERGED nova: Handle @safe_connect returns None side effect in _ensure_resource_provider (If9e1581db9c1ae14340b787d03c815d243d5a... 2018-11-07 13:12:03 UTC
OpenStack gerrit 600444 0 None stable/queens: MERGED tripleo-heat-templates: Don't use keystone admin endpoint for nova placement (I9aabf0ac8f1fb21f835937fa4354cad3c8443211) 2018-11-07 13:11:55 UTC
OpenStack gerrit 658389 0 None None None 2019-05-10 15:11:35 UTC

Description Marius Cornea 2018-07-10 14:10:49 UTC
Description of problem:

FFU: Launching an instance post upgrade when Nova services are running on a different role than controllers fails

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1. Deploy OSP10 with 3 controllers + 2 serviceapi + 2 computes + 3 ceph nodes; controllers are running pacemaker managed services only and serviceapi nodes are running systemd managed services(including all nova services)
 
2. Upgrade the environment via FFU procedure

3. Once upgrade completes successful launch an instance on the overcloud

Actual results:
Instance fails to start.

Expected results:
Instance start successfully. 

Additional info:
Attaching links to job and artifacts including logs.

Comment 8 Joanne O'Flynn 2018-08-13 10:06:00 UTC
This bug is marked for inclusion in the errata but does not currently contain draft documentation text. To ensure the timely release of this advisory please provide draft documentation text for this bug as soon as possible.

If you do not think this bug requires errata documentation, set the requires_doc_text flag to "-".


To add draft documentation text:

* Select the documentation type from the "Doc Type" drop down field.

* A template will be provided in the "Doc Text" field based on the "Doc Type" value selected. Enter draft text in the "Doc Text" field.

Comment 17 Jiri Stransky 2018-09-06 13:25:25 UTC
We were able to reproduce authentication issues even without FFU. After some debugging, it seems that depending on the service composition/ordering in roles data, sometimes Nova can end up with:

"nova::keystone::authtoken::auth_url": "http://192.168.24.10:35357"

while all other services have:

"swift::proxy::authtoken::auth_url": "http://172.17.1.10:5000"

This looks like a missing backport.

Comment 18 Jiri Stransky 2018-09-06 13:26:11 UTC
Backport posted: https://review.openstack.org/#/c/600444/

Comment 20 Carlos Camacho 2018-09-07 13:30:34 UTC
The fix is on the gate.

Comment 21 Jiri Stransky 2018-09-07 13:35:10 UTC
Master gate though :) I'll be backporting to Rocky when it lands to master.

Comment 23 Jiri Stransky 2018-10-16 10:05:05 UTC
Included in upstream t-h-t 8.0.7 so this must have gone in with the rebase, setting to MODIFIED.

Comment 26 Marius Cornea 2018-11-02 17:21:56 UTC
The issue is still present, I'm wondering if this is not related to https://bugzilla.redhat.com/1626140

Comment 27 Steve Linabery 2018-11-06 22:34:06 UTC
(In reply to Marius Cornea from comment #26)
> The issue is still present, I'm wondering if this is not related to
> https://bugzilla.redhat.com/1626140

Is there any workaround for this issue? Should this be considered a blocker for shipping osp13 next z-stream update? Thanks.

Comment 28 Marius Cornea 2018-11-06 23:00:31 UTC
(In reply to Steve Linabery from comment #27)
> (In reply to Marius Cornea from comment #26)
> > The issue is still present, I'm wondering if this is not related to
> > https://bugzilla.redhat.com/1626140
> 
> Is there any workaround for this issue? Should this be considered a blocker
> for shipping osp13 next z-stream update? Thanks.

There is no workaround for this issue. Given that this scenario has never worked properly I don't think it should be considered a blocker.

Comment 42 Lon Hohberger 2019-09-04 10:44:43 UTC
According to our records, this should be resolved by openstack-tripleo-heat-templates-8.3.1-79.el7ost.  This build is available now.


Note You need to log in before you can comment on or make changes to this bug.