Bug 1532753 - OSP11 -> OSP12 upgrade: post-upgrade undercloud upgrade fail: The `HorizonPublic` endpoint is not defined in the `EndpointMap` of the deployed stack.
Summary: OSP11 -> OSP12 upgrade: post-upgrade undercloud upgrade fail: The `HorizonPub...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-validations
Version: 12.0 (Pike)
Hardware: Unspecified
OS: Unspecified
high
urgent
Target Milestone: z3
: 12.0 (Pike)
Assignee: Ryan Brady
QA Contact: Udi Kalifon
URL:
Whiteboard:
Depends On:
Blocks: 1860256
TreeView+ depends on / blocked
 
Reported: 2018-01-09 17:47 UTC by Chris Janiszewski
Modified: 2020-07-24 06:18 UTC (History)
17 users (show)

Fixed In Version: openstack-tripleo-validations-7.4.9-1.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1860256 (view as bug list)
Environment:
Last Closed: 2019-03-13 11:13:49 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
OpenStack gerrit 535789 0 'None' MERGED Invert validation task order 2020-09-28 23:43:41 UTC

Description Chris Janiszewski 2018-01-09 17:47:02 UTC
Description of problem:

Undercloud major upgrade from 11 -> 12 fails validation:

2018-01-09 12:12:39,652 INFO: Configuring an hourly cron trigger for tripleo-ui logging
2018-01-09 12:12:42,138 INFO: Added _member_ role to admin user
2018-01-09 12:12:42,434 INFO: Starting and waiting for validation groups ['post-upgrade'] 
2018-01-09 12:12:59,125 ERROR: ERROR error running the validation groups ['post-upgrade']   {"stderr": "[DEPRECATION WARNING]: DEFAULT_SUDO_FLAGS option, In favor of become which is a\n generic framework . This feature will be removed in version 2.8. Deprecation \nwarnings can be disabled by setting deprecation_warnings=False in ansible.cfg.\n", "stdout": "Task 'fail' failed:\nHost: localhost\nMessage: The `HorizonPublic` endpoint is not defined in the `EndpointMap` of the deployed stack. This means Horizon may not have been deployed correctly.\n\nFailure! The validation failed for all hosts:\n* localhost\n"} {"stderr": "[DEPRECATION WARNING]: DEFAULT_SUDO_FLAGS option, In favor of become which is a\n generic framework . This feature will be removed in version 2.8. Deprecation \nwarnings can be disabled by setting deprecation_warnings=False in ansible.cfg.\n", "stdout": "Task 'Check stack resource statuses' failed:\nHost: localhost\nMessage: An unhandled exception occurred while running the lookup plugin 'stack_resources'. Error was a <class 'heatclient.exc.HTTPNotFound'>, original message: ERROR: The Stack (overcloud) could not be found.\n\nFailure! The validation failed for all hosts:\n* localhost\n"} Mistral execution ID: 1f740d91-472d-4c56-ac30-7a6472fe6dc9
2018-01-09 12:12:59,129 INFO: 
#############################################################################
Undercloud upgrade complete.

The file containing this installation's passwords is at
/home/stack/undercloud-passwords.conf.

There is also a stackrc file at /home/stack/stackrc.

These files are needed to interact with the OpenStack services, and should be
secured.

#############################################################################


This issues seems to be related to:
https://bugzilla.redhat.com/show_bug.cgi?id=1493131

Dan Trainor in Comment 18 mentioned he is going to open another bug report for this, but I couldn't find one, so I am opening one up.


Version-Release number of selected component (if applicable):
[stack@chrisj-undercloud ~]$ rpm -qa | grep validation
openstack-tripleo-validations-7.4.2-1.el7ost.noarch
[stack@chrisj-undercloud ~]$ rpm -qa | grep instack-undercloud
instack-undercloud-7.4.3-5.el7ost.noarch
[stack@chrisj-undercloud ~]$ rpm -qa | grep openstack-tripleo-common
openstack-tripleo-common-containers-7.6.3-8.el7ost.noarch
openstack-tripleo-common-7.6.3-8.el7ost.noarch

How reproducible:


Steps to Reproduce:
1. Deploy OSP11 undercloud 
2. Upgrade to OSP12

Actual results:


Expected results:


Additional info:
sosreport from undercloud (too big to upload):
http://chrisj.cloud/sosreport-chrisj-undercloud.hextupleo.lab-20180109123132.tar.xz

Comment 1 Chris Paquin 2018-01-09 21:41:14 UTC
I was able to recreate.

ERROR error running the validation groups ['post-upgrade']   {"stderr": "[DEPRECATION WARNING]: DEFAULT_SUDO_FLAGS option, In favor of become which is a\n generic framework . This feature will be removed in version 2.8. Deprecation \nwarnings can be disabled by setting deprecation_warnings=False in ansible.cfg.\n", "stdout": "Task 'Check stack resource statuses' failed:\nHost: localhost\nMessage: An unhandled exception occurred while running the lookup plugin 'stack_resources'. Error was a <class 'heatclient.exc.HTTPNotFound'>, original message: ERROR: The Stack (overcloud) could not be found.\n\nFailure! The validation failed for all hosts:\n* localhost\n"} {"stderr": "[DEPRECATION WARNING]: DEFAULT_SUDO_FLAGS option, In favor of become which is a\n generic framework . This feature will be removed in version 2.8. Deprecation \nwarnings can be disabled by setting deprecation_warnings=False in ansible.cfg.\n", "stdout": "Task 'fail' failed:\nHost: localhost\nMessage: The `HorizonPublic` endpoint is not defined in the `EndpointMap` of the deployed stack. This means Horizon may not have been deployed correctly.\n\nFailure! The validation failed for all hosts:\n* localhost\n"} Mistral execution ID: db01b2be-7833-467e-a7ac-9d5f7308422f
2018-01-09 16:25:38,882 INFO:

Comment 2 Chris Paquin 2018-01-09 21:44:55 UTC
Note that I am not using undercloud as my stack name

q^C
[stack@chrisp-undercloud ~]$ heat stack-list
WARNING (shell) "heat stack-list" is deprecated, please use "openstack stack list" instead
+--------------------------------------+------------+-----------------+----------------------+----------------------+----------------------------------+
| id                                   | stack_name | stack_status    | creation_time        | updated_time         | project                          |
+--------------------------------------+------------+-----------------+----------------------+----------------------+----------------------------------+
| e530c566-2ab0-4ae5-9285-1c7c7c68cfe2 | chrisp     | UPDATE_COMPLETE | 2018-01-09T17:21:58Z | 2018-01-09T19:49:58Z | 5a6696d810994de899b976dd0df244d0 |
+--------------------------------------+------------+-----------------+----------------------+----------------------+----------------------------------+

Comment 3 Carlos Camacho 2018-01-24 14:46:46 UTC
Fixed here: https://bugs.launchpad.net/tripleo/+bug/1742947

Comment 5 Radomir Dopieralski 2018-02-02 17:21:02 UTC
I don't think Horizon has an endpoint anymore — I think we don't have it since several releases now, since there is no longer tuskar-ui that would need it.

Comment 6 Radomir Dopieralski 2018-02-05 09:59:01 UTC
Backported downstream.

Comment 11 errata-xmlrpc 2018-08-20 12:53:41 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:2521

Comment 12 Jeremy 2019-03-08 15:47:12 UTC
Still hitting this issue even in osp13 re-opening.


2019-03-08 10:40:28,141 INFO: Starting and waiting for validation groups ['post-upgrade'] 
2019-03-08 10:41:00,291 ERROR: ERROR error running the validation groups ['post-upgrade']   {"stderr": "[DEPRECATION WARNING]: DEFAULT_SUDO_FLAGS option, In favor of become which is a\n generic framework . This feature will be removed in version 2.8. Deprecation \nwarnings can be disabled by setting deprecation_warnings=False in ansible.cfg.\n", "stdout": "Task 'Check stack resource statuses' failed:\nHost: undercloud\nMessage: An unhandled exception occurred while running the lookup plugin 'stack_resources'. Error was a <class 'heatclient.exc.HTTPNotFound'>, original message: ERROR: The Stack (overcloud) could not be found.\n\nFailure! The validation failed for all hosts:\n* undercloud\n"} {"stderr": "[DEPRECATION WARNING]: DEFAULT_SUDO_FLAGS option, In favor of become which is a\n generic framework . This feature will be removed in version 2.8. Deprecation \nwarnings can be disabled by setting deprecation_warnings=False in ansible.cfg.\n", "stdout": "Task 'Fail if the HorizonPublic endpoint is not defined' failed:\nHost: undercloud\nMessage: The `HorizonPublic` endpoint is not defined in the `EndpointMap` of the deployed stack. This means Horizon may not have been deployed correctly.\n\nFailure! The validation failed for all hosts:\n* undercloud\n"} Mistral execution ID: d29034fd-131a-4fbb-b10e-359394e1de9a
2019-03-08 10:41:00,292 INFO: 
#############################################################################
Undercloud upgrade complete.

The file containing this installation's passwords is at
/home/stack/undercloud-passwords.conf.

There is also a stackrc file at /home/stack/stackrc.

These files are needed to interact with the OpenStack services, and should be
secured.

#############################################################################

(undercloud) [stack@director13 ~]$ rpm -qa |grep validation
openstack-tripleo-validations-8.4.4-1.el7ost.noarch


Note You need to log in before you can comment on or make changes to this bug.