Bug 1569751 - OSP10->11->12->13 undercloud upgrade post validation fail: Task 'Check Keystone' failed:\nHost: localhost\nMessage: Status code was not [201]: HTTP Error 404: Not Found\n\nFailure! The validation failed for all hosts:\n*
Summary: OSP10->11->12->13 undercloud upgrade post validation fail: Task 'Check Keysto...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-validations
Version: 13.0 (Queens)
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: rc
: 13.0 (Queens)
Assignee: Carlos Camacho
QA Contact: Marius Cornea
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-04-19 23:54 UTC by Marius Cornea
Modified: 2018-06-27 13:53 UTC (History)
15 users (show)

Fixed In Version: openstack-tripleo-validations-8.4.1-5.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-06-27 13:52:02 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1766238 0 None None None 2018-04-23 12:26:16 UTC
OpenStack Storyboard 2001897 0 None None None 2018-04-25 10:26:58 UTC
OpenStack gerrit 567122 0 None MERGED Use clean keystone URL in the keystone validation 2020-05-06 13:08:19 UTC
Red Hat Product Errata RHEA-2018:2086 0 None None None 2018-06-27 13:53:11 UTC

Description Marius Cornea 2018-04-19 23:54:13 UTC
Description of problem:
OSP10->11->12->13 undercloud upgrade post validation fail: 

2018-04-19 19:48:37,946 INFO: Starting and waiting for validation groups ['post-upgrade'] 
2018-04-19 19:49:35,343 ERROR: ERROR error running the validation groups ['post-upgrade']   {"stderr": "[DEPRECATION WARNING]: DEFAULT_SUDO_FLAGS option, In favor of become which is a\n generic framework . This feature will be removed in version 2.8. Deprecation \nwarnings can be disabled by setting deprecation_warnings=False in ansible.cfg.\n", "stdout": "Task 'Check Keystone' failed:\nHost: localhost\nMessage: Status code was not [201]: HTTP Error 404: Not Found\n\nFailure! The validation failed for all hosts:\n* localhost\n"} Mistral execution ID: ef78e9c4-3f5e-4776-87eb-91992d5b02fd

The error is only related to the post upgrade validation and it does not block the upgrade.

Version-Release number of selected component (if applicable):
instack-undercloud-8.4.0-4.el7ost.noarch
openstack-tripleo-validations-8.4.0-2.el7ost.noarch

How reproducible:
100%

Steps to Reproduce:
1. Deploy OSP10
2. Upgrade undercloud to OSP11
3. Upgrade undercloud to OSP12
4. Upgrade undercloud to OSP13

Actual results:
post-upgrade undercloud validation fail:
2018-04-19 19:49:35,343 ERROR: ERROR error running the validation groups ['post-upgrade']   {"stderr": "[DEPRECATION WARNING]: DEFAULT_SUDO_FLAGS option, In favor of become which is a\n generic framework . This feature will be removed in version 2.8. Deprecation \nwarnings can be disabled by setting deprecation_warnings=False in ansible.cfg.\n", "stdout": "Task 'Check Keystone' failed:\nHost: localhost\nMessage: Status code was not [201]: HTTP Error 404: Not Found\n\nFailure! The validation failed for all hosts:\n* localhost\n"} Mistral execution ID: ef78e9c4-3f5e-4776-87eb-91992d5b02fd

Expected results:
post-upgrade validations succeed

Additional info:

Comment 2 Yurii Prokulevych 2018-04-23 09:50:26 UTC
Got same issue during regular 12 -> 13 major upgrade.
Package: openstack-tripleo-validations-8.4.0-2.el7ost.noarch

Comment 3 Yurii Prokulevych 2018-04-23 10:05:18 UTC
So after running validation the problem seems to be a malformed url used for 'Check Keystone' task:
    https://10.0.0.101:13000/v2.0/v3/auth/tokens

The url is defined as 'url: "{{ overcloud_keystone_url }}/v3/auth/tokens"'
and 'overcloud_keystone_url' that is generated with:

 tripleo-ansible-inventory --static-yaml-inventory inventory

has a trailing 'v2.0' suffix

Comment 4 Jose Luis Franco 2018-04-23 12:06:19 UTC
overcloud_keystone_url parameter gets its value from the KeystoneURL stack parameter:

https://github.com/openstack/tripleo-common/blob/master/tripleo_common/inventory.py#L177-L179

This stack output is at the same time defined in tripleo-heat-templates from the value of KeystonePublic.uri

https://github.com/openstack/tripleo-heat-templates/blob/stable/pike/overcloud.j2.yaml#L933

Which in stable/pike (which is the version we come from before upgrading the undercloud) is defined appending /v2.0:

https://github.com/openstack/tripleo-heat-templates/blob/stable/pike/network/endpoints/endpoint_map.yaml#L4085

For that reason, Yurii has defined in its inventory the value: https://10.0.0.101:13000/v2.0/ for 'overcloud_keystone_url'.

I'm preparing a patch for it so we leave the version aside when creating overcloud_keystone_url.

Comment 7 Carlos Camacho 2018-05-09 16:43:43 UTC
We need the blocker flag in this please. The fix is merged downstream but we cant create the package

Comment 8 Maria Bracho 2018-05-09 17:23:04 UTC
acked.

Comment 15 errata-xmlrpc 2018-06-27 13:52:02 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:2086


Note You need to log in before you can comment on or make changes to this bug.