Bug 1630129 - openstack-nova-placement-api reports unhealthy
Summary: openstack-nova-placement-api reports unhealthy
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-common
Version: 13.0 (Queens)
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: z4
: 13.0 (Queens)
Assignee: Martin Schuppert
QA Contact: Archit Modi
URL:
Whiteboard:
: 1684728 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-09-18 03:39 UTC by Robin Cernin
Modified: 2020-05-18 00:01 UTC (History)
17 users (show)

Fixed In Version: openstack-tripleo-common-8.6.6-3.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-01-16 17:55:03 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Launchpad 1766375 None None None 2018-09-18 03:39:48 UTC
OpenStack gerrit 604272 None MERGED Fix nova-placement healthcheck 2020-07-01 01:46:12 UTC
OpenStack gerrit 616548 None MERGED Revert "Fix nova-placement healthcheck" 2020-07-01 01:46:12 UTC
Red Hat Knowledge Base (Solution) 3729661 None None None 2018-12-11 16:38:36 UTC
Red Hat Product Errata RHBA-2019:0068 None None None 2019-01-16 17:55:17 UTC

Description Robin Cernin 2018-09-18 03:39:49 UTC
Description of problem:

After OSP13 deployment, the nova-placement container reports unhealthy.

Version-Release number of selected component (if applicable):

192.168.24.1:8787/rhosp13/openstack-nova-placement-api:2018-09-13.1

How reproducible:

                "Status": "unhealthy",
                "FailingStreak": 1198,
                "Log": [
                    {
                        "Start": "2018-09-18T03:33:44.97705435Z",
                        "End": "2018-09-18T03:33:45.279829307Z",
                        "ExitCode": 1,
                        "Output": "\n403 172.17.1.10:8778 0.005 seconds\ncurl: (22) The requested URL returned error: 403 Forbidden\n"
                    },
                    {
                        "Start": "2018-09-18T03:34:15.28011505Z",
                        "End": "2018-09-18T03:34:15.599014323Z",
                        "ExitCode": 1,
                        "Output": "curl: (22) The requested URL returned error: 403 Forbidden\n\n403 172.17.1.10:8778 0.005 seconds\n"
                    },
                    {
                        "Start": "2018-09-18T03:34:45.599266364Z",
                        "End": "2018-09-18T03:34:45.916324672Z",
                        "ExitCode": 1,
                        "Output": "curl: (22) The requested URL returned error: 403 Forbidden\n\n403 172.17.1.10:8778 0.005 seconds\n"
                    },
                    {
                        "Start": "2018-09-18T03:35:15.916586391Z",
                        "End": "2018-09-18T03:35:16.235676709Z",
                        "ExitCode": 1,
                        "Output": "curl: (22) The requested URL returned error: 403 Forbidden\n\n403 172.17.1.10:8778 0.006 seconds\n"
                    },
                    {
                        "Start": "2018-09-18T03:35:46.236038235Z",
                        "End": "2018-09-18T03:35:46.539115641Z",
                        "ExitCode": 1,
                        "Output": "curl: (22) The requested URL returned error: 403 Forbidden\n\n403 172.17.1.10:8778 0.006 seconds\n"
                    }
                ]


Steps to Reproduce:
1. Deploy OSP13
2. Check container nova-placement
3.

Actual results:

Wrong placement url in healthcheck

https://bugs.launchpad.net/tripleo/+bug/1766375

Comment 9 Alexander Chuzhoy 2018-11-07 23:07:16 UTC
FailedQA:
Environment:
openstack-tripleo-common-8.6.6-2.el7ost.noarch
openstack-tripleo-common-containers-8.6.6-2.el7ost.noarch


[heat-admin@overcloud-controller-1 ~]$ sudo docker ps|grep nova_placement
74c6787d2b65        192.168.0.1:8787/rhosp13/openstack-nova-placement-api:13.0-66          "kolla_start"            21 hours ago        Up 21 hours (unhealthy)                       nova_placement

Comment 10 Martin Schuppert 2018-11-08 13:48:15 UTC
with 7f9606ff47f03162e5d7305aaca2af5d1faecd87 backported to healthcheck/common.sh the wsgi_alias
is now automatically received from the vhost config. As a result we check again a wrong url:

http://controller-0.internalapi.localdomain:8778/placement/placement/

$ git show 7f9606ff47f03162e5d7305aaca2af5d1faecd87
commit 7f9606ff47f03162e5d7305aaca2af5d1faecd87
Author: Jose Luis Franco Arza <jfrancoa@redhat.com>
Date:   Mon Jul 16 09:40:11 2018 +0200

    Take WSGIScriptAlias into account in docker healthcheck.
    
    Some services running as vhost, as nova_placement make
    use of an  alias in which the wsgi service is mapped.
    Currently, the nova_placement healthcheck has hardcoded
    the alias in the curl [0] request, but if this alias
    changes or other service modifies its from / to another,
    then the check will start failing again.
    
    [0] - https://github.com/openstack/tripleo-common/blob/master/healthcheck/nova-placement#L6
    
    Change-Id: I5a325be424740e80eafd6b4abd8eb4b3111740aa
    Closes-Bug: #1781623
    (cherry picked from commit f1a1c324fb149d40c960f1eaba650f0e8c71d91e)

diff --git a/healthcheck/common.sh b/healthcheck/common.sh
index 85eb9a18..ab55e4dc 100644
--- a/healthcheck/common.sh
+++ b/healthcheck/common.sh
@@ -50,9 +50,13 @@ get_url_from_vhost () {
   server_name=$(awk '/ServerName/ {print $2}' $vhost_file)
   ssl_enabled=$(awk '/SSLEngine/ {print $2}' $vhost_file)
   bind_port=$(grep -h "<VirtualHost .*>" $vhost_file | sed 's/<VirtualHost .*:\(.*\)>/\1/')
+  wsgi_alias=$(awk '/WSGIScriptAlias/ {print $2}' $vhost_file)
   proto=http
   if [[ $ssl_enabled == "on" ]]; then
     proto=https
   fi
-  echo ${proto}://${server_name}:${bind_port}/
+  if [[ $wsgi_alias != "/" ]]; then
+    wsgi_alias="${wsgi_alias}/"
+  fi
+  echo ${proto}://${server_name}:${bind_port}${wsgi_alias}
 }


The change in rocky f1a1c324fb149d40c960f1eaba650f0e8c71d91e also modifies the  healthcheck/nova-placement script, whereas the backport to queens does not:
diff --git a/healthcheck/nova-placement b/healthcheck/nova-placement
index f8e29267..1e54c93e 100755
--- a/healthcheck/nova-placement
+++ b/healthcheck/nova-placement
@@ -3,4 +3,4 @@
 . ${HEALTHCHECK_SCRIPTS:-/usr/share/openstack-tripleo-common/healthcheck}/common.sh
 
 check_url=$(get_url_from_vhost /etc/httpd/conf.d/10-placement_wsgi.conf)
-healthcheck_curl ${check_url}/placement/
+healthcheck_curl ${check_url}

We'd need to revert the patch again which we backported in this BZ.

Comment 39 errata-xmlrpc 2019-01-16 17:55:03 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:0068

Comment 40 Martin Schuppert 2019-03-14 16:41:34 UTC
*** Bug 1684728 has been marked as a duplicate of this bug. ***


Note You need to log in before you can comment on or make changes to this bug.