Description of problem: nova_metadata container is in unhealthy state on undercloud and overcloud nodes: [root@undercloud-0 stack]# docker ps | grep nova_metadata 2ee724246dec 192.168.24.1:8787/rhosp14/openstack-nova-api:2018-10-08.4 "kolla_start" 4 hours ago Up 4 hours (unhealthy) nova_metadata [root@controller-0 heat-admin]# docker ps | grep nova_metadata 0e3143263535 192.168.24.1:8787/rhosp14/openstack-nova-api:2018-10-08.4 "kolla_start" 2 hours ago Up 42 minutes (unhealthy) nova_metadata Version-Release number of selected component (if applicable): 14 -p 2018-10-08.4 How reproducible: 100% Steps to Reproduce: 1. Deploy OSP14 undercloud and overcloud 2. Check nova_metadata container status Actual results: It reports unhealthy state Expected results: It reports healthy state. Additional info: Attaching job artifacts
Wrong healthcheck script is used in the nova_metadata container. (undercloud) [stack@undercloud-0 ~]$ docker exec -it -u root nova_metadata /bin/bash ()[root@undercloud-0 /]# cd openstack/ ()[root@undercloud-0 openstack]# ls -la total 0 drwxr-xr-x. 2 root root 25 Oct 10 02:10 . drwxr-xr-x. 1 root root 81 Oct 11 08:53 .. lrwxrwxrwx. 1 root root 56 Oct 10 02:10 healthcheck -> /usr/share/openstack-tripleo-common/healthcheck/nova-api
While we have a dedicated nova_metadata healthcheck script, the nova_metadata and nova_api container the same image and the current nova api healtcheck script still checks the non wsgi implementation. Currently we have: ~~~ # cat healthcheck #!/bin/sh . ${HEALTHCHECK_SCRIPTS:-/usr/share/openstack-tripleo-common/healthcheck}/common.sh if ps -ef | grep --quiet nova-metadata; then bind_host=$(get_config_val /etc/nova/nova.conf DEFAULT metadata_listen 127.0.0.1) bind_port=$(get_config_val /etc/nova/nova.conf DEFAULT metadata_listen_port 8775) check_url="http://${bind_host}:${bind_port}/" else check_url=$(get_url_from_vhost /etc/httpd/conf.d/10-nova_api_wsgi.conf) fi healthcheck_curl ${check_url} ~~~ Proposed change in https://review.openstack.org/609927 . This changes the nova_api healthcheck script to check the metadata wsgi vhost config for details instead of the details in nova.conf.
Verified the status on both the undercloud and the controller node: [stack@undercloud-0 ~]$ docker ps | grep nova_metadata 519f3f1040fc 192.168.24.1:8787/rhosp14/openstack-nova-api:2018-11-09.3 "kolla_start" 7 days ago Up 7 days (healthy) nova_metadata [heat-admin@controller-0 ~]$ sudo docker ps | grep nova_metadata bcd148c24426 192.168.24.1:8787/rhosp14/openstack-nova-api:2018-11-09.3 "kolla_start" 7 days ago Up 7 days (healthy) nova_metadata
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2019:0045