Description of problem: Our deployment failed with error "503 Service Unavailable". Apparently the heat service is down: (undercloud) [stack@undercloud-0 ~]$ sudo docker ps |grep heat 2db1dff00df6 192.168.24.1:8787/rhosp14/openstack-heat-api-cfn:2018-10-08.4 "kolla_start" 5 hours ago Up 5 hours (unhealthy) heat_api_cfn 35b191b639f2 192.168.24.1:8787/rhosp14/openstack-heat-api:2018-10-08.4 "kolla_start" 5 hours ago Up 5 hours heat_api_cron c804c8eb85aa 192.168.24.1:8787/rhosp14/openstack-heat-engine:2018-10-08.4 "kolla_start" 5 hours ago Up 5 hours (healthy) heat_engine 33157896efc6 192.168.24.1:8787/rhosp14/openstack-heat-api:2018-10-08.4 "kolla_start" 5 hours ago Up 5 hours (unhealthy) heat_api (undercloud) [stack@undercloud-0 ~]$ openstack stack list +--------------------------------------+------------+----------------------------------+---------------+----------------------+--------------+ | ID | Stack Name | Project | Stack Status | Creation Time | Updated Time | +--------------------------------------+------------+----------------------------------+---------------+----------------------+--------------+ | f1517388-13c5-48c5-b2cc-5fc7a2468502 | overcloud | 57c16282a45444fe9c4fe49c8ae4206e | CREATE_FAILED | 2018-10-10T09:09:58Z | None | +--------------------------------------+------------+----------------------------------+---------------+----------------------+--------------+ (undercloud) [stack@undercloud-0 ~]$ openstack overcloud status +-----------+---------------------+---------------------+-------------------+ | Plan Name | Created | Updated | Deployment Status | +-----------+---------------------+---------------------+-------------------+ | overcloud | 2018-10-10 09:05:58 | 2018-10-10 09:05:58 | DEPLOY_SUCCESS | +-----------+---------------------+---------------------+-------------------+ And this issue confuses the GUI. The GUI is showed: Deployment success. Version-Release number of selected component (if applicable): openstack-tripleo-common-container-base-9.3.1-0.20180923215327.d22cb3e.el7ost.noarch How reproducible: 100% Steps to Reproduce: 1. Install OSP14 and deploy overcloud 2. Stop the heat service shortly after the deployment started 3. Actual results: Expected results: Additional info: SOSReport will be attached
You can also see the opposite situation, where the overcloud status (and the GUI) show FAILED and the stack status shows CREATE_COMPLETE: (undercloud) [stack@undercloud-0 ~]$ openstack overcloud status --plan plan +-----------+---------------------+---------------------+-------------------+ | Plan Name | Created | Updated | Deployment Status | +-----------+---------------------+---------------------+-------------------+ | plan | 2018-11-02 10:50:30 | 2018-11-02 10:50:30 | DEPLOY_FAILED | +-----------+---------------------+---------------------+-------------------+ (undercloud) [stack@undercloud-0 ~]$ openstack stack list +-----------+------------+--------------+-----------------+--------------+ | ID | Stack Name | Project | Stack Status | Updated Time | +-----------+------------+--------------+-----------------+--------------+ | 3fd4ed.,. | plan | cf304e09d... | CREATE_COMPLETE | None | +-----------+------------+--------------+-----------------+--------------+
Workaround (see also bug 1644861): 1) openstack object delete <<plan-name>>-messages deployment_status.yaml 2) F5 in the GUI 3) Click on "recover deployment status"
*** Bug 1637947 has been marked as a duplicate of this bug. ***
Verified on puddle 2018-11-13.1 [stack@undercloud-0 ~]$ rpm -q openstack-tripleo-common openstack-tripleo-common-9.4.1-0.20181012010875.67bab16.el7ost.noarch
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2019:0045