Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1636412

Summary: Mistral replies 504
Product: Red Hat OpenStack Reporter: Joe Talerico <jtaleric>
Component: openstack-mistralAssignee: RHOS Maint <rhos-maint>
Status: CLOSED DUPLICATE QA Contact: nlevinki <nlevinki>
Severity: high Docs Contact:
Priority: unspecified    
Version: 14.0 (Rocky)CC: jjoyce, jschluet, slinaber, therve, tvignaud
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-10-05 11:25:56 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Joe Talerico 2018-10-05 10:58:43 UTC
Description of problem:
The Undercloud has been deploying for ~24hrs (not seeing this issue)-- different attempts to get an overcloud. However, this is what we get now, when trying to deploy :

Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/osc_lib/shell.py", line 135, in run
    ret_val = super(OpenStackShell, self).run(argv)
  File "/usr/lib/python2.7/site-packages/cliff/app.py", line 281, in run
    result = self.run_subcommand(remainder)
  File "/usr/lib/python2.7/site-packages/osc_lib/shell.py", line 175, in run_subcommand
    ret_value = super(OpenStackShell, self).run_subcommand(argv)
  File "/usr/lib/python2.7/site-packages/cliff/app.py", line 402, in run_subcommand
    result = cmd.run(parsed_args)
  File "/usr/lib/python2.7/site-packages/tripleoclient/command.py", line 25, in run
    super(Command, self).run(parsed_args)
  File "/usr/lib/python2.7/site-packages/osc_lib/command/command.py", line 41, in run
    return super(Command, self).run(parsed_args)
  File "/usr/lib/python2.7/site-packages/cliff/command.py", line 184, in run
    return_code = self.take_action(parsed_args) or 0
  File "/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_deploy.py", line 886, in take_action
    self._deploy_tripleo_heat_templates_tmpdir(stack, parsed_args)
  File "/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_deploy.py", line 359, in _deploy_tripleo_heat_templates_tmpdir
    new_tht_root, tht_root)
  File "/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_deploy.py", line 371, in _deploy_tripleo_heat_templates
    plans = plan_management.list_deployment_plans(self.clients)
  File "/usr/lib/python2.7/site-packages/tripleoclient/workflows/plan_management.py", line 130, in list_deployment_plans
    workflow_input=workflow_input
  File "/usr/lib/python2.7/site-packages/tripleoclient/workflows/base.py", line 39, in start_workflow
    workflow_input=workflow_input
  File "/usr/lib/python2.7/site-packages/mistralclient/api/v2/executions.py", line 65, in create
    return self._create('/executions', data)
  File "/usr/lib/python2.7/site-packages/mistralclient/api/base.py", line 97, in _create
    self._raise_api_exception(ex.response)
  File "/usr/lib/python2.7/site-packages/mistralclient/api/base.py", line 160, in _raise_api_exception
    error_message=error_data)
APIException: <html><body><h1>504 Gateway Time-out</h1>
The server didn't respond in time.
</body></html>


Debug output from the failure. 

REQ: curl -g -i --cacert "/etc/pki/ca-trust/source/anchors/cm-local-ca.pem" -X POST https://192.168.24.2:13989/v2/executions -H "User-Agent: openstacksdk/0.17.2 keystoneauth1/3.10.0 python-requests/2.19.1 CPytho
n/2.7.5" -H "X-Auth-Token: {SHA1}e74a52266f7f6cf987b0bacc12ccc0a9af47d330" -H "content-type: application/json" -d '{"workflow_name": "tripleo.plan_management.v1.list_plans", "description": ""}'
Starting new HTTPS connection (1): 192.168.24.2
^[https://192.168.24.2:13989 "POST /v2/executions HTTP/1.1" 504 None
RESP: [504] Cache-Control: no-cache Connection: close Content-Type: text/html
RESP BODY: Omitted, Content-Type is set to text/html. Only application/json responses have their bodies logged.
Request returned failure status: 504
<html><body><h1>504 Gateway Time-out</h1>
The server didn't respond in time.
</body></html>


(undercloud) [stack@microbrow-19 ~]$ docker ps | grep unhea
5ca4f3f178ed        192.168.24.1:8787/rhosp14/openstack-heat-api-cfn:2018-09-26.1                "kolla_start"            35 hours ago        Up 20 hours (unhealthy)                       heat_api_cfn
be45b941f78e        192.168.24.1:8787/rhosp14/openstack-heat-api:2018-09-26.1                    "kolla_start"            35 hours ago        Up 20 hours (unhealthy)                       heat_api
(undercloud) [stack@microbrow-19 ~]$ 


Version-Release number of selected component (if applicable):
instack-undercloud-9.3.1-0.20180918171407.b0205ab.el7ost.noarch


How reproducible:
N/A

Steps to Reproduce:
Long living undercloud

Comment 1 Thomas Hervé 2018-10-05 11:25:56 UTC

*** This bug has been marked as a duplicate of bug 1628319 ***