Bug 1547539 - [UPDATES] Failed to setup heat-output with custom stack name
Summary: [UPDATES] Failed to setup heat-output with custom stack name
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-common
Version: 12.0 (Pike)
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: z3
: 12.0 (Pike)
Assignee: Ryan Brady
QA Contact: Raviv Bar-Tal
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-02-21 14:06 UTC by Yurii Prokulevych
Modified: 2018-08-20 12:59 UTC (History)
16 users (show)

Fixed In Version: openstack-tripleo-common-7.6.13-1.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-08-20 12:59:02 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2018:2331 None None None 2018-08-20 12:59:53 UTC
OpenStack gerrit 527699 None master: MERGED tripleo-common: Get the config for update outside of package update (I8a92e4f4cfe8e3567e71f9ab60b4aef4142c3874) 2018-08-07 14:51:53 UTC

Description Yurii Prokulevych 2018-02-21 14:06:12 UTC
Description of problem:
-----------------------

Attempt to setup update playbook with custom's stack name fails:

openstack overcloud update stack --init-minor-update \
    --stack qe-cloud \
    --container-registry-file /home/stack/virt/docker-images.yaml 2>&1
...
Waiting for messages on queue '55d27f18-3a8e-470d-8a1c-377000c0fc9b' with no timeout.
{u'execution': {u'created_at': u'2018-02-21 11:52:03',
                u'id': u'bf3cef92-4d6d-4175-90ce-f52354626934',
                u'input': {u'ceph_ansible_playbook': u'/usr/share/ceph-ansible/site-docker.yml.sample',
                           u'config_dir': u'/tmp/',
                           u'container': u'qe-cloud',
                           u'container_registry': {u'parameter_defaults': {u'DockerAodhApiImage': u...
                           u'queue_name': u'55d27f18-3a8e-470d-8a1c-377000c0fc9b',
                           u'skip_deploy_identifier': False,
                           u'timeout': 240},
                u'name': u'tripleo.package_update.v1.package_update_plan',
                u'params': {u'namespace': u''},
                u'spec': {u'description': u'Take a container and perform a package update with possible breakpoints',
                          u'input': [u'container',
                                     u'container_registry',
                                     u'ceph_ansible_playbook',
                                     {u'timeout': 240},
                                     {u'queue_name': u'tripleo'},
                                     {u'skip_deploy_identifier': False},
                                     {u'config_dir': u'/tmp/'}],
                          u'name': u'package_update_plan',
                          u'tags': [u'tripleo-common-managed'],
                          u'tasks': {u'clean_plan': {u'action': u'tripleo.plan.update_plan_environment',
                                                     u'input': {u'container': u'<% $.container %>',
                                                                u'delete': True,
                                                                u'env_key': u'parameter_defaults',
                                                                u'parameter': u'CephAnsiblePlaybook'},
                                                     u'name': u'clean_plan',
                                                     u'on-error': u'set_update_failed',
                                                     u'on-success': u'get_config',
                                                     u'type': u'direct',
                                                     u'version': u'2.0'},
                                     u'get_config': {u'action': u'tripleo.config.get_overcloud_config',
                                                     u'name': u'get_config',
                                                     u'on-error': u'set_update_failed',
                                                     u'on-success': u'send_message',
                                                     u'type': u'direct',
                                                     u'version': u'2.0'},
                                     u'send_message': {u'action': u'zaqar.queue_post',
                                                       u'input': {u'messages': {u'body': {u'payload': {u'execution': u'<% execution() %>',
                                                                                                       u'message': u"<% $.get('message', '') %>",
                                                                                                       u'status': u"<% $.get('status', 'SUCCESS') %>"},
                                                                                          u'type': u'tripleo.package_update.v1.package_update_plan'}},
                                                                  u'queue_name': u'<% $.queue_name %>'},
                                                       u'name': u'send_message',
                                                       u'on-success': [{u'fail': u'<% $.get(\'get_config\') = "FAILED" %>'}],
                                                       u'type': u'direct',
                                                       u'version': u'2.0'},
                                     u'set_update_failed': {u'name': u'set_update_failed',
                                                            u'on-success': u'send_message',
                                                            u'publish': {u'message': u'<% task(update).result %>',
                                                                         u'status': u'FAILED'},
                                                            u'type': u'direct',
                                                            u'version': u'2.0'},
                                     u'update': {u'action': u'tripleo.package_update.update_stack container=<% $.container %> timeout=<% $.timeout %> container_registry=<% $.container_registry %> ceph_ansible_pl
aybook=<% $.ceph_ansible_playbook %>',
                                                 u'input': {u'ceph_ansible_playbook': u'<% $.ceph_ansible_playbook %>',
                                                            u'container': u'<% $.container %>',
                                                            u'container_registry': u'<% $.container_registry %>',
                                                            u'timeout': u'<% $.timeout %>'},
                                                 u'name': u'update',
                                                 u'on-error': u'set_update_failed',
                                                 u'on-success': u'clean_plan',
                                                 u'type': u'direct',
                                                 u'version': u'2.0'},
                                     u'update_plan': {u'action': u'tripleo.plan.update_from_dir',
                                                      u'input': {u'container': u'<% $.container %>'},
                                                      u'name': u'update_plan',
                                                      u'on-error': u'set_update_failed',
                                                      u'on-success': u'update',
                                                      u'type': u'direct',
                                                      u'version': u'2.0'}},
                          u'version': u'2.0'}},
 u'message': None,
 u'status': u'FAILED'}
Started Mistral Workflow tripleo.package_update.v1.package_update_plan. Execution ID: bf3cef92-4d6d-4175-90ce-f52354626934

Problem is that stack is updated, just client terminates.
openstack stack list
+--------------------------------------+------------+----------------------------------+-----------------+----------------------+----------------------+
| ID                                   | Stack Name | Project                          | Stack Status    | Creation Time        | Updated Time         |
+--------------------------------------+------------+----------------------------------+-----------------+----------------------+----------------------+
| df7ce7ad-2d6b-4e80-b4d9-a73a48e65050 | qe-cloud   | 65046383148545c283dc87a5172d29e0 | UPDATE_COMPLETE | 2018-02-21T10:19:08Z | 2018-02-21T11:53:31Z |
+--------------------------------------+------------+----------------------------------+-----------------+----------------------+----------------------+

Version-Release number of selected component (if applicable):
-------------------------------------------------------------
openstack-tripleo-common-7.6.3-10.el7ost.noarch
openstack-mistral-engine-5.2.0-1.el7ost.noarch
python-mistralclient-3.1.3-2.el7ost.noarch
python-mistral-5.2.0-1.el7ost.noarch
puppet-mistral-11.3.0-2.el7ost.noarch
openstack-mistral-api-5.2.0-1.el7ost.noarch
python-mistral-lib-0.3.1-1.el7ost.noarch
openstack-mistral-executor-5.2.0-1.el7ost.noarch
openstack-mistral-common-5.2.0-1.el7ost.noarch

Steps to Reproduce:
-------------------
1. Deploy oc with custom name
2. Update uc
3. Try prepare playbooks for minor update

Comment 1 Yurii Prokulevych 2018-02-21 14:48:26 UTC
Some info from mistral

 mistral task-list -f json bf3cef92-4d6d-4175-90ce-f52354626934
[                                                                                                                                                                                                         [10/1924]
  {
    "Workflow name": "tripleo.package_update.v1.package_update_plan", 
    "Name": "update_plan", 
    "Execution ID": "bf3cef92-4d6d-4175-90ce-f52354626934", 
    "Created at": "2018-02-21 11:52:03", 
    "State": "SUCCESS", 
    "State info": null, 
    "Updated at": "2018-02-21 11:52:46", 
    "ID": "14d4c077-a29e-40ae-b5e8-513a7d9c731d"
  }, 
  {
    "Workflow name": "tripleo.package_update.v1.package_update_plan", 
    "Name": "update", 
    "Execution ID": "bf3cef92-4d6d-4175-90ce-f52354626934", 
    "Created at": "2018-02-21 11:52:46", 
    "State": "SUCCESS", 
    "State info": null, 
    "Updated at": "2018-02-21 11:53:31", 
    "ID": "59bf2288-f1ba-4fa7-abfe-cc25fa9e599a"
  }, 
  {
    "Workflow name": "tripleo.package_update.v1.package_update_plan", 
    "Name": "clean_plan", 
    "Execution ID": "bf3cef92-4d6d-4175-90ce-f52354626934", 
    "Created at": "2018-02-21 11:53:31", 
    "State": "SUCCESS", 
    "State info": null, 
    "Updated at": "2018-02-21 11:53:32", 
    "ID": "9a36a691-94dc-462b-8c76-987569e9e910"
  }, 
  {
    "Workflow name": "tripleo.package_update.v1.package_update_plan", 
    "Name": "set_update_failed", 
    "Execution ID": "bf3cef92-4d6d-4175-90ce-f52354626934", 
    "Created at": "2018-02-21 11:53:32", 
    "State": "SUCCESS", 
    "State info": null, 
    "Updated at": "2018-02-21 11:53:32", 
    "ID": "1f5ba246-71fa-48b0-ae37-2ec6296a39d6"
  }, 
  {
    "Workflow name": "tripleo.package_update.v1.package_update_plan", 
    "Name": "send_message", 
    "Execution ID": "bf3cef92-4d6d-4175-90ce-f52354626934", 
    "Created at": "2018-02-21 11:53:32", 
    "State": "SUCCESS", 
    "State info": null, 
    "Updated at": "2018-02-21 11:53:33", 
    "ID": "5a758922-6944-49bd-a317-b3dc05887ee0"
  }, 
  {
    "Workflow name": "tripleo.package_update.v1.package_update_plan", 
    "Name": "get_config", 
    "Execution ID": "bf3cef92-4d6d-4175-90ce-f52354626934", 
    "Created at": "2018-02-21 11:53:32", 
    "State": "ERROR", 
    "State info": "Failed to run action [act...", 
    "Updated at": "2018-02-21 11:53:32", 
    "ID": "c650d728-a8a2-4de3-beb5-cabb3896dce8"
  }
]

mistral task-get -fjson c650d728-a8a2-4de3-beb5-cabb3896dce
{
  "Workflow name": "tripleo.package_update.v1.package_update_plan", 
  "Name": "get_config", 
  "Execution ID": "bf3cef92-4d6d-4175-90ce-f52354626934", 
  "Created at": "2018-02-21 11:53:32", 
  "State": "ERROR", 
  "State info": "Failed to run action [action_ex_id=f79e94e2-3604-4eea-856a-f144b57987fc, action_cls='<class 'mistral.actions.action_factory.GetOvercloudConfig'>', attributes='{}', params='{}']\n ERROR: The Stack (overcloud) could not be found.", 
  "Updated at": "2018-02-21 11:53:32", 
  "ID": "c650d728-a8a2-4de3-beb5-cabb3896dce8"
}

And from mistral/executor.log
-----------------------------018-02-21 06:53:32.372 1336 ERROR mistral.executors.default_executor [req-8de3c9bf-15d2-49ad-abf6-e95268a284c8 2aa4908492354a0bb3e50092b48f1e8f 65046383148545c283dc87a5172d29e0 - default default] Failed to run action [action_ex_id=f79e94e2-3604-4eea-856a-f144b57987fc, action_cls='<class 'mistral.actions.action_factory.GetOvercloudConfig'>', attributes='{}', params='{}']
 ERROR: The Stack (overcloud) could not be found.: HTTPNotFound: ERROR: The Stack (overcloud) could not be found.
2018-02-21 06:53:32.372 1336 ERROR mistral.executors.default_executor Traceback (most recent call last):
2018-02-21 06:53:32.372 1336 ERROR mistral.executors.default_executor   File "/usr/lib/python2.7/site-packages/mistral/executors/default_executor.py", line 109, in run_action
2018-02-21 06:53:32.372 1336 ERROR mistral.executors.default_executor     result = action.run(context.ctx())
2018-02-21 06:53:32.372 1336 ERROR mistral.executors.default_executor   File "/usr/lib/python2.7/site-packages/tripleo_common/actions/config.py", line 50, in run
2018-02-21 06:53:32.372 1336 ERROR mistral.executors.default_executor     config_path = config.download_config(self.container, self.config_dir)
2018-02-21 06:53:32.372 1336 ERROR mistral.executors.default_executor   File "/usr/lib/python2.7/site-packages/tripleo_common/utils/config.py", line 97, in download_config
2018-02-21 06:53:32.372 1336 ERROR mistral.executors.default_executor     stack = self.client.stacks.get(name)
2018-02-21 06:53:32.372 1336 ERROR mistral.executors.default_executor   File "/usr/lib/python2.7/site-packages/heatclient/v1/stacks.py", line 280, in get
2018-02-21 06:53:32.372 1336 ERROR mistral.executors.default_executor     resp = self.client.get('/stacks/%s' % stack_id, **kwargs)
2018-02-21 06:53:32.372 1336 ERROR mistral.executors.default_executor   File "/usr/lib/python2.7/site-packages/heatclient/common/http.py", line 286, in get
2018-02-21 06:53:32.372 1336 ERROR mistral.executors.default_executor     return self.client_request("GET", url, **kwargs)
2018-02-21 06:53:32.372 1336 ERROR mistral.executors.default_executor   File "/usr/lib/python2.7/site-packages/heatclient/common/http.py", line 279, in client_request
2018-02-21 06:53:32.372 1336 ERROR mistral.executors.default_executor     resp, body = self.json_request(method, url, **kwargs)
2018-02-21 06:53:32.372 1336 ERROR mistral.executors.default_executor   File "/usr/lib/python2.7/site-packages/heatclient/common/http.py", line 268, in json_request
2018-02-21 06:53:32.372 1336 ERROR mistral.executors.default_executor     resp = self._http_request(url, method, **kwargs)
2018-02-21 06:53:32.372 1336 ERROR mistral.executors.default_executor   File "/usr/lib/python2.7/site-packages/heatclient/common/http.py", line 231, in _http_request
2018-02-21 06:53:32.372 1336 ERROR mistral.executors.default_executor     raise exc.from_response(resp)
2018-02-21 06:53:32.372 1336 ERROR mistral.executors.default_executor HTTPNotFound: ERROR: The Stack (overcloud) could not be found.
2018-02-21 06:53:32.372 1336 ERROR mistral.executors.default_executor
2018-02-21 06:53:32.538 1336 INFO mistral.executors.executor_server [req-8de3c9bf-15d2-49ad-abf6-e95268a284c8 2aa4908492354a0bb3e50092b48f1e8f 65046383148545c283dc87a5172d29e0 - default default] Received RPC request 'run_action'[action_ex_id=2c5787ce-53eb-4637-b882-53d693f4b403, action_cls_str=mistral.actions.std_actions.NoOpAction, action_cls_attrs={}, params={}]

Comment 7 Ryan Brady 2018-07-20 14:57:35 UTC
switching the needs_info to the DFG responsible for the work on this BZ.

Comment 9 errata-xmlrpc 2018-08-20 12:59:02 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:2331


Note You need to log in before you can comment on or make changes to this bug.