Bug 1601673 - overcloud deploy does not correctly report YAML syntax errors to the operator
Summary: overcloud deploy does not correctly report YAML syntax errors to the operator
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-common
Version: 13.0 (Queens)
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: zstream
: 13.0 (Queens)
Assignee: Adriano Petrich
QA Contact: Marius Cornea
URL:
Whiteboard:
: 1603401 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-07-17 02:49 UTC by Lars Kellogg-Stedman
Modified: 2019-03-14 13:54 UTC (History)
10 users (show)

Fixed In Version: openstack-tripleo-common-8.6.6-14.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-03-14 13:54:50 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1804268 0 None None None 2018-11-20 16:52:48 UTC
OpenStack gerrit 632104 0 'None' MERGED Set heatclient exceptions to verbose 2021-02-05 09:39:54 UTC
Red Hat Product Errata RHBA-2019:0448 0 None None None 2019-03-14 13:54:58 UTC

Description Lars Kellogg-Stedman 2018-07-17 02:49:52 UTC
Description of problem:

I was deploying using some custom network configurations, and one of the YAML templates had a syntax error.  The deploy failed with:

  Action tripleo.parameters.update execution failed: Error validating environment for plan overcloud: ERROR: Internal Error

That doesn't really provide any information about the nature of the problem.  /var/log/heat-engine.log has:

  Traceback (most recent call last):
    File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 163, in _process_incoming
      res = self.dispatcher.dispatch(message)
    File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 220, in dispatch
      return self._do_dispatch(endpoint, method, ctxt, args)
    File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 190, in _do_dispatch
      result = func(ctxt, **new_args)
    File "/usr/lib/python2.7/site-packages/osprofiler/profiler.py", line 158, in wrapper
      result = f(*args, **kwargs)
    File "/usr/lib/python2.7/site-packages/heat/common/context.py", line 409, in wrapped
      return func(self, ctx, *args, **kwargs)
    File "/usr/lib/python2.7/site-packages/heat/engine/service.py", line 1241, in validate_template
      result.update(stack.get_nested_parameters(filter_parameter))
    File "/usr/lib/python2.7/site-packages/heat/engine/stack.py", line 1081, in get_nested_parameters
      params.update(nested.get_nested_parameters(filter_func))
    File "/usr/lib/python2.7/site-packages/heat/engine/stack.py", line 1081, in get_nested_parameters
      params.update(nested.get_nested_parameters(filter_func))
    File "/usr/lib/python2.7/site-packages/heat/engine/stack.py", line 1064, in get_nested_parameters
      nested = rsrc.get_nested_parameters_stack()
    File "/usr/lib/python2.7/site-packages/heat/engine/resources/stack_resource.py", line 210, in get_nested_parameters_stack
      child_template = self.child_template()
    File "/usr/lib/python2.7/site-packages/heat/engine/resources/template_resource.py", line 179, in child_template
      self.template_url)
    File "/usr/lib/python2.7/site-packages/heat/common/template_format.py", line 116, in parse
      tpl = simple_parse(tmpl_str, tmpl_url)
    File "/usr/lib/python2.7/site-packages/heat/common/template_format.py", line 79, in simple_parse
      raise ValueError(msg)
  ValueError: Error parsing template http://192.168.24.1:8080/v1/AUTH_5d88b80a41834ca7a6abe7c675884591/overcloud/user-files/home/stack/rhosp-director-config/templates/network/config/networker.yaml while parsing a block mapping
    in "<unicode string>", line 111, column 17:
                    - type: interface
                      ^
  expected <block end>, but found '-'
    in "<unicode string>", line 118, column 17:
                      - ip_netmask: {get_param: EC2Met ...
                      ^
 
Version-Release number of selected component (if applicable):

  openstack-heat-engine-10.0.1-0.20180411125640.el7ost.noarch
  python-tripleoclient-9.2.1-12.el7ost.noarch

Actual results:

Heat logged a traceback in a log file and provided an opaque error message to the operator.

Expected results:

The YAML syntax error should be reported to the operator by the 'overcloud deploy' command.

Comment 1 Dan Trainor 2018-07-18 15:18:21 UTC
DFG:UI believes this should be in DFG:DF, can you confirm?

Comment 2 Emilien Macchi 2018-07-18 15:32:55 UTC
Lars, have you tried to use this script to validate a template?

https://github.com/openstack/tripleo-heat-templates/blob/master/tools/yaml-validate.py

We have a few other tools in this directory, you might want to try them and tell us if it helped. Otherwise, we don't have (AFIK) other validations when deploying the overcloud.

Comment 4 Lars Kellogg-Stedman 2018-11-12 14:55:52 UTC
I wasn't requesting additional validations as much as I was requesting that the deployment framework should report these errors to the operator in the output of the 'overcloud deploy' command, rather than hiding them in the logs.  This would transform form the problem from a mysterious and frustrating failure to someone not already familiar with the internal mechanics of tripleo into something that is easy to diagnose and resolve.

Comment 6 Dale Bewley 2018-11-20 05:14:14 UTC
(In reply to Lars Kellogg-Stedman from comment #4)
> I wasn't requesting additional validations as much as I was requesting that
> the deployment framework should report these errors to the operator in the
> output of the 'overcloud deploy' command, rather than hiding them in the
> logs.  This would transform form the problem from a mysterious and
> frustrating failure to someone not already familiar with the internal
> mechanics of tripleo into something that is easy to diagnose and resolve.

I had this same problem, and this bz report helped me find the error in /var/log/heat/heat-engine.log and correct my YAML malformation.

Comment 7 Thomas Hervé 2018-11-27 08:48:44 UTC
*** Bug 1603401 has been marked as a duplicate of this bug. ***

Comment 26 Marius Cornea 2019-03-01 22:21:05 UTC
(undercloud) [stack@undercloud-0 ~]$ bash overcloud_deploy.sh 
Started Mistral Workflow tripleo.validations.v1.check_pre_deployment_validations. Execution ID: 3ae2cd59-4310-476f-bf9e-937652f4469f
Waiting for messages on queue 'tripleo' with no timeout.
Removing the current plan files
Uploading new plan files
Started Mistral Workflow tripleo.plan_management.v1.update_deployment_plan. Execution ID: 2ec6343e-2673-4e68-b221-aea1d2763980
Plan updated.
Processing templates in the directory /tmp/tripleoclient-Y_cZft/tripleo-heat-templates
Action tripleo.parameters.update execution failed: Error validating environment for plan overcloud: ERROR: Internal Error
Traceback (most recent call last):

  File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 163, in _process_incoming
    res = self.dispatcher.dispatch(message)

  File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 220, in dispatch
    return self._do_dispatch(endpoint, method, ctxt, args)

  File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 190, in _do_dispatch
    result = func(ctxt, **new_args)

  File "/usr/lib/python2.7/site-packages/osprofiler/profiler.py", line 158, in wrapper
    result = f(*args, **kwargs)

  File "/usr/lib/python2.7/site-packages/heat/common/context.py", line 409, in wrapped
    return func(self, ctx, *args, **kwargs)

  File "/usr/lib/python2.7/site-packages/heat/engine/service.py", line 1281, in validate_template
    result.update(stack.get_nested_parameters(filter_parameter))

  File "/usr/lib/python2.7/site-packages/heat/engine/stack.py", line 1083, in get_nested_parameters
    params.update(nested.get_nested_parameters(filter_func))

  File "/usr/lib/python2.7/site-packages/heat/engine/stack.py", line 1083, in get_nested_parameters
    params.update(nested.get_nested_parameters(filter_func))

  File "/usr/lib/python2.7/site-packages/heat/engine/stack.py", line 1066, in get_nested_parameters
    nested = rsrc.get_nested_parameters_stack()

  File "/usr/lib/python2.7/site-packages/heat/engine/resources/stack_resource.py", line 210, in get_nested_parameters_stack
    child_template = self.child_template()

  File "/usr/lib/python2.7/site-packages/heat/engine/resources/template_resource.py", line 179, in child_template
    self.template_url)

  File "/usr/lib/python2.7/site-packages/heat/common/template_format.py", line 116, in parse
    tpl = simple_parse(tmpl_str, tmpl_url)

  File "/usr/lib/python2.7/site-packages/heat/common/template_format.py", line 79, in simple_parse
    raise ValueError(msg)

ValueError: Error parsing template https://192.168.24.2:13808/v1/AUTH_505e127aca5d4df69624aa3ba7e3f044/overcloud/user-files/home/stack/virt/network/three-nics-vlans/controller.yaml while parsing a block mapping
  in "<unicode string>", line 85, column 20:
                       $network_config:
                       ^
expected <block end>, but found '<block sequence start>'
  in "<unicode string>", line 153, column 25:

Comment 28 errata-xmlrpc 2019-03-14 13:54:50 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:0448


Note You need to log in before you can comment on or make changes to this bug.