Bug 1298617

Summary: Remote error: UnicodeEncodeError 'ascii' codec can't encode character u'\u200b' in position 39: ordinal not in range(128)
Product: Red Hat OpenStack Reporter: Benjamin Schmaus <bschmaus>
Component: openstack-heatAssignee: Zane Bitter <zbitter>
Status: CLOSED CURRENTRELEASE QA Contact: Amit Ugol <augol>
Severity: high Docs Contact:
Priority: high    
Version: 7.0 (Kilo)CC: felipe.alfaro, mburns, mcornea, mlopes, rhel-osp-director-maint, sbaker, shardy, srevivo, zbitter
Target Milestone: gaKeywords: TestOnly, ZStream
Target Release: 8.0 (Liberty)   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: openstack-heat-5.0.1-1.el7ost Doc Type: Bug Fix
Doc Text:
Previously, non-ASCII characters were not supported in parameter input values. Consequently, showing a stack, or listing stacks, would fail with an error if any of the parameters values contained non-ASCII characters. With this fix, all unicode characters are now supported. As a result, stacks can be listed or shown even if their input parameters contain non-ASCII characters.
Story Points: ---
Clone Of:
: 1332040 (view as bug list) Environment:
Last Closed: 2016-04-28 13:51:41 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On:    
Bug Blocks: 1332040    
Attachments:
Description Flags
Example of group of YAML files that contain the unicode character none

Description Benjamin Schmaus 2016-01-14 14:59:59 UTC
Created attachment 1114839 [details]
Example of group of YAML files that contain the unicode character

Description of problem:  When a customer uses custom heat template files in OSP Director, sometimes stray unicode characters get introduced and the customer will get the following error when trying to do an overcoud deploy:

ERROR: openstack ERROR: Remote error: UnicodeEncodeError 'ascii' codec can't encode character u'\u200b' in position 39: ordinal not in range(128)
[u'
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/cliff/app.py", line 295, in run_subcommand
    result = cmd.run(parsed_args)
  File "/usr/lib/python2.7/site-packages/cliff/command.py", line 53, in run
    self.take_action(parsed_args)
  File "/usr/lib/python2.7/site-packages/rdomanager_oscplugin/v1/overcloud_deploy.py", line 1223, in take_action
    self._deploy_tripleo_heat_templates(stack, parsed_args)
  File "/usr/lib/python2.7/site-packages/rdomanager_oscplugin/v1/overcloud_deploy.py", line 527, in _deploy_tripleo_heat_templates
    environments, parsed_args.timeout)
  File "/usr/lib/python2.7/site-packages/rdomanager_oscplugin/v1/overcloud_deploy.py", line 446, in _heat_deploy
    orchestration_client, stack_name)
  File "/usr/lib/python2.7/site-packages/rdomanager_oscplugin/utils.py", line 145, in wait_for_stack_ready
    stack = orchestration_client.stacks.get(stack_name)
  File "/usr/lib/python2.7/site-packages/heatclient/v1/stacks.py", line 202, in get
    resp, body = self.client.json_request('GET', '/stacks/%s' % stack_id)
  File "/usr/lib/python2.7/site-packages/heatclient/common/http.py", line 265, in json_request
    resp = self._http_request(url, method, **kwargs)
  File "/usr/lib/python2.7/site-packages/heatclient/common/http.py", line 227, in _http_request
    resp = self._http_request(path, method, **kwargs)
  File "/usr/lib/python2.7/site-packages/heatclient/common/http.py", line 220, in _http_request
    raise exc.from_response(resp)
HTTPInternalServerError: ERROR: Remote error: UnicodeEncodeError 'ascii' codec can't encode character u'\u200b' in position 39: ordinal not in range(128)
[u

Would it be possible to fix this by scrubbing in the incoming yaml files before processing?

Attached are example yaml files that exhibit the issue.

Version-Release number of selected component (if applicable):7.1, 7.2


How reproducible:
100%

Steps to Reproduce:
1.Create YAML files with unicode character within them
2.Deploy overcloud
3.Error during overcloud deploy

Actual results:
Unicode error and deploy fails

Expected results:
Deployment should be successful

Additional info:

We can work around the issue if we run the following on the yaml files:

perl -CD -i.bak -pe 's/\x{200b}//g' *.yaml

Then delete the overcloud stack:

heat stack-delete overcloud

Then attempt a new deploy which will be successful.

Comment 3 Zane Bitter 2016-01-14 20:05:46 UTC
It makes sense to improve Heat's support for unicode, but it doesn't make sense to use a toolchain that inserts random garbage (in this case zero-width spaces) into your templates. Having Heat scrub the incoming yaml files doesn't make a lot of sense to me - there's no way we can maintain a blacklist of every non-printing unicode character.

My best guess is that this is occurring as a result of copying-and-pasting from a web page.

Incidentally, none of the attached files contain the offending character, or any other non-ASCII character ('cat -v *.yaml | grep M-' prints nothing).

Without a traceback from heat-engine itself, it's impossible to know where the problem is manifesting.

Comment 4 Benjamin Schmaus 2016-01-14 23:28:38 UTC
@Zane here is the traceback from the heat-engine.log:

2016-01-13 10:59:54.321 3852 INFO heat.common.urlfetch [-] Fetching data from file:///home/stack/templates/puppet/swift-devices-and-proxy-config.yaml
2016-01-13 10:59:54.322 3853 ERROR oslo_messaging.rpc.dispatcher [req-e5b19a3a-718c-4953-9e95-8b9ad2101270 30deef294e274d5aaa13dbcbc55b4dfa 9fbfdd7dfc5c48e88aadb47d59756257] Exception during message handling: 'ascii' codec can't encode character u'\u200b' in position 39: ordinal not in range(128)
2016-01-13 10:59:54.322 3853 TRACE oslo_messaging.rpc.dispatcher Traceback (most recent call last):
2016-01-13 10:59:54.322 3853 TRACE oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply
2016-01-13 10:59:54.322 3853 TRACE oslo_messaging.rpc.dispatcher     executor_callback))
2016-01-13 10:59:54.322 3853 TRACE oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch
2016-01-13 10:59:54.322 3853 TRACE oslo_messaging.rpc.dispatcher     executor_callback)
2016-01-13 10:59:54.322 3853 TRACE oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 130, in _do_dispatch
2016-01-13 10:59:54.322 3853 TRACE oslo_messaging.rpc.dispatcher     result = func(ctxt, **new_args)
2016-01-13 10:59:54.322 3853 TRACE oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/osprofiler/profiler.py", line 105, in wrapper
2016-01-13 10:59:54.322 3853 TRACE oslo_messaging.rpc.dispatcher     return f(*args, **kwargs)
2016-01-13 10:59:54.322 3853 TRACE oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/heat/common/context.py", line 300, in wrapped
2016-01-13 10:59:54.322 3853 TRACE oslo_messaging.rpc.dispatcher     return func(self, ctx, *args, **kwargs)
2016-01-13 10:59:54.322 3853 TRACE oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/heat/engine/service.py", line 502, in list_stacks
2016-01-13 10:59:54.322 3853 TRACE oslo_messaging.rpc.dispatcher     return [api.format_stack(stack) for stack in stacks]
2016-01-13 10:59:54.322 3853 TRACE oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/heat/engine/api.py", line 99, in format_stack
2016-01-13 10:59:54.322 3853 TRACE oslo_messaging.rpc.dispatcher     rpc_api.STACK_PARAMETERS: stack.parameters.map(str),
2016-01-13 10:59:54.322 3853 TRACE oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/heat/engine/parameters.py", line 526, in map
2016-01-13 10:59:54.322 3853 TRACE oslo_messaging.rpc.dispatcher     for n, p in six.iteritems(self.params) if filter_func(p))
2016-01-13 10:59:54.322 3853 TRACE oslo_messaging.rpc.dispatcher   File "/usr/lib/python2.7/site-packages/heat/engine/parameters.py", line 526, in <genexpr>
2016-01-13 10:59:54.322 3853 TRACE oslo_messaging.rpc.dispatcher     for n, p in six.iteritems(self.params) if filter_func(p))
2016-01-13 10:59:54.322 3853 TRACE oslo_messaging.rpc.dispatcher UnicodeEncodeError: 'ascii' codec can't encode character u'\u200b' in position 39: ordinal not in range(128)
2016-01-13 10:59:54.322 3853 TRACE oslo_messaging.rpc.dispatcher 
2016-01-13 10:59:54.322 3853 ERROR oslo_messaging._drivers.common [req-e5b19a3a-718c-4953-9e95-8b9ad2101270 30deef294e274d5aaa13dbcbc55b4dfa 9fbfdd7dfc5c48e88aadb47d59756257] Returning exception 'ascii' codec can't encode character u'\u200b' in position 39: ordinal not in range(128) to caller
2016-01-13 10:59:54.322 3853 ERROR oslo_messaging._drivers.common [req-e5b19a3a-718c-4953-9e95-8b9ad2101270 30deef294e274d5aaa13dbcbc55b4dfa 9fbfdd7dfc5c48e88aadb47d59756257] ['Traceback (most recent call last):\n', '  File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply\n    executor_callback))\n', '  File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch\n    executor_callback)\n', '  File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 130, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/usr/lib/python2.7/site-packages/osprofiler/profiler.py", line 105, in wrapper\n    return f(*args, **kwargs)\n', '  File "/usr/lib/python2.7/site-packages/heat/common/context.py", line 300, in wrapped\n    return func(self, ctx, *args, **kwargs)\n', '  File "/usr/lib/python2.7/site-packages/heat/engine/service.py", line 502, in list_stacks\n    return [api.format_stack(stack) for stack in stacks]\n', '  File "/usr/lib/python2.7/site-packages/heat/engine/api.py", line 99, in format_stack\n    rpc_api.STACK_PARAMETERS: stack.parameters.map(str),\n', '  File "/usr/lib/python2.7/site-packages/heat/engine/parameters.py", line 526, in map\n    for n, p in six.iteritems(self.params) if filter_func(p))\n', '  File "/usr/lib/python2.7/site-packages/heat/engine/parameters.py", line 526, in <genexpr>\n    for n, p in six.iteritems(self.params) if filter_func(p))\n', "UnicodeEncodeError: 'ascii' codec can't encode character u'\\u200b' in position 39: ordinal not in range(128)\n"]

Comment 5 Zane Bitter 2016-01-15 00:15:42 UTC
Thanks! Upstream bug created.

Comment 10 Zane Bitter 2016-01-28 18:44:59 UTC
OK, this is fixed in the latest RHOS 8 build; we can clone this bug to RHOS 7 if someone wants a backport.

Comment 15 Felipe Alfaro Solana 2016-04-11 11:03:58 UTC
Please, do backport to OSP7. This is preventing our OpenStack cluster from working properly.

Comment 16 Steve Baker 2016-04-11 21:55:55 UTC
(In reply to Felipe Alfaro Solana from comment #15)
> Please, do backport to OSP7. This is preventing our OpenStack cluster from
> working properly.

Felipe, is there a reason you cannot do the recommended workaround of removing the non-ascii characters from your parameter values? These are likely to cause other problems anyway.

Comment 18 Zane Bitter 2016-04-14 15:19:18 UTC
Felipe - to be more specific, it depends on the reason that you're passing non-ASCII data. If it's non-breaking space characters specifically ('\u200b') as in the original bug report then you *really* want to not do that. If it's just that you have unicode data that you need to pass as parameters then a backport will help.

Comment 19 Amit Ugol 2016-04-21 09:18:05 UTC
if it can handle Klingon...

[stack@undercloud ~]$ heat stack-show a002
[trimmed]
|                       |     "output_value": "",