Red Hat Bugzilla – Bug 1477836
No Compatibility for Unicode Characters in RHOSP 10
Last modified: 2017-11-13 09:24:38 EST
Description of problem:
IHAC, working on RHOSP 10 deployment. They have raised an issue with TripleO Heat Deployment since it fails in phase 3 when a project has umlauts in name or description.
Per Cu. as the overcloud CLI and API is not available after that failure, this results in a denial of service [DoS] if undetected in advance
Here is the detailed error log:
Version-Release number of selected component (if applicable):
RHOSP 10, openstack-keystone-10.0.0-4.el7ost.noarch
How to reproduce?
Create a project with an umlaut in it, e.g.:
[stack@ospd ~]$ openstack project list | grep nord
| 2b2296ed7ec04673b3ac3a5507623301 | bq-nordström
[stack@ospd ~]$ openstack project show 2b2296ed7ec04673b3ac3a5507623301
| Field | Value |
| description | Project for Karl Nordström - Uni Saarland |
| domain_id | 61f897cb055f412e81c82b42de718728 |
| enabled | True |
| id | 2b2296ed7ec04673b3ac3a5507623301 |
| is_domain | False |
| name | bq-nordström |
| parent_id | 61f897cb055f412e81c82b42de718728 |
and redeploy the cloud. It will fail in step 3 of the overcloud deployment with the error message as mentioned in log.
"/usr/bin/openstack project list --quiet --format csv --long"
returned 1: 'ascii' codec can't decode byte 0xc3 in position 46: ordinal not in range(128)\n\"ID\",\"Name\",\"Domain ID\",\"Description\",\"Enabled
Throws below error:
Error: Could not prefetch keystone_tenant provider 'openstack': Execution of '/usr/bin/openstack project list --quiet --format csv --long' returned 1: 'ascii' codec can't decode byte 0xc3 in position 46: ordinal not in range(128)\n\"ID\",\"Name\",\"Domain ID\",\"Description\",\"Enabled\"\n\"01331d19d8954450b9e02a9b8871125e\",\"service\",\"default\",\"Tenant for the openstack services\",True\n\"287dae5c8f4043dd96dfbeda8a717a02\",\"B080\",\"default\",\"\",True (tried 40, for a total of 170 seconds)\u001b[0m\n\u001b[1;31mError: Not managing Keystone_tenant[service] due to earlier Keystone API failures.\u001b[0m\n\u001b[1;31mError: /Stage[main]/Keystone::Roles::Admin/Keystone_tenant[service]/ensure: change from absent to present failed: Not managing Keystone_tenant[service] due to earlier Keystone API failures.\u001b[0m\n\u001b[1;31mError: Not managing Keystone_tenant[admin] due to earlier Keystone API failures.\u001b[0m\n\u001b[1;31mError: /Stage[main]/Keystone::Roles::Admin/Keystone_tenant[admin]/ensure: change from absent to present failed: Not managing Keystone_tenant[admin] due to earlier Keystone API failures.
No support/Compatibility for Unicode characters
OpenStack Compatibility for Unicode characters
I have also found the similar BZ for RHOSP 7:
Not sure if the fix proposed upstream was merged, hence opening the new case for latest RHOSP version.
The initial bug description references bug#1327893 which I believe is the likely cause. Although I have not had a chance to verify how puppet is invoking the openstack client I believe it is very likely doing so with it's output redirected which is the topic of bug#1327893. That bug report has comment #13 which lists 2 upstream patches that need to be applied for UTF-8 to work correctly with openstack client. I am the author of both those patches. The first patch (https://review.openstack.org/#/c/342914/5) is the one which should solve the issue reported here. It should have been included in the python-cliff RPM. Upstream cliff has included that patch in the following tags:
As far as I can tell OSP-10 should have shipped with python-cliff-2.2.0 which should have included that fix.
We need to know the version of python-cliff on the node showing the failure.
We also need to know the value of the locale in the environment in which the script executes.
The patch will have modified /usr/lib/python2.7/site-packages/cliff/app.py to add code to the _set_streams() method
Part of that patch includes these lines:
encoding = locale.getpreferredencoding()
If that patch is missing or if locale.getpreferredencoding() does not return
an encoding the if block won't be entered which sets the output encoder
for redirected streams. An incorrect encoding (e.g. ascii) will also cause the failure.
The python-cliff-2.2.0-1.el7ost.noarch package should have the fix in it so my guess is the fix is not executing because of the environment.
The locale is normally defined by the LANG environment variable.
Probably the easiest way to get the information we need is by temporarily adding a print statement to the Python code.
in /usr/lib/python2.7/site-packages/cliff/app.py edit the code to add the print statement so it looks like this.
encoding = locale.getpreferredencoding()
print "encoding=%s LANG=%s" % (encoding, os.environ['LANG'])
You then need to rerun things *exactly* as when the error occurs. That's because many of the configuration tools (e.g. Puppet) modify the environment. As a matter of fact I seem to recall one or more of the tools sets the locale to "C" without any encoding specification which would certainly cause problems with internationalized strings.
A fix for this issue was committed upstream:
The upstream bug is:
The Red Hat bugzilla where this work is being tracked is:
This bug should be closed as a duplicate of the above.
*** This bug has been marked as a duplicate of bug 1437402 ***