Bug 1295887

Summary: Certificate organization with characters outside the ASCII causes a setup failure
Product: [oVirt] otopi Reporter: Juan Hernández <juan.hernandez>
Component: GeneralAssignee: Yedidyah Bar David <didi>
Status: CLOSED CURRENTRELEASE QA Contact: Lukas Svaty <lsvaty>
Severity: high Docs Contact:
Priority: unspecified    
Version: 1.4.0CC: bugs, didi, lsvaty, lveyde, nsednev, oourfali, rmartins, sbonazzo, stirabos, ylavi
Target Milestone: ovirt-4.0.0-alphaFlags: rule-engine: ovirt-4.0.0+
rule-engine: planning_ack+
sbonazzo: devel_ack+
pstehlik: testing_ack+
Target Release: 1.5.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: ovirt 4.0.0 alpha1 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-07-05 07:38:31 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Integration RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Juan Hernández 2016-01-05 17:16:54 UTC
When a string containing characters outside the ASCII character set is provided as the name of the organization of the certificate it produces a failure of the 
setup, with the following error message:

---8<---
[ INFO  ] Generating post install configuration file '/etc/ovirt-engine-setup.conf.d/20-setup-ovirt-post.conf'
[ ERROR ] Failed to execute stage 'Misc configuration': 'ascii' codec can't decode byte 0xc3 in position 38: ordinal not in range(128)
--->8---

The log says the following:

---8<---
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/otopi/context.py", line 146, in _executeMethod
    method['method']()
  File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-common/base/core/postinstall.py", line 86, in _misc
    otopicons.CoreEnv.MODIFIED_FILES
  File "/usr/lib/python2.7/site-packages/otopi/filetransaction.py", line 161, in __init__
    self._content = '\n'.join(content)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 38: ordinal not in range(128)
--->8---

I believe that the root cause of the problem is in otopi, when it reads the string from the input. The text given by the user is converted into a Python str object, but encoded using UTF-8. For example, if the text "á" is given then it will be translated into the byte string [195, 173] (an instance of "str", not "unicode"). This works well with most Python string functions, as they don't care about the fact that 195 and 173 aren't valid ASCII characters. But functions that try to convert this byte string to Unicode will fail, with an exception similar to this:

  File "./test.py", line 6, in <module>
    name = name.decode('ascii')
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 0: ordinal not in range(128)

One notable case is the string "join" method. This methods joins a list of strings with a separator, and it supports both "str" and "unicode", even mixed:

  list = [ byte_string, unicode_string ]
  joined = '\n'.join(list)

In the above example, if "byte_string" is the result of reading "á" then the UnicodeDecodeError exception will be raised.

The "join" method is used like this in multiple places inside otopi, in particular inside the FileTransaction.__init__ method, to convert a list of strings to the content of a file, and this method is used by engine-setup in order to generate the /etc/ovirt-engine-setup.conf.d/20-setup-ovirt-post.conf file.

Comment 1 Nikolai Sednev 2016-04-11 14:04:10 UTC
Please take a look at the https://bugzilla.redhat.com/show_bug.cgi?id=1316908#c5, might be useful for this bug too.

Comment 2 Lukas Svaty 2016-07-01 11:11:12 UTC
verified in otopi-1.5.0-3.el7ev.noarch ovirt-engine-4.0.0.6-0.1.el7ev.noarch

Comment 3 Sandro Bonazzola 2016-07-05 07:38:31 UTC
oVirt 4.0.0 has been released, closing current release.