Bug 1801159 - subunit2html failed with "UnicodeDecodeError: 'ascii' codec can't decode byte 0xef in position ..."
Summary: subunit2html failed with "UnicodeDecodeError: 'ascii' codec can't decode byte...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: python-os-testr
Version: 13.0 (Queens)
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: ---
Assignee: Chandan Kumar
QA Contact: Martin Kopec
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-02-10 11:49 UTC by lkuchlan
Modified: 2022-08-18 17:16 UTC (History)
8 users (show)

Fixed In Version: python-os-testr-1.0.1-0.20200218144109.7dd678e.el7ost
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-03-10 11:53:27 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1857901 0 None None None 2020-02-10 11:50:15 UTC
OpenStack gerrit 700778 0 None MERGED subunit2html dealing with incorrect data format 2020-03-04 11:46:03 UTC
Red Hat Issue Tracker OSP-3823 0 None None None 2022-08-18 17:16:28 UTC
Red Hat Product Errata RHBA-2020:0769 0 None None None 2020-03-10 11:53:36 UTC

Description lkuchlan 2020-02-10 11:49:48 UTC
The exception is caused by the following code line:
https://github.com/openstack/os-testr/blob/master/os_testr/subunit2html.py#L685

683 script = TemplateData.REPORT_TEST_OUTPUT_TMPL % dict(
684 id=tid,
685 -> output=saxutils.escape(o + e),
686 )
687
688 row = tmpl % dict(
689 tid=tid,
690 Class=((n == 0 or n == 3) and 'hiddenRow' or 'none'),
##################################
For some reason the type of variable is different between o to e, as you can see below:
(Pdb) type(o)
<type 'unicode'>
(Pdb) type(e)
<type 'str'>
(Pdb) o + e
*** UnicodeDecodeError: 'ascii' codec can't decode byte 0xef in position 123639: ordinal not in range(128)

Actual results:

Traceback (most recent call last):
  File "/usr/bin/subunit2html", line 10, in <module>
    sys.exit(main())
  File "/usr/lib/python2.7/site-packages/os_testr/subunit2html.py", line 763, in main
    result.stopTestRun()
  File "/usr/lib/python2.7/site-packages/testtools/testresult/real.py", line 549, in stopTestRun
    sink.stopTestRun()
  File "/usr/lib/python2.7/site-packages/testtools/testresult/real.py", line 1775, in stopTestRun
    self.decorated.stopTestRun()
  File "/usr/lib/python2.7/site-packages/testtools/testresult/real.py", line 1529, in stopTestRun
    return self.decorated.stopTestRun()
  File "/usr/lib/python2.7/site-packages/os_testr/subunit2html.py", line 517, in stopTestRun
    report = self._generate_report()
  File "/usr/lib/python2.7/site-packages/os_testr/subunit2html.py", line 604, in _generate_report
    self._generate_report_test(rows, cid, tid, n, t, o, e)
  File "/usr/lib/python2.7/site-packages/os_testr/subunit2html.py", line 688, in _generate_report_test
    output=saxutils.escape(o + e),
UnicodeDecodeError: 'ascii' codec can't decode byte 0xef in position 123639: ordinal not in range(128)

Comment 1 David Sariel 2020-02-18 04:53:41 UTC
hi, changed assignee to default. lkuchlan was opened the issue.  Any ETA for the fix? 

There is a partial workaround [1] for this issue. The problem with [1] is that manila tempest results will be missed 
due to the presence of the unicode char. So it is still an AutomationBlocker for manila. 



[1]
https://review.gerrithub.io/c/redhat-openstack/infrared/+/478718

Comment 2 lkuchlan 2020-02-18 06:35:12 UTC
(In reply to David Sariel from comment #1)
> hi, changed assignee to default. lkuchlan was opened the issue.  Any ETA for
> the fix? 

There is already fix for this https://review.opendev.org/#/c/700778/


> 
> There is a partial workaround [1] for this issue. The problem with [1] is
> that manila tempest results will be missed 
> due to the presence of the unicode char. So it is still an AutomationBlocker
> for manila. 
> 
> 
> 
> [1]
> https://review.gerrithub.io/c/redhat-openstack/infrared/+/478718

Comment 3 David Sariel 2020-02-18 10:32:27 UTC
@lkuchlan, merging 700778 is definitely required. But osp13 puddle shell be also updated with the fixed version 
the os-testr. That's the reason it should be assigned to the python-os-testr guys that will package it downstream
and for some reason it was assigned to you.

Comment 4 Martin Kopec 2020-02-18 16:18:32 UTC
The reviews is being reviewed upstream. I pinged a few people and brought it up during QA office hours. I expect it will be merged soon. After that it's a matter only a few days it's backported and a new package is built. Then it will need to be included in a puddle, I need to check when the next z-stream for RHOS 13 is gonna happen.

Comment 5 Luigi Toscano 2020-02-18 16:32:01 UTC
(In reply to Martin Kopec from comment #4)
> The reviews is being reviewed upstream. I pinged a few people and brought it
> up during QA office hours. I expect it will be merged soon. 

But it may be dropped, as it is for py2 only and there is an open review to drop py2: https://review.opendev.org/#/c/705522/

> After that it's
> a matter only a few days it's backported and a new package is built. 
We need a manual backport it anyway (os-testr 1.0 vs 1.1).

Comment 17 Martin Kopec 2020-02-24 16:06:05 UTC
Automated jobs are passing with the 'Fixed in version' package.

Comment 19 errata-xmlrpc 2020-03-10 11:53:27 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0769


Note You need to log in before you can comment on or make changes to this bug.