Bug 590032

Summary: UnicodeDecodeError in journalling.py
Product: [Fedora] Fedora Reporter: David Kutálek <dkutalek>
Component: beakerlibAssignee: Petr Muller <pmuller>
Status: CLOSED ERRATA QA Contact:
Severity: medium Docs Contact:
Priority: medium    
Version: 19CC: ohudlick, pmuller, psplicha, qa-errata-list
Target Milestone: ---Keywords: Patch
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: beakerlib-1.7-1.fc19 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 950472 (view as bug list) Environment:
Last Closed: 2013-05-22 03:11:32 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description David Kutálek 2010-05-07 14:57:08 UTC
Description of problem:

In one of my rhts tests, beakerlib tracebacks when using unusual locale in test.
Specifically, what fails is journalling.py.

RHTS test: /CoreOS/sos/Regression/bz462824-sosreport-chkconfig-test-in-cluster-plugin-fails

locale: LC_ALL=bokmål

Relevant output of test with tracebacks:

Step 4
:: [   PASS   ] :: Creating tmp file
Traceback (most recent call last):
  File "/usr/lib/beakerlib//python/journalling.py", line 451, in <module>
    addTest(options.testid, options.message, result)
  File "/usr/lib/beakerlib//python/journalling.py", line 365, in addTest
    msg.setAttribute("message", unicode(message,'utf-8').translate(xmlTrans))
UnicodeDecodeError: 'utf8' codec can't decode bytes in position 47-49: invalid data
:: [   PASS   ] :: File /tmp/sosreport-timothy-20100507155508-ef0f.tar.xz should exist
Traceback (most recent call last):
  File "/usr/lib/beakerlib//python/journalling.py", line 451, in <module>
    addTest(options.testid, options.message, result)
  File "/usr/lib/beakerlib//python/journalling.py", line 365, in addTest
    msg.setAttribute("message", unicode(message,'utf-8').translate(xmlTrans))
UnicodeDecodeError: 'utf8' codec can't decode bytes in position 39-41: invalid data



Version-Release number of selected component (if applicable):

beaker-lib-0.4.95-0.1271345738.beaker0_4.el5.noarch

How reproducible:

Always

Steps to Reproduce:
1. run /CoreOS/sos/Regression/bz462824-sosreport-chkconfig-test-in-cluster-plugin-fails in RHTS
2. see OUTPUT.log for tracebacks
  
Actual results:

Locale-related tracebacks of journalling.py

Expected results:

No tracebacks.

Additional info:

Comment 1 Petr Muller 2010-10-20 12:00:11 UTC
Sorry for the lag. Could you try if it still breaks with newish beakerlib? If so, please move this to ASSIGNED, I'll investigate.

Comment 2 David Kutálek 2010-10-20 12:17:40 UTC
Unfortunately still breaks with beakerlib-1.3-4.el6.noarch:

Traceback (most recent call last):
  File "/usr/bin/beakerlib-journalling", line 474, in <module>
    addTest(options.testid, options.message, result)
  File "/usr/bin/beakerlib-journalling", line 388, in addTest
    msg.setAttribute("message", unicode(message,'utf-8').translate(xmlTrans))
UnicodeDecodeError: 'utf8' codec can't decode bytes in position 47-49: invalid data

(line numbers are different now with this version).

Comment 3 Fedora End Of Life 2013-04-03 19:56:41 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 19 development cycle.
Changing version to '19'.

(As we did not run this process for some time, it could affect also pre-Fedora 19 development
cycle bugs. We are very sorry. It will help us with cleanup during Fedora 19 End Of Life. Thank you.)

More information and reason for this action is here:
https://fedoraproject.org/wiki/BugZappers/HouseKeeping/Fedora19

Comment 4 Petr Muller 2013-04-09 16:17:17 UTC
I strongly believe this was fixed in the recent unicode robustification of the journal (not yet in Beaker).

Comment 6 David Kutálek 2013-04-10 10:23:35 UTC
When tested with latest git version I got:

Step 99 - 'bokm�l'
:: [   PASS   ] :: Creating tmp file
:: [   PASS   ] :: Creating cluster sosreport tarball (locale bokm�l )
:: [   PASS   ] :: File /tmp/sosreport-dhcp-25-51-20130410121822-9b7c.tar.xz should exist
:: [   PASS   ] :: Removing sosreport tarball (locale bokm�l)

So no crash, great! Not sure why å letter is crippled, though.

Comment 7 David Kutálek 2013-04-10 10:30:59 UTC
So it seems crippled letter is produced by locale -a, so not beakerlib issue. Simpler reproducer passes:

$ . /usr/lib/beakerlib/beakerlib.sh 
$ rlJournalStart
$ rlLog "å"
:: [12:25:54] ::  å
$ rlRun "true" 0 "å"

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: [   LOG    ] :: Asserts collected outside of a phase
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

:: [   FAIL   ] :: TEST BUG: Assertion not in phase
:: [   PASS   ] :: å

Comment 8 Fedora Update System 2013-04-30 15:12:12 UTC
beakerlib-1.7-1.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/beakerlib-1.7-1.fc19

Comment 9 Fedora Update System 2013-04-30 20:05:34 UTC
Package beakerlib-1.7-1.fc19:
* should fix your issue,
* was pushed to the Fedora 19 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing beakerlib-1.7-1.fc19'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2013-7067/beakerlib-1.7-1.fc19
then log in and leave karma (feedback).

Comment 10 Fedora Update System 2013-05-22 03:11:32 UTC
beakerlib-1.7-1.fc19 has been pushed to the Fedora 19 stable repository.  If problems still persist, please make note of it in this bug report.