Bug 1457606

Summary: beaker-watchdog crashes in report_panic with UnicodeDecodeError when job whiteboard contains non-ASCII characters
Product: [Community] Beaker Reporter: Dan Callaghan <dcallagh>
Component: generalAssignee: Dan Callaghan <dcallagh>
Status: CLOSED CURRENTRELEASE QA Contact: Dan Callaghan <dcallagh>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 24CC: dcallagh, mjia, rjoost
Target Milestone: 24.4Keywords: Patch
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-10-03 03:57:44 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Dan Callaghan 2017-06-01 02:26:49 UTC
Description of problem:


Version-Release number of selected component (if applicable):
24.2

How reproducible:


Steps to Reproduce:
1. Make a job with U+3000 IDEOGRAPHIC SPACE into the job whiteboard
2. Trigger the panic detection somehow(?)

Actual results:
beaker-watchdog crashes

Expected results:
beaker-watchdog should properly detect and handle the panic (abort the recipe), no crash

Additional info:
2017-06-01 10:17:52,236 bkr.labcontroller.proxy DEBUG get_recipe recipe_id:3895188
Traceback (most recent call last):
  File "/usr/bin/beaker-watchdog", line 9, in <module>
    load_entry_point('beaker-lab-controller==24.2', 'console_scripts', 'beaker-watchdog')()
  File "/usr/lib/python2.6/site-packages/bkr/labcontroller/watchdog.py", line 118, in main
    main_loop(watchdog, conf)
  File "/usr/lib/python2.6/site-packages/bkr/labcontroller/watchdog.py", line 66, in main_loop
    if not watchdog.run():
  File "/usr/lib/python2.6/site-packages/bkr/labcontroller/proxy.py", line 552, in run
    updated |= monitor.run()
  File "/usr/lib/python2.6/site-packages/bkr/labcontroller/proxy.py", line 617, in run
    return self.console_watch.update()
  File "/usr/lib/python2.6/site-packages/bkr/labcontroller/proxy.py", line 287, in update
    self.process_log(block)
  File "/usr/lib/python2.6/site-packages/bkr/labcontroller/proxy.py", line 244, in process_log
    self.proxy.report_panic(self.watchdog, panic_found)
  File "/usr/lib/python2.6/site-packages/bkr/labcontroller/proxy.py", line 624, in report_panic
    dict(recipe_id=watchdog['recipe_id']))).recipeSet
  File "/usr/lib/python2.6/site-packages/xmltramp.py", line 259, in parse
    return seed(StringIO(text))
  File "/usr/lib/python2.6/site-packages/xmltramp.py", line 254, in seed
    parser.parse(fileobj)
  File "/usr/lib64/python2.6/xml/sax/expatreader.py", line 107, in parse
    xmlreader.IncrementalParser.parse(self, source)
  File "/usr/lib64/python2.6/xml/sax/xmlreader.py", line 123, in parse
    self.feed(buffer)
  File "/usr/lib64/python2.6/xml/sax/expatreader.py", line 207, in feed
    self._parser.Parse(data, isFinal)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u3000' in position 54: ordinal not in range(128)

Comment 1 Dan Callaghan 2017-06-01 02:52:46 UTC
Workaround is to find the problematic recipe (the recipe id is mentioned in the debug logs as per above), go to the Beaker web ui for that recipe, click through to the job which contains it, and edit the whiteboard to delete any non-ASCII characters. In this case we replaced the U+3000 with a normal space character and beaker-watchdog was able to proceed as usual.

Comment 2 Dan Callaghan 2017-06-01 04:07:46 UTC
Ultimately this is because xmltramp is passing its input to cStringIO.StringIO which fails to handle unicode:

>>> from cStringIO import StringIO
>>> StringIO(u'fat\u3000space')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeEncodeError: 'ascii' codec can't encode character u'\u3000' in position 3: ordinal not in range(128)

We should just switch it over to lxml. It looks to be one of the last remaining usages of xmltramp we have left.

Comment 3 Dan Callaghan 2017-06-01 04:12:21 UTC
https://gerrit.beaker-project.org/5708

Comment 6 Dan Callaghan 2017-10-03 03:57:44 UTC
Beaker 24.4 has been released.