Bug 1164267
| Summary: | sosreport: UnicodeDecodeError: 'utf8' codec can't decode byte 0xd2 in position 8003: invalid continuation byte | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Bruno Goncalves <bgoncalv> |
| Component: | sos | Assignee: | Bryn M. Reeves <bmr> |
| Status: | CLOSED ERRATA | QA Contact: | Martin Kyral <mkyral> |
| Severity: | medium | Docs Contact: | |
| Priority: | medium | ||
| Version: | 7.1 | CC: | agk, bgoncalv, dkutalek, mkyral, sbradley |
| Target Milestone: | rc | Keywords: | EasyFix, Patch, Regression |
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | sos-3.2-9.el7 | Doc Type: | Bug Fix |
| Doc Text: |
no docs needed
|
Story Points: | --- |
| Clone Of: | Environment: | ||
| Last Closed: | 2015-03-05 11:24:02 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Hit the same problem on sos-3.2-3.el7.noarch
on sos-3.2-1.el7.noarch hit different problem as sosreport fails with:
# sosreport --batch
Traceback (most recent call last):
File "/usr/sbin/sosreport", line 25, in <module>
main(sys.argv[1:])
File "/usr/lib/python2.7/site-packages/sos/sosreport.py", line 1407, in main
sos = SoSReport(args)
File "/usr/lib/python2.7/site-packages/sos/sosreport.py", line 633, in __init__
self.policy = sos.policies.load()
File "/usr/lib/python2.7/site-packages/sos/policies/__init__.py", line 40, in load
cache['policy'] = policy()
File "/usr/lib/python2.7/site-packages/sos/policies/redhat.py", line 122, in __init__
super(RHELPolicy, self).__init__()
File "/usr/lib/python2.7/site-packages/sos/policies/redhat.py", line 50, in __init__
if self.package_manager.all_pkgs()['filesystem']['version'][0] == '3':
File "/usr/lib/python2.7/site-packages/sos/policies/__init__.py", line 112, in all_pkgs
self.packages = self.get_pkg_list()
File "/usr/lib/python2.7/site-packages/sos/policies/__init__.py", line 95, in get_pkg_list
pkg_list = shell_out(self.query_command).splitlines()
File "/usr/lib/python2.7/site-packages/sos/utilities.py", line 185, in shell_out
return sos_get_command_output(cmd, runat=runat)['output']
File "/usr/lib/python2.7/site-packages/sos/utilities.py", line 142, in sos_get_command_output
if six.PY2:
AttributeError: 'module' object has no attribute 'PY2'
The problems in comment #0 and comment #2 have very different causes. The first is a unicode UnicodeDecodeError exception due to an invalid byte sequence for the UTF-8 codec. The second is an AttributeError due to a missing 'PY2' member in the python-six compatibility module: this was fixed in sos-six-compat.patch (added in sos-3.2-3.el7): commit 711093509012adbe8d6852f7dad6164c7a755fea Author: Bryn M. Reeves <bmr> Date: Wed Oct 1 17:36:55 2014 +0100 Packaging fix - archive class also needs PY2 vs. PY3 fix Resolves: bz1026962 For the unicode problem this isn't something I've seen in testing; what locale values are you using when you run sosreport? E.g. the output of the locale command: $ locale LANG=en_GB.UTF-8 LC_CTYPE="en_GB.UTF-8" LC_NUMERIC="en_GB.UTF-8" LC_TIME="en_GB.UTF-8" LC_COLLATE="en_GB.UTF-8" LC_MONETARY="en_GB.UTF-8" LC_MESSAGES="en_GB.UTF-8" LC_PAPER="en_GB.UTF-8" LC_NAME="en_GB.UTF-8" LC_ADDRESS="en_GB.UTF-8" LC_TELEPHONE="en_GB.UTF-8" LC_MEASUREMENT="en_GB.UTF-8" LC_IDENTIFICATION="en_GB.UTF-8" LC_ALL= I had a feeling this may be journalctl-related (it's the only command run from the logs plugin):
Running 1/1: logs...
[plugin:logs] collecting path '/var/log/secure'
[plugin:logs] collecting path '/var/log/messages'
[plugin:logs] collecting path '/etc/rsyslog.conf'
[plugin:logs] collecting path '/var/log/boot.log'
[plugin:logs] collecting output of 'journalctl --all --this-boot --no-pager'
Traceback (most recent call last):
File "/usr/sbin/sosreport", line 25, in <module>
main(sys.argv[1:])
File "/usr/lib/python2.7/site-packages/sos/sosreport.py", line 1411, in main
sos.execute()
UnicodeDecodeError: 'utf8' codec can't decode byte 0xd2 in position 8003: invalid continuation byte
> /usr/lib64/python2.7/encodings/utf_8.py(16)decode()
-> return codecs.utf_8_decode(input, errors, True)
Turns out the root cause is garbage characters in the kernel dmesg due to the HP Proliant BIOS:
<D2>? 0000162E)
Nov 15 00:16:08 hp-dl385pg8-03.rhts.eng.nay.redhat.com kernel: ACPI: FACP 00000000bdde1980 000F4 (v03 HP ProLiant 00000002 <D2>? 0000162E)
Nov 15 00:16:08 hp-dl385pg8-03.rhts.eng.nay.redhat.com kernel: ACPI: DSDT 00000000bdde1a80 0D2F7 (v01 HP DSDT 00000001 INTL 20061109)
Nov 15 00:16:08 hp-dl385pg8-03.rhts.eng.nay.redhat.com kernel: ACPI: FACS 00000000bddde140 00040
Nov 15 00:16:08 hp-dl385pg8-03.rhts.eng.nay.redhat.com kernel: ACPI: SPCR 00000000bddde180 00050 (v01 HP SPCRRBSU 00000001 <D2>? 0000162E)
Nov 15 00:16:08 hp-dl385pg8-03.rhts.eng.nay.redhat.com kernel: ACPI: MCFG 00000000bddde200 0003C (v01 HP ProLiant 00000001 00000000)
Nov 15 00:16:08 hp-dl385pg8-03.rhts.eng.nay.redhat.com kernel: ACPI: HPET 00000000bddde240 00038 (v01 HP ProLiant 00000002 <D2>? 0000162E)
Nov 15 00:16:08 hp-dl385pg8-03.rhts.eng.nay.redhat.com kernel: ACPI: SPMI 00000000bddde280 00040 (v05 HP ProLiant 00000001 <D2>? 0000162E)
Nov 15 00:16:08 hp-dl385pg8-03.rhts.eng.nay.redhat.com kernel: ACPI: ERST 00000000bddde2c0 00230 (v01 HP ProLiant 00000001 <D2>? 0000162E)
Nov 15 00:16:08 hp-dl385pg8-03.rhts.eng.nay.redhat.com kernel: ACPI: APIC 00000000bddde500 0011E (v01 HP ProLiant 00000002 00000000)
Nov 15 00:16:08 hp-dl385pg8-03.rhts.eng.nay.redhat.com kernel: ACPI: SRAT 00000000bddde800 002A0 (v02 AMD AGESA 00000001 AMD 00000001)
Nov 15 00:16:08 hp-dl385pg8-03.rhts.eng.nay.redhat.com kernel: ACPI: FFFF 00000000bdddf000 00176 (v01 HP ProLiant 00000001 <D2>? 0000162E)
Nov 15 00:16:08 hp-dl385pg8-03.rhts.eng.nay.redhat.com kernel: ACPI: BERT 00000000bdddf180 00030 (v01 HP ProLiant 00000001 <D2>? 0000162E)
Nov 15 00:16:08 hp-dl385pg8-03.rhts.eng.nay.redhat.com kernel: ACPI: HEST 00000000bdddf1c0 0018C (v01 HP ProLiant 00000001 <D2>? 0000162E)
Nov 15 00:16:08 hp-dl385pg8-03.rhts.eng.nay.redhat.com kernel: ACPI: FFFF 00000000bdddf380 00064 (v02 HP ProLiant 00000002 <D2>? 0000162E)
So this is technically a HP BIOS bug.. Those 0xd2s are neither valid ASCII nor UTF-8.
I'll look for a way for us to handle this more gracefully.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2015-0532.html |
Description of problem: The following traceback happens when running sosreport Running 32/73: logs... logs Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/sos/sosreport.py", line 1163, in collect plug.collect() File "/usr/lib/python2.7/site-packages/sos/plugins/__init__.py", line 629, in collect self._collect_cmd_output() File "/usr/lib/python2.7/site-packages/sos/plugins/__init__.py", line 609, in _collect_cmd_output timeout=timeout, runat=runat) File "/usr/lib/python2.7/site-packages/sos/plugins/__init__.py", line 554, in get_cmd_output_now result = self.get_command_output(exe, timeout=timeout, runat=runat) File "/usr/lib/python2.7/site-packages/sos/plugins/__init__.py", line 464, in get_command_output result = sos_get_command_output(prog, timeout=timeout, runat=runat) File "/usr/lib/python2.7/site-packages/sos/utilities.py", line 162, in sos_get_command_output return {'status': p.returncode, 'output': stdout.decode('utf-8')} File "/usr/lib64/python2.7/encodings/utf_8.py", line 16, in decode return codecs.utf_8_decode(input, errors, True) UnicodeDecodeError: 'utf8' codec can't decode byte 0xd2 in position 8003: invalid continuation byte Version-Release number of selected component (if applicable): sos-3.2-7.el7.noarch How reproducible: 100% Steps to Reproduce: 1.run sosreport # sosreport --batch