Bug 1830236
| Summary: | subscription-manager output in sosreport has encoding issues | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 8 | Reporter: | Kenny Tordeurs <ktordeur> | ||||
| Component: | sos | Assignee: | Pavel Moravec <pmoravec> | ||||
| Status: | CLOSED DUPLICATE | QA Contact: | Upgrades and Supportability <upgrades-and-supportability> | ||||
| Severity: | low | Docs Contact: | |||||
| Priority: | unspecified | ||||||
| Version: | 8.2 | CC: | agk, bmr, jjansky, khoes, plambri, sbradley | ||||
| Target Milestone: | rc | Flags: | pm-rhel:
mirror+
|
||||
| Target Release: | 8.0 | ||||||
| Hardware: | x86_64 | ||||||
| OS: | Linux | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2020-05-01 14:18:58 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Attachments: |
|
||||||
Some additional information:
[root@ktordeur-rhel82-test-register ~]# locale
LANG=ja_JP.UTF-8
LC_CTYPE=en_US.UTF-8
LC_NUMERIC="ja_JP.UTF-8"
LC_TIME="ja_JP.UTF-8"
LC_COLLATE="ja_JP.UTF-8"
LC_MONETARY="ja_JP.UTF-8"
LC_MESSAGES="ja_JP.UTF-8"
LC_PAPER="ja_JP.UTF-8"
LC_NAME="ja_JP.UTF-8"
LC_ADDRESS="ja_JP.UTF-8"
LC_TELEPHONE="ja_JP.UTF-8"
LC_MEASUREMENT="ja_JP.UTF-8"
LC_IDENTIFICATION="ja_JP.UTF-8"
LC_ALL=
[root@ktordeur-rhel82-test-register ~]# localectl
System Locale: LANG=ja_JP.UTF-8
VC Keymap: us
X11 Layout: n/a
[root@ktordeur-rhel82-test-register ~]# subscription-manager facts --list | grep default_locale
system.default_locale: en_US.UTF-8
[root@ktordeur-rhel82-test-register ~]# LANG=ja_JP.UTF-8 subscription-manager facts --list | grep default_locale
system.default_locale: en_US.UTF-8
Created attachment 1683634 [details]
sosreport
> The subscription-manager commands work fine when running them on the CLI, it's only failing when they are run via sos.
That's because when you run them at the CLI they are using the ja_JP.UTF-8 text locale (which allows non-ASCII characters in UTF8 encoding). When sos runs commands it explicitly sets "LC_ALL=C" so that collation etc. is predictable regardless of the locale that the system is configured to use. Since this does not permit non-ASCII characters in ASCII encodings it leads to errors when these appear in the output of the command (probably due to some names or other values that are using characters that do not exist in ASCII).
That's why it is better to set codecs explicitly or to set the 'errors' keyword argument to a value that either ignores these non-encodable values, or that replaces them with hex escapes:
string.encode('utf8', errors='ignore')
string.encode('utf8', errors='replace')
It's possible that today sos would be better off setting LC_ALL to a value that /does/ allow Unicode, i.e. LC_ALL=C.UTF-8 but this would still leave this bug in subscription-manager - a user setting LC_ALL=C would still see the same problem.
To confirm, do you get the same exception if you run subscription-manager like this?
# LC_ALL=C subscription-manager list --consumed
jjansky pointed out that s-m does call setdefaultencoding() to try to avoid this but that setting can be overridden by the environment so it's not a general solution. @bmr
Yes, I can reproduce the issue by running it like that:
[root@ktordeur-rhel82-test-register ~]# LC_ALL=C subscription-manager list --consumed
+-------------------------------------------+
Traceback (most recent call last):
File "/usr/sbin/subscription-manager", line 11, in <module>
load_entry_point('subscription-manager==1.26.16', 'console_scripts', 'subscription-manager')()
File "/usr/lib64/python3.6/site-packages/subscription_manager/scripts/subscription_manager.py", line 87, in main
return managercli.ManagerCLI().main()
File "/usr/lib64/python3.6/site-packages/subscription_manager/managercli.py", line 2996, in main
ret = CLI.main(self)
File "/usr/lib64/python3.6/site-packages/subscription_manager/cli.py", line 183, in main
return cmd.main()
File "/usr/lib64/python3.6/site-packages/subscription_manager/managercli.py", line 536, in main
return_code = self._do_command()
File "/usr/lib64/python3.6/site-packages/subscription_manager/managercli.py", line 2704, in _do_command
self.print_consumed(service_level=self.options.service_level, filter_string=self.options.filter_string, pid_only=self.options.pid_only)
File "/usr/lib64/python3.6/site-packages/subscription_manager/managercli.py", line 2721, in print_consumed
print(" " + _("Consumed Subscriptions"))
UnicodeEncodeError: 'ascii' codec can't encode characters in position 3-18: ordinal not in range(128)
Thanks for the clarification, I went ahead and filed the bugzilla directly against the subscription-manager component: https://bugzilla.redhat.com/show_bug.cgi?id=1830294 *** This bug has been marked as a duplicate of bug 1830294 *** |
Description of problem: The following error is shown when trying to cat contents under sos_commands/subscription_manager/* while the commands themselves work fine on CLI. ~~~ UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-3: ordinal not in range(128) ~~~ Version-Release number of selected component (if applicable): sos-3.8-2.el8.noarch How reproducible: 100% Steps to Reproduce: # yum install glibc-langpack-ja # localectl set-locale LANG=ja_JP.UTF-8 - Register the system - Run sos Actual results: # cat sos_commands/subscription_manager/subscription-manager_identity ~~~ --- Logging error --- Traceback (most recent call last): File "/usr/lib64/python3.6/site-packages/subscription_manager/managercli.py", line 917, in _do_command print(_('system identity: %s') % consumerid) UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-3: ordinal not in range(128) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib64/python3.6/logging/__init__.py", line 996, in emit stream.write(msg) UnicodeEncodeError: 'ascii' codec can't encode characters in position 90-92: ordinal not in range(128) Call stack: File "/usr/sbin/subscription-manager", line 11, in <module> load_entry_point('subscription-manager==1.26.16', 'console_scripts', 'subscription-manager')() File "/usr/lib64/python3.6/site-packages/subscription_manager/scripts/subscription_manager.py", line 87, in main return managercli.ManagerCLI().main() File "/usr/lib64/python3.6/site-packages/subscription_manager/managercli.py", line 2996, in main ret = CLI.main(self) File "/usr/lib64/python3.6/site-packages/subscription_manager/cli.py", line 183, in main return cmd.main() File "/usr/lib64/python3.6/site-packages/subscription_manager/managercli.py", line 536, in main return_code = self._do_command() File "/usr/lib64/python3.6/site-packages/subscription_manager/managercli.py", line 950, in _do_command handle_exception(_("Error: Unable to generate a new identity for the system"), e) File "/usr/lib64/python3.6/site-packages/subscription_manager/managercli.py", line 216, in handle_exception log.error(msg) Message: '\u30a8\u30e9\u30fc: \u30b7\u30b9\u30c6\u30e0\u7528\u306b\u65b0\u898f\u306e\u8b58\u5225\u5b50\u3092\u751f\u6210\u3067\u304d\u307e\u305b\u3093' Arguments: () 'ascii' codec can't encode characters in position 0-3: ordinal not in range(128) ~~~ # cat sos_commands/subscription_manager/subscription-manager_list_--installed | head -n5 ~~~ +-------------------------------------------+ Traceback (most recent call last): File "/usr/sbin/subscription-manager", line 11, in <module> load_entry_point('subscription-manager==1.26.16', 'console_scripts', 'subscription-manager')() File "/usr/lib64/python3.6/site-packages/subscription_manager/scripts/subscription_manager.py", line 87, in main ~~~ # cat sos_commands/subscription_manager/subscription-manager_list_--consumed | head -n5 ~~~ +-------------------------------------------+ Traceback (most recent call last): File "/usr/sbin/subscription-manager", line 11, in <module> load_entry_point('subscription-manager==1.26.16', 'console_scripts', 'subscription-manager')() File "/usr/lib64/python3.6/site-packages/subscription_manager/scripts/subscription_manager.py", line 87, in main ~~~ Expected results: # subscription-manager identity ~~~ システム ID: d85f22b4-6cee-4bb4-a374-512355231a91 名前: ktordeur-rhel82-test-register.sysmgmt.lan 組織名: Default Organization 組織 ID: Default_Organization 環境名: Library ~~~ # subscription-manager list --installed ~~~ +-------------------------------------------+ インストール済み製品のステータス +-------------------------------------------+ 製品名: Red Hat Enterprise Linux for x86_64 製品 ID: 479 バージョン: 8.2 アーキテクチャー: x86_64 状態: サブスクライブ済み 状態の詳細: 開始: 2013年04月24日 終了: 2022年01月01日 ~~~ Additional info: All the commands execute correctly when done on the CLI and provide the correct output, only the contents in the sosreport are erroring out.