Bug 1830294

Summary: subscription-manager encoding issues
Product: Red Hat Enterprise Linux 8 Reporter: Kenny Tordeurs <ktordeur>
Component: subscription-managerAssignee: candlepin-bugs
Status: CLOSED MIGRATED QA Contact: Red Hat subscription-manager QE Team <rhsm-qe>
Severity: medium Docs Contact:
Priority: medium    
Version: 8.2CC: cww, jhnidek, jjansky, masanari.iida, redakkan
Target Milestone: rcKeywords: MigratedToJIRA, Triaged
Target Release: 8.0Flags: pm-rhel: mirror+
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-09-22 12:52:30 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Kenny Tordeurs 2020-05-01 14:16:01 UTC
Description of problem:
The following error is shown when running `LC_ALL=C subscription-manager list --consumed`
~~~
UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-3: ordinal not in range(128)
~~~

When sos runs commands it explicitly sets "LC_ALL=C" so that collation etc. is predictable regardless of the locale that the system is configured to use. Since this does not permit non-ASCII characters in ASCII encodings it leads to errors when these appear in the output of the command (probably due to some names or other values that are using characters that do not exist in ASCII).

That's why it is better to set codecs explicitly or to set the 'errors' keyword argument to a value that either ignores these non-encodable values, or that replaces them with hex escapes:

    string.encode('utf8', errors='ignore')
    string.encode('utf8', errors='replace')

It's possible that today sos would be better off setting LC_ALL to a value that /does/ allow Unicode, i.e. LC_ALL=C.UTF-8 but this would still leave this bug in subscription-manager - a user setting LC_ALL=C would still see the same problem.

Version-Release number of selected component (if applicable):
subscription-manager-cockpit-1.26.16-1.el8.noarch
subscription-manager-rhsm-certificates-1.26.16-1.el8.x86_64
python3-subscription-manager-rhsm-1.26.16-1.el8.x86_64
subscription-manager-initial-setup-addon-1.26.16-1.el8.x86_64
dnf-plugin-subscription-manager-1.26.16-1.el8.x86_64
subscription-manager-1.26.16-1.el8.x86_64


How reproducible:
100%

Steps to Reproduce:
# yum install glibc-langpack-ja
# localectl set-locale LANG=ja_JP.UTF-8
- Register the system
- `LC_ALL=C subscription-manager list --consumed` or run sos and try to cat the contents of sos_commands/subscription_manager/*


Actual results:
# LC_ALL=C subscription-manager list --consumed
~~~
+-------------------------------------------+
Traceback (most recent call last):
  File "/usr/sbin/subscription-manager", line 11, in <module>
    load_entry_point('subscription-manager==1.26.16', 'console_scripts', 'subscription-manager')()
  File "/usr/lib64/python3.6/site-packages/subscription_manager/scripts/subscription_manager.py", line 87, in main
    return managercli.ManagerCLI().main()
  File "/usr/lib64/python3.6/site-packages/subscription_manager/managercli.py", line 2996, in main
    ret = CLI.main(self)
  File "/usr/lib64/python3.6/site-packages/subscription_manager/cli.py", line 183, in main
    return cmd.main()
  File "/usr/lib64/python3.6/site-packages/subscription_manager/managercli.py", line 536, in main
    return_code = self._do_command()
  File "/usr/lib64/python3.6/site-packages/subscription_manager/managercli.py", line 2704, in _do_command
    self.print_consumed(service_level=self.options.service_level, filter_string=self.options.filter_string, pid_only=self.options.pid_only)
  File "/usr/lib64/python3.6/site-packages/subscription_manager/managercli.py", line 2721, in print_consumed
    print("   " + _("Consumed Subscriptions"))
UnicodeEncodeError: 'ascii' codec can't encode characters in position 3-18: ordinal not in range(128)
~~~

Expected results:
# subscription-manager identity
~~~
システム ID: d85f22b4-6cee-4bb4-a374-512355231a91
名前: ktordeur-rhel82-test-register.sysmgmt.lan
組織名: Default Organization
組織 ID: Default_Organization
環境名: Library
~~~

# subscription-manager list --installed
~~~
+-------------------------------------------+
    インストール済み製品のステータス
+-------------------------------------------+
製品名:           Red Hat Enterprise Linux for x86_64
製品 ID:          479
バージョン:       8.2
アーキテクチャー: x86_64
状態:             サブスクライブ済み
状態の詳細:       
開始:             2013年04月24日
終了:             2022年01月01日
~~~

Additional info:
[root@ktordeur-rhel82-test-register ~]# locale
LANG=ja_JP.UTF-8
LC_CTYPE=en_US.UTF-8
LC_NUMERIC="ja_JP.UTF-8"
LC_TIME="ja_JP.UTF-8"
LC_COLLATE="ja_JP.UTF-8"
LC_MONETARY="ja_JP.UTF-8"
LC_MESSAGES="ja_JP.UTF-8"
LC_PAPER="ja_JP.UTF-8"
LC_NAME="ja_JP.UTF-8"
LC_ADDRESS="ja_JP.UTF-8"
LC_TELEPHONE="ja_JP.UTF-8"
LC_MEASUREMENT="ja_JP.UTF-8"
LC_IDENTIFICATION="ja_JP.UTF-8"
LC_ALL=

[root@ktordeur-rhel82-test-register ~]# localectl 
   System Locale: LANG=ja_JP.UTF-8
       VC Keymap: us
      X11 Layout: n/a

[root@ktordeur-rhel82-test-register ~]# subscription-manager facts --list | grep default_locale
system.default_locale: en_US.UTF-8


[root@ktordeur-rhel82-test-register ~]# LANG=ja_JP.UTF-8 subscription-manager facts --list | grep default_locale
system.default_locale: en_US.UTF-8

Comment 1 Kenny Tordeurs 2020-05-01 14:18:58 UTC
*** Bug 1830236 has been marked as a duplicate of this bug. ***

Comment 7 RHEL Program Management 2023-09-22 12:51:02 UTC
Issue migration from Bugzilla to Jira is in process at this time. This will be the last message in Jira copied from the Bugzilla bug.

Comment 8 RHEL Program Management 2023-09-22 12:52:30 UTC
This BZ has been automatically migrated to the issues.redhat.com Red Hat Issue Tracker. All future work related to this report will be managed there.

Due to differences in account names between systems, some fields were not replicated.  Be sure to add yourself to Jira issue's "Watchers" field to continue receiving updates and add others to the "Need Info From" field to continue requesting information.

To find the migrated issue, look in the "Links" section for a direct link to the new issue location. The issue key will have an icon of 2 footprints next to it, and begin with "RHEL-" followed by an integer.  You can also find this issue by visiting https://issues.redhat.com/issues/?jql= and searching the "Bugzilla Bug" field for this BZ's number, e.g. a search like:

"Bugzilla Bug" = 1234567

In the event you have trouble locating or viewing this issue, you can file an issue by sending mail to rh-issues. You can also visit https://access.redhat.com/articles/7032570 for general account information.