Bug 1820245

Summary: reports in JSON format can't handle unicode characters
Product: Red Hat Satellite Reporter: Lukáš Hellebrandt <lhellebr>
Component: ReportingAssignee: Dominik Matoulek <dmatoule>
Status: CLOSED ERRATA QA Contact: Lukáš Hellebrandt <lhellebr>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 6.7.0CC: afeferku, dmatoule, mhulan, oprazak, tbrisker
Target Milestone: 6.8.0Keywords: Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: foreman-2.1.0 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-10-27 13:01:16 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
foreman-tail.log none

Description Lukáš Hellebrandt 2020-04-02 15:12:06 UTC
Description of problem:
When a Product exists with special character, e.g. 'Ã' (=\xC3 ASCII), the Subscriptions report in JSON format doesn't get generated, the generating page shows error: "\xC3 ASCII-8BIT to UTF-8". No traceback.

Version-Release number of selected component (if applicable):
Reproduced with Sat 6.7, previous versions didn't support JSON format

How reproducible:
Deterministic

Steps to Reproduce:
1. Have a Satellite with some Subscriptions
2. Create a product "Ãproduct"
3. Generate a Subscriptions report in JSON

Actual results:
Error

Expected results:
Report successfully generated

Comment 3 Lukáš Hellebrandt 2020-04-02 15:15:40 UTC
Created attachment 1675800 [details]
foreman-tail.log

Comment 5 Lukáš Hellebrandt 2020-04-07 14:25:50 UTC
I can confirm it happened to me with a custom product and I'm not aware of any RH product containing such a character.

Comment 7 Tomer Brisker 2020-05-31 14:25:36 UTC
I can confirm this occurs for any report containing unicode characters and generated as JSON (reproduced with registered users report with a user that had utf characters).

Relevant part of the stacktrace: 

2020-05-31T17:19:52 [D|app|f7cfc3a6] Backtrace for 'Action failed' error (Encoding::UndefinedConversionError): "\xC3" from ASCII-8BIT to UTF-8
 | /home/tbrisker/.rvm/gems/ruby-2.6.3@foreman/gems/activesupport-6.0.2.2/lib/active_support/core_ext/object/json.rb:39:in `to_json'
 | /home/tbrisker/.rvm/gems/ruby-2.6.3@foreman/gems/activesupport-6.0.2.2/lib/active_support/core_ext/object/json.rb:39:in `to_json'
 | /home/tbrisker/.rvm/gems/ruby-2.6.3@foreman/gems/activesupport-6.0.2.2/lib/active_support/json/encoding.rb:57:in `to_json'
 | /home/tbrisker/.rvm/gems/ruby-2.6.3@foreman/gems/json-2.3.0/lib/json/common.rb:224:in `generate'
 | /home/tbrisker/.rvm/gems/ruby-2.6.3@foreman/gems/json-2.3.0/lib/json/common.rb:224:in `generate'
 | /home/tbrisker/.rvm/gems/ruby-2.6.3@foreman/gems/activesupport-6.0.2.2/lib/active_support/json/encoding.rb:106:in `stringify'
 | /home/tbrisker/.rvm/gems/ruby-2.6.3@foreman/gems/activesupport-6.0.2.2/lib/active_support/json/encoding.rb:35:in `encode'
 | /home/tbrisker/.rvm/gems/ruby-2.6.3@foreman/gems/activesupport-6.0.2.2/lib/active_support/json/encoding.rb:22:in `encode'
 | /home/tbrisker/.rvm/gems/ruby-2.6.3@foreman/gems/activesupport-6.0.2.2/lib/active_support/core_ext/object/json.rb:42:in `to_json'
 | /home/tbrisker/foreman/app/controllers/api/v2/report_templates_controller.rb:185:in `report_data'

Comment 8 Bryan Kearney 2020-06-08 16:05:36 UTC
Upstream bug assigned to dmatoule

Comment 9 Bryan Kearney 2020-06-08 16:05:38 UTC
Upstream bug assigned to dmatoule

Comment 10 Bryan Kearney 2020-06-16 16:05:39 UTC
Moving this bug to POST for triage into Satellite 6 since the upstream issue https://projects.theforeman.org/issues/30056 has been resolved.

Comment 12 Lukáš Hellebrandt 2020-08-04 15:25:08 UTC
Verified with Sat 6.8 snap 10 using reproducers from OP and comment 7.

Comment 15 errata-xmlrpc 2020-10-27 13:01:16 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Satellite 6.8 release), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:4366