Bug 1221149 - SmartState fails to show UTF-8 encoded files
Summary: SmartState fails to show UTF-8 encoded files
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat CloudForms Management Engine
Classification: Red Hat
Component: SmartState Analysis
Version: 5.3.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: GA
: 5.5.3
Assignee: Milan Zázrivec
QA Contact: Ramesh A
URL:
Whiteboard: retest:smartstate
: 1222508 (view as bug list)
Depends On:
Blocks: 1289329
TreeView+ depends on / blocked
 
Reported: 2015-05-13 11:39 UTC by Christian Jung
Modified: 2016-08-02 14:49 UTC (History)
11 users (show)

Fixed In Version: 5.5.3.2
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 1289329 (view as bug list)
Environment:
Last Closed: 2016-04-13 18:39:08 UTC
Category: ---
Cloudforms Team: ---
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
mrt.log which causes the error (874 bytes, application/octet-stream)
2015-05-13 11:40 UTC, Christian Jung
no flags Details
wlms incorrectly reported as being binary (24.68 KB, image/png)
2015-09-07 15:08 UTC, Christian Jung
no flags Details
mrt.log incorrectly reported as bein binary (24.10 KB, image/png)
2015-09-07 15:08 UTC, Christian Jung
no flags Details
UTF Error (60.13 KB, image/png)
2016-01-20 11:55 UTC, Ramesh A
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2016:0616 0 normal SHIPPED_LIVE CFME 5.5.3 bug fixes and enhancement update 2016-04-13 22:38:44 UTC

Description Christian Jung 2015-05-13 11:39:55 UTC
Description of problem:
I configured SmartState Analysis in CF and extended the list of included files to c:\windows\debug\* and check the "Collect contents" check box.

After running a SSA the following files are retrieved:
mrt.log
passwd.log
sammui.log
wlms.log

I can open all files except mrt.log. When opening this file to view in the CF web UI, the following error is shown:
incompatible encoding regexp match (UTF-8 regexp with ASCII-8BIT string) [vm_infra/filesystems] 

Version-Release number of selected component (if applicable):
cfme-5.3.4.2-1.el6cf.x86_64

How reproducible:
always for specific files

Steps to Reproduce:
1. configure SSA profile to include c:\windows\debug\* 
2. run SSA
3. view the content of mrt.log

Actual results:
error

Expected results:
view content of file

Additional info:
I tested this with Windows 7 (all updates installed) and Windows 2012 R2 (all updates installed). Both versions of Windows were English.

Note that mrt.log only exists after the Microsoft MRT Tool was running at least once. Other files with similar encoding should show similar results though.

This might also be helpful:
mrt.log:    Little-endian UTF-16 Unicode text, with CRLF, CR, LF line terminators
PASSWD.LOG: empty
sammui.log: ASCII text, with CRLF line terminators
wlms.log:   ASCII text, with CRLF line terminators

Comment 1 Christian Jung 2015-05-13 11:40:44 UTC
Created attachment 1025020 [details]
mrt.log which causes the error

Comment 4 Jason Frey 2015-05-14 14:18:11 UTC
In summary, UTF-16 file contents can't be shown in the UI and give an error about UTF8 conversion.

Comment 6 Martin Povolny 2015-09-07 09:59:47 UTC
Christian, we have changed the logic around displaying these files in upstream.

As the target release is now 5.5, the upstream code is, what is relevant now.

Can you, please, verify that this bug is present in upstream?

Thx!

Comment 7 Christian Jung 2015-09-07 15:07:15 UTC
HiHo,
I tested this again today with Botvinik. Botvinik falsely reports all files as being binary and unable to view.

Even files which were shown fine with 5.3 are now reported as being binary (which is not correct). 

Regards,
Christian

Comment 8 Christian Jung 2015-09-07 15:08:00 UTC
Created attachment 1071051 [details]
wlms incorrectly reported as being binary

Comment 9 Christian Jung 2015-09-07 15:08:44 UTC
Created attachment 1071053 [details]
mrt.log incorrectly reported as bein binary

Comment 10 Martin Povolny 2015-11-05 15:58:27 UTC
So we have 2 options here I'd say

1) try to do someting smart like guess the encoding, try to convert the file to utf8 to display it

2) offer a download link for files that we cannot display

I don't think that 1) is a case for a simple fix.

Would 2) work for you?

Comment 12 Milan Zázrivec 2015-11-23 16:33:55 UTC
https://github.com/ManageIQ/manageiq/pull/5568

Comment 13 CFME Bot 2015-11-24 03:20:18 UTC
New commit detected on ManageIQ/manageiq/master:
https://github.com/ManageIQ/manageiq/commit/56f5ac2c74f1550d7af9d8e68ec9e87f287d34d1

commit 56f5ac2c74f1550d7af9d8e68ec9e87f287d34d1
Author:     Milan Zazrivec <mzazrivec>
AuthorDate: Mon Nov 23 17:30:56 2015 +0100
Commit:     Milan Zazrivec <mzazrivec>
CommitDate: Mon Nov 23 18:18:05 2015 +0100

    Add download link for filesystem data
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1221149

 app/controllers/vm_common.rb               |  5 +++++
 app/views/layouts/_item.html.haml          |  2 ++
 config/routes.rb                           |  2 ++
 spec/factories/miq_report.rb               | 11 +++++++++++
 spec/routing/vm_cloud_routing_spec.rb      |  6 ++++++
 spec/routing/vm_infra_routing_spec.rb      |  6 ++++++
 spec/views/layouts/_item.html.haml_spec.rb | 22 ++++++++++++++++++++++
 7 files changed, 54 insertions(+)
 create mode 100644 spec/views/layouts/_item.html.haml_spec.rb

Comment 14 Martin Povolny 2015-11-24 10:06:06 UTC
*** Bug 1222508 has been marked as a duplicate of this bug. ***

Comment 16 CFME Bot 2015-12-07 22:42:29 UTC
New commit detected on cfme/5.5.z:
https://code.engineering.redhat.com/gerrit/gitweb?p=cfme.git;a=commitdiff;h=f9a50ec9c42b721af115d107c49c2a190b5609f5

commit f9a50ec9c42b721af115d107c49c2a190b5609f5
Author:     Milan Zazrivec <mzazrivec>
AuthorDate: Mon Nov 23 17:30:56 2015 +0100
Commit:     Milan Zazrivec <mzazrivec>
CommitDate: Tue Nov 24 13:30:19 2015 +0100

    Add download link for filesystem data
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1221149
    (cherry picked from commit 56f5ac2c74f1550d7af9d8e68ec9e87f287d34d1)

 app/controllers/vm_common.rb               |  5 +++++
 app/views/layouts/_item.html.haml          |  2 ++
 config/routes.rb                           |  2 ++
 spec/factories/miq_report.rb               | 11 +++++++++++
 spec/routing/vm_cloud_routing_spec.rb      |  6 ++++++
 spec/routing/vm_infra_routing_spec.rb      |  6 ++++++
 spec/views/layouts/_item.html.haml_spec.rb | 22 ++++++++++++++++++++++
 7 files changed, 54 insertions(+)
 create mode 100644 spec/views/layouts/_item.html.haml_spec.rb

Comment 17 CFME Bot 2015-12-07 22:42:35 UTC
New commit detected on cfme/5.5.z:
https://code.engineering.redhat.com/gerrit/gitweb?p=cfme.git;a=commitdiff;h=6077f611821d0ebd8c7a15234546747d6d6d74ed

commit 6077f611821d0ebd8c7a15234546747d6d6d74ed
Merge: 539830f f9a50ec
Author:     Dan Clarizio <dclarizi>
AuthorDate: Mon Dec 7 17:40:00 2015 -0500
Commit:     Dan Clarizio <dclarizi>
CommitDate: Mon Dec 7 17:40:00 2015 -0500

    Merge branch 'filesystem_download_link' into '5.5.z'
    
    Add download link for filesystem data
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1221149
    
    Clean cherry pick from commit 56f5ac2c74f1550d7af9d8e68ec9e87f287d34d1
    
    See merge request !529

 app/controllers/vm_common.rb               |  5 +++++
 app/views/layouts/_item.html.haml          |  2 ++
 config/routes.rb                           |  2 ++
 spec/factories/miq_report.rb               | 11 +++++++++++
 spec/routing/vm_cloud_routing_spec.rb      |  6 ++++++
 spec/routing/vm_infra_routing_spec.rb      |  6 ++++++
 spec/views/layouts/_item.html.haml_spec.rb | 22 ++++++++++++++++++++++
 7 files changed, 54 insertions(+)

Comment 20 Ramesh A 2016-01-20 11:54:03 UTC
Forgot to mention, it happens only for mrt.log.  For other logs - passwd.log, sammui.log, download button is available.

Comment 21 Ramesh A 2016-01-20 11:55:05 UTC
Created attachment 1116628 [details]
UTF Error

Comment 22 Ramesh A 2016-01-20 11:55:45 UTC
Verified in 5.5.2.1.20160114044944_395c086

Comment 24 Milan Zázrivec 2016-01-21 14:55:20 UTC
https://github.com/ManageIQ/manageiq/pull/6282

Comment 26 CFME Bot 2016-02-10 19:00:45 UTC
New commit detected on cfme/5.5.z:
https://code.engineering.redhat.com/gerrit/gitweb?p=cfme.git;a=commitdiff;h=d062d845f43351b57cee6c124866407ab7f53bdb

commit d062d845f43351b57cee6c124866407ab7f53bdb
Author:     Milan Zazrivec <mzazrivec>
AuthorDate: Thu Jan 21 15:26:10 2016 +0100
Commit:     Milan Zazrivec <mzazrivec>
CommitDate: Tue Jan 26 11:48:48 2016 +0100

    Fix rendering of UTF-16 blobs by conversion to UTF-8
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1221149
    (cherry picked from commit 1cf749c1942840e2dd1331a8b71fa0c826fb1162)

 app/helpers/ui_constants.rb       |  3 +++
 app/models/filesystem.rb          | 10 ++++++++++
 app/views/layouts/_item.html.haml |  2 +-
 spec/models/filesystem_spec.rb    | 19 +++++++++++++++++++
 4 files changed, 33 insertions(+), 1 deletion(-)

Comment 27 CFME Bot 2016-02-10 19:01:24 UTC
New commit detected on cfme/5.5.z:
https://code.engineering.redhat.com/gerrit/gitweb?p=cfme.git;a=commitdiff;h=a984ddac1a4866e282ef2b79090faa5d354b4fd8

commit a984ddac1a4866e282ef2b79090faa5d354b4fd8
Merge: b5b0670 d062d84
Author:     Dan Clarizio <dclarizi>
AuthorDate: Wed Feb 10 13:44:17 2016 -0500
Commit:     Dan Clarizio <dclarizi>
CommitDate: Wed Feb 10 13:44:17 2016 -0500

    Merge branch 'fix_displaying_of_utf16_blobs' into '5.5.z'
    
    Fix rendering of UTF-16 blobs by conversion to UTF-8
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1221149
    (cleanly cherry picked from commit 1cf749c1942840e2dd1331a8b71fa0c826fb1162)
    
    See merge request !748

 app/helpers/ui_constants.rb       |  3 +++
 app/models/filesystem.rb          | 10 ++++++++++
 app/views/layouts/_item.html.haml |  2 +-
 spec/models/filesystem_spec.rb    | 19 +++++++++++++++++++
 4 files changed, 33 insertions(+), 1 deletion(-)

Comment 29 Ramesh A 2016-04-01 04:24:10 UTC
Good to go.  Verified and working fine in 5.5.3.2.20160318152106_92d2d67.

Now CFME appliance displays the mrt.log file contents and download button is also available.  Clicking on the download button, downloads the file correctly.

Comment 31 errata-xmlrpc 2016-04-13 18:39:08 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:0616

Comment 32 Ramesh A 2016-08-02 07:04:04 UTC
Test case added : RHCF3-11385 -  test_ssa_files[windows-utf-8-files]


Note You need to log in before you can comment on or make changes to this bug.