Bug 1221149

Summary: SmartState fails to show UTF-8 encoded files
Product: Red Hat CloudForms Management Engine Reporter: Christian Jung <cbolz>
Component: SmartState AnalysisAssignee: Milan Zázrivec <mzazrivec>
Status: CLOSED ERRATA QA Contact: Ramesh A <rananda>
Severity: medium Docs Contact:
Priority: medium    
Version: 5.3.0CC: cpelland, dajohnso, dclarizi, gblomqui, hkataria, jfrey, jhardy, jprause, mcornea, mpovolny, obarenbo
Target Milestone: GAKeywords: ZStream
Target Release: 5.5.3   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: retest:smartstate
Fixed In Version: 5.5.3.2 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1289329 (view as bug list) Environment:
Last Closed: 2016-04-13 18:39:08 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1289329    
Attachments:
Description Flags
mrt.log which causes the error
none
wlms incorrectly reported as being binary
none
mrt.log incorrectly reported as bein binary
none
UTF Error none

Description Christian Jung 2015-05-13 11:39:55 UTC
Description of problem:
I configured SmartState Analysis in CF and extended the list of included files to c:\windows\debug\* and check the "Collect contents" check box.

After running a SSA the following files are retrieved:
mrt.log
passwd.log
sammui.log
wlms.log

I can open all files except mrt.log. When opening this file to view in the CF web UI, the following error is shown:
incompatible encoding regexp match (UTF-8 regexp with ASCII-8BIT string) [vm_infra/filesystems] 

Version-Release number of selected component (if applicable):
cfme-5.3.4.2-1.el6cf.x86_64

How reproducible:
always for specific files

Steps to Reproduce:
1. configure SSA profile to include c:\windows\debug\* 
2. run SSA
3. view the content of mrt.log

Actual results:
error

Expected results:
view content of file

Additional info:
I tested this with Windows 7 (all updates installed) and Windows 2012 R2 (all updates installed). Both versions of Windows were English.

Note that mrt.log only exists after the Microsoft MRT Tool was running at least once. Other files with similar encoding should show similar results though.

This might also be helpful:
mrt.log:    Little-endian UTF-16 Unicode text, with CRLF, CR, LF line terminators
PASSWD.LOG: empty
sammui.log: ASCII text, with CRLF line terminators
wlms.log:   ASCII text, with CRLF line terminators

Comment 1 Christian Jung 2015-05-13 11:40:44 UTC
Created attachment 1025020 [details]
mrt.log which causes the error

Comment 4 Jason Frey 2015-05-14 14:18:11 UTC
In summary, UTF-16 file contents can't be shown in the UI and give an error about UTF8 conversion.

Comment 6 Martin Povolny 2015-09-07 09:59:47 UTC
Christian, we have changed the logic around displaying these files in upstream.

As the target release is now 5.5, the upstream code is, what is relevant now.

Can you, please, verify that this bug is present in upstream?

Thx!

Comment 7 Christian Jung 2015-09-07 15:07:15 UTC
HiHo,
I tested this again today with Botvinik. Botvinik falsely reports all files as being binary and unable to view.

Even files which were shown fine with 5.3 are now reported as being binary (which is not correct). 

Regards,
Christian

Comment 8 Christian Jung 2015-09-07 15:08:00 UTC
Created attachment 1071051 [details]
wlms incorrectly reported as being binary

Comment 9 Christian Jung 2015-09-07 15:08:44 UTC
Created attachment 1071053 [details]
mrt.log incorrectly reported as bein binary

Comment 10 Martin Povolny 2015-11-05 15:58:27 UTC
So we have 2 options here I'd say

1) try to do someting smart like guess the encoding, try to convert the file to utf8 to display it

2) offer a download link for files that we cannot display

I don't think that 1) is a case for a simple fix.

Would 2) work for you?

Comment 12 Milan Zázrivec 2015-11-23 16:33:55 UTC
https://github.com/ManageIQ/manageiq/pull/5568

Comment 13 CFME Bot 2015-11-24 03:20:18 UTC
New commit detected on ManageIQ/manageiq/master:
https://github.com/ManageIQ/manageiq/commit/56f5ac2c74f1550d7af9d8e68ec9e87f287d34d1

commit 56f5ac2c74f1550d7af9d8e68ec9e87f287d34d1
Author:     Milan Zazrivec <mzazrivec>
AuthorDate: Mon Nov 23 17:30:56 2015 +0100
Commit:     Milan Zazrivec <mzazrivec>
CommitDate: Mon Nov 23 18:18:05 2015 +0100

    Add download link for filesystem data
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1221149

 app/controllers/vm_common.rb               |  5 +++++
 app/views/layouts/_item.html.haml          |  2 ++
 config/routes.rb                           |  2 ++
 spec/factories/miq_report.rb               | 11 +++++++++++
 spec/routing/vm_cloud_routing_spec.rb      |  6 ++++++
 spec/routing/vm_infra_routing_spec.rb      |  6 ++++++
 spec/views/layouts/_item.html.haml_spec.rb | 22 ++++++++++++++++++++++
 7 files changed, 54 insertions(+)
 create mode 100644 spec/views/layouts/_item.html.haml_spec.rb

Comment 14 Martin Povolny 2015-11-24 10:06:06 UTC
*** Bug 1222508 has been marked as a duplicate of this bug. ***

Comment 16 CFME Bot 2015-12-07 22:42:29 UTC
New commit detected on cfme/5.5.z:
https://code.engineering.redhat.com/gerrit/gitweb?p=cfme.git;a=commitdiff;h=f9a50ec9c42b721af115d107c49c2a190b5609f5

commit f9a50ec9c42b721af115d107c49c2a190b5609f5
Author:     Milan Zazrivec <mzazrivec>
AuthorDate: Mon Nov 23 17:30:56 2015 +0100
Commit:     Milan Zazrivec <mzazrivec>
CommitDate: Tue Nov 24 13:30:19 2015 +0100

    Add download link for filesystem data
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1221149
    (cherry picked from commit 56f5ac2c74f1550d7af9d8e68ec9e87f287d34d1)

 app/controllers/vm_common.rb               |  5 +++++
 app/views/layouts/_item.html.haml          |  2 ++
 config/routes.rb                           |  2 ++
 spec/factories/miq_report.rb               | 11 +++++++++++
 spec/routing/vm_cloud_routing_spec.rb      |  6 ++++++
 spec/routing/vm_infra_routing_spec.rb      |  6 ++++++
 spec/views/layouts/_item.html.haml_spec.rb | 22 ++++++++++++++++++++++
 7 files changed, 54 insertions(+)
 create mode 100644 spec/views/layouts/_item.html.haml_spec.rb

Comment 17 CFME Bot 2015-12-07 22:42:35 UTC
New commit detected on cfme/5.5.z:
https://code.engineering.redhat.com/gerrit/gitweb?p=cfme.git;a=commitdiff;h=6077f611821d0ebd8c7a15234546747d6d6d74ed

commit 6077f611821d0ebd8c7a15234546747d6d6d74ed
Merge: 539830f f9a50ec
Author:     Dan Clarizio <dclarizi>
AuthorDate: Mon Dec 7 17:40:00 2015 -0500
Commit:     Dan Clarizio <dclarizi>
CommitDate: Mon Dec 7 17:40:00 2015 -0500

    Merge branch 'filesystem_download_link' into '5.5.z'
    
    Add download link for filesystem data
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1221149
    
    Clean cherry pick from commit 56f5ac2c74f1550d7af9d8e68ec9e87f287d34d1
    
    See merge request !529

 app/controllers/vm_common.rb               |  5 +++++
 app/views/layouts/_item.html.haml          |  2 ++
 config/routes.rb                           |  2 ++
 spec/factories/miq_report.rb               | 11 +++++++++++
 spec/routing/vm_cloud_routing_spec.rb      |  6 ++++++
 spec/routing/vm_infra_routing_spec.rb      |  6 ++++++
 spec/views/layouts/_item.html.haml_spec.rb | 22 ++++++++++++++++++++++
 7 files changed, 54 insertions(+)

Comment 20 Ramesh A 2016-01-20 11:54:03 UTC
Forgot to mention, it happens only for mrt.log.  For other logs - passwd.log, sammui.log, download button is available.

Comment 21 Ramesh A 2016-01-20 11:55:05 UTC
Created attachment 1116628 [details]
UTF Error

Comment 22 Ramesh A 2016-01-20 11:55:45 UTC
Verified in 5.5.2.1.20160114044944_395c086

Comment 24 Milan Zázrivec 2016-01-21 14:55:20 UTC
https://github.com/ManageIQ/manageiq/pull/6282

Comment 26 CFME Bot 2016-02-10 19:00:45 UTC
New commit detected on cfme/5.5.z:
https://code.engineering.redhat.com/gerrit/gitweb?p=cfme.git;a=commitdiff;h=d062d845f43351b57cee6c124866407ab7f53bdb

commit d062d845f43351b57cee6c124866407ab7f53bdb
Author:     Milan Zazrivec <mzazrivec>
AuthorDate: Thu Jan 21 15:26:10 2016 +0100
Commit:     Milan Zazrivec <mzazrivec>
CommitDate: Tue Jan 26 11:48:48 2016 +0100

    Fix rendering of UTF-16 blobs by conversion to UTF-8
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1221149
    (cherry picked from commit 1cf749c1942840e2dd1331a8b71fa0c826fb1162)

 app/helpers/ui_constants.rb       |  3 +++
 app/models/filesystem.rb          | 10 ++++++++++
 app/views/layouts/_item.html.haml |  2 +-
 spec/models/filesystem_spec.rb    | 19 +++++++++++++++++++
 4 files changed, 33 insertions(+), 1 deletion(-)

Comment 27 CFME Bot 2016-02-10 19:01:24 UTC
New commit detected on cfme/5.5.z:
https://code.engineering.redhat.com/gerrit/gitweb?p=cfme.git;a=commitdiff;h=a984ddac1a4866e282ef2b79090faa5d354b4fd8

commit a984ddac1a4866e282ef2b79090faa5d354b4fd8
Merge: b5b0670 d062d84
Author:     Dan Clarizio <dclarizi>
AuthorDate: Wed Feb 10 13:44:17 2016 -0500
Commit:     Dan Clarizio <dclarizi>
CommitDate: Wed Feb 10 13:44:17 2016 -0500

    Merge branch 'fix_displaying_of_utf16_blobs' into '5.5.z'
    
    Fix rendering of UTF-16 blobs by conversion to UTF-8
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1221149
    (cleanly cherry picked from commit 1cf749c1942840e2dd1331a8b71fa0c826fb1162)
    
    See merge request !748

 app/helpers/ui_constants.rb       |  3 +++
 app/models/filesystem.rb          | 10 ++++++++++
 app/views/layouts/_item.html.haml |  2 +-
 spec/models/filesystem_spec.rb    | 19 +++++++++++++++++++
 4 files changed, 33 insertions(+), 1 deletion(-)

Comment 29 Ramesh A 2016-04-01 04:24:10 UTC
Good to go.  Verified and working fine in 5.5.3.2.20160318152106_92d2d67.

Now CFME appliance displays the mrt.log file contents and download button is also available.  Clicking on the download button, downloads the file correctly.

Comment 31 errata-xmlrpc 2016-04-13 18:39:08 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:0616

Comment 32 Ramesh A 2016-08-02 07:04:04 UTC
Test case added : RHCF3-11385 -  test_ssa_files[windows-utf-8-files]