Bug 1810217 - Building a Migration Analytics payload with a SSA profile including a file with non ASCII chars fails
Summary: Building a Migration Analytics payload with a SSA profile including a file wi...
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat CloudForms Management Engine
Classification: Red Hat
Component: Migration Analytics
Version: 5.11.4
Hardware: Unspecified
OS: Unspecified
high
medium
Target Milestone: GA
: cfme-future
Assignee: Mike Turley
QA Contact: Ganesh Hubale
Red Hat CloudForms Documentation
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-03-04 17:57 UTC by Miguel Perez Colino
Modified: 2020-09-17 19:31 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-08-18 22:41:08 UTC
Category: Bug
Cloudforms Team: V2V
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
Screenshot (18.22 KB, image/png)
2020-03-04 17:57 UTC, Miguel Perez Colino
no flags Details

Description Miguel Perez Colino 2020-03-04 17:57:32 UTC
Created attachment 1667558 [details]
Screenshot

Description of problem:
Building a Migration Analytics payload with a SSA profile including a file with non ASCII chars fails

Version-Release number of selected component (if applicable):
CF 5.0.4 (CFME 5.11.4)

How reproducible:
Add a file to SSA profile that includes weird characters.
Sample file used `/usr/java/latest/COPYRIGHT` of Oracle jdk 1.8.0

Steps to Reproduce:
1. Install Oracle JDK 1.8.0 to a VM
2. Add `/usr/java/latest/COPYRIGHT` file (with content) to the SSA default profile
3. Run SSA
4. Go to Migration Analytics plugin and try to obtain a payload.

Actual results:
Error received ... `Failed to collect inventory data, "\xA9" from ASCII-8BIT to UTF-8`

Expected results:
Page with download link to obtain payload file


Additional info:

Comment 2 Mike Turley 2020-03-05 14:13:04 UTC
Miguel, do you have a CF appliance running that I can access with this issue present? I don't have access to a real VMware environment right now. Does the error appear immediately or does it take some time? If I can look at the appliance I can see if the error is coming from the initial request to start data collection or the task that is updated during data collection.

This is definitely an issue on the backend, though. The data collection happens as a result of this request (where 1,2,3 are the CF ids corresponding to the VMware providers):
POST /api/red_hat_migration_analytics
Body: { "action": "bundle", "provider_ids": ["1", "2", "3"] }

The response will either directly contain the error message or it will contain a `task_href` which can be used to fetch the bundle task, which will contain the error message.

This API request is handled on the backend by this code (from Drew and Adam): https://github.com/RedHatCloudForms/cfme-migration_analytics/blame/master/app/controllers/api/red_hat_migration_analytics_controller.rb#L16
And the bundle itself is handled by this code (from Adam): https://github.com/RedHatCloudForms/cfme-cloud_services/blame/master/lib/cfme/cloud_services/inventory_sync.rb#L38

So the issue is probably somewhere in there. Do either of you guys know what might cause this?

Comment 7 Mike Turley 2020-03-17 20:14:36 UTC
Adam, do you know why this file would be causing `Encoding::UndefinedConversionError ("\xA9" from ASCII-8BIT to UTF-8)`? I'm not too familiar with how ASCII and UTF work in ruby/rails, but from what I can find, \xA9 (169) in ASCII is the ® symbol which isn't present in this file.

Would you mind if I assign this one to you? (or Miguel, who else do you think it should be assigned to? Maybe Peter?) I'm not sure where to start here.

Comment 8 Mike Turley 2020-03-17 21:29:46 UTC
Hey Jerry, Adam advised me to assign this to you. Let me know if I can help.

Comment 9 Jerry Keselman 2020-03-24 19:30:22 UTC
I've just looked at the file in question on the VM in question, and collected it via SSA and downloaded the collected file to my own laptop.  The checksums of the files match.  The files are identical.  I can see the *copyright* symbol in both files.  IF there is an issue it is not with SSA.  Reassigning back to Mike.

Comment 10 Mike Turley 2020-03-24 21:49:16 UTC
Jerry, where in that yml file are you seeing the *copyright* symbol? I'm searching for it and not seeing it... I'm sure I'm just missing it though, because it's clearly causing the error here, just curious for my own sanity :)

I don't have a lot of Ruby experience and I don't know much about the code in question, but based on https://stackoverflow.com/a/13008475 it seems like maybe we need to call `force_encoding` before we call `to_json` here: https://github.com/RedHatCloudForms/cfme-cloud_services/blob/master/lib/cfme/cloud_services/data_packager.rb#L9
I'm not sure which ASCII encoding we would need to specify though. I could try to draft a PR, but I'd like to run this by someone who might catch something I'm not seeing here. Not sure who that would be.

Comment 11 Mike Turley 2020-03-24 22:03:15 UTC
Hey Peter, sorry to drop this in your lap but Brett told me you may be a good resource to help with this, as you're well versed in Ruby. Do you know the right way to fix this `Encoding::UndefinedConversionError`?

Comment 14 Mike Turley 2020-08-18 17:37:39 UTC
Hi Brett,

Sorry I lost track of this. I determined it was a backend issue, but we never found someone with that domain knowledge to reassign it to. We should assign this to someone who can troubleshoot the Ruby error detailed above (I tracked it down to this line: https://github.com/RedHatCloudForms/cfme-cloud_services/blob/master/lib/cfme/cloud_services/data_packager.rb#L9). I tried to figure it out and gave up, but I could give it another shot.

Comment 16 Peter McGowan 2020-08-18 20:34:16 UTC
I think we can close this as we wouldn't expect SmartState to be able to read and display the contents of a binary file anyway.

Comment 17 Brett Thurber 2020-08-18 22:41:08 UTC
Closing per Peter's recommend.


Note You need to log in before you can comment on or make changes to this bug.