Bug 1528419 - Smart State Analysis Profiles Unable to Capture File Contents
Summary: Smart State Analysis Profiles Unable to Capture File Contents
Keywords:
Status: CLOSED DUPLICATE of bug 1553808
Alias: None
Product: Red Hat CloudForms Management Engine
Classification: Red Hat
Component: SmartState Analysis
Version: 5.8.0
Hardware: All
OS: All
high
high
Target Milestone: GA
: 5.9.3
Assignee: Jerry Keselman
QA Contact: Satyajit Bulage
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-12-21 19:45 UTC by Dustin Scott
Modified: 2021-09-09 12:58 UTC (History)
12 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-04-20 12:25:07 UTC
Category: ---
Cloudforms Team: ---
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Dustin Scott 2017-12-21 19:45:45 UTC
Description of problem:

When configuring Smart State Analysis Profiles to capture file contents, configuring a control policy to apply the profile, and running Smart State Analysis, everything is captured besides the specified files from the Analysis Profile.

This works in version 5.7.1.3.20170221135006_818f133
This is broke in version cfme-5.7.3.2-1.el7cf.x86_64 and later (including up to 5.9.0-12 beta).  Discovered during the 4.6 test-a-thon and in version 5.8.2.

This is for Microsoft Azure AND VMware providers (VDDK 6.0.2 used).  Both do not work.


Version-Release number of selected component (if applicable):

5.7.3.2-1 thru 5.9.0-12.  Discovered in the field in version 5.8.2.


How reproducible:

100% - Cannot pull in file contents


Steps to Reproduce:
1. See https://access.redhat.com/documentation/en-us/red_hat_cloudforms/4.5/html/assigning_a_custom_analysis_profile_to_a_virtual_machine/ for specific steps to reproduce


Actual results:

File contents are not retrieved.  Files displays 0 in Infrastructure > VMs view.


Expected results:

File contents are listed and displayed.


Additional info:

Comment 2 Jerome Marc 2017-12-21 19:48:39 UTC
Experienced the same issue with Smart-State Analysis on AWS EC2 on CFME3.9.0.13.

Comment 3 Jerome Marc 2017-12-21 20:06:34 UTC
File content is captured for SmartState on VMware for:
- 5.8.2.3.20171016155816_aaec796
- 5.9.0.14.20171219165537_e3dea29

But no file at all for AWS EC2 SmartState on:
- 5.9.0.14.20171219165537_e3dea29

Comment 5 Adam Goossens 2018-02-20 00:02:33 UTC
Also occurs with an OpenStack provider, 5.7.4.2.

Unsure if this is related, but it seems that the analysis profile either isn't applied, or isn't respected: in addition to ignoring the files to collect, it seems SSA is also gathering information on categories I have left unchecked in my analysis profile.

E.g., I will have all categories unchecked, but will still see a line such as:

"MIQ(ManageIQ::Providers::Openstack::CloudManager::Vm#scan_metadata) category=[["vmconfig", "accounts", "software", "services", "system"]] [Array]

A policy simulation on a target VM shows that my chosen analysis profile should be applied.

I can also see in the MiqQueue.put for the smartproxy scan_metadata command, the "vmScanProfiles" argument is an empty list.

Comment 10 Jerry Keselman 2018-02-26 18:46:36 UTC
Still waiting for info about where this doesn't work, possibly with logs and/or access to the offending appliance(s).  I have successfully tested with on VMware, Azure, and AWS.

Comment 11 Adam Goossens 2018-03-05 19:11:19 UTC
Apologies; I have been travelling for work and will return next week. Access to the appliances won't be possible, but I can get you a copy of the evm.log on the smartproxy and the worker that conducted the analysis. I'll have the appliances set to debug first, run the scan, and provide you the result.

I will look to take care of this next week when I'm back on site.

I see that Jerome has also been able to duplicate this issue; are you able to source logs from him as well?

Comment 12 Jerry Keselman 2018-03-05 19:56:28 UTC
Thanks, Adam.  Looking forward to connecting to get that info next week.  Jerome do you have access to logs showing this issue?  Alternately, access to the offending VMs would and/or appliances would be incredibly useful.

Comment 13 Jerome Marc 2018-03-05 23:13:10 UTC
This doesn't seem to be an issue anymore as I can see files for SSA on EC2 with build 5.9.0.22.20180221205805_f93a675.  Can be closed.

Comment 14 Jerry Keselman 2018-03-05 23:17:09 UTC
Ok. Closing.

Comment 16 Jerry Keselman 2018-03-06 14:21:45 UTC
Have you named the analysis profile "default"?  I believe that is required although it may be missing from the documentation.

Comment 17 Adam Goossens 2018-03-20 02:56:01 UTC
Hi Jerry,

Apologies again for the delayed reply. I can confirm that labelling the analysis profile "default" does cause it to be picked up and applied, at least as of 5.7.4.2.

IMO that's a bug, as (unless I've misunderstood somewhere) it means I can't have multiple analysis profiles assigned separately within the same region - only one of those profiles ('default') will actually be honoured by SSA. 

I haven't tested it but I imagine that under that model I couldn't, for example, have different analysis profiles for different providers.

If there isn't already one, I'll raise a BZ for the latter issue for consideration.

Comment 18 Jerry Keselman 2018-03-20 17:19:10 UTC
Adam,

You actually can have more than one profile.  There is one *default* profile.  You can have other profiles named whatever you want but you need to assign them to a policy.  You can have as many policies as you want.  You can have a policy for each VM if thats what you want.  There is no reason to raise another BZ.

Hope this helps.

Comment 19 Adam Goossens 2018-03-22 21:30:58 UTC
Hi Jerry,

Thanks for the additional context. In this case I wasn't making myself particularly clear; I apologise. I do have a policy profile set at the provider level, with control policies within that assign my custom analysis profile in response to the VM Analysis Start event. I've also attempted setting this policy at the VM level. No scopes are applied anywhere.

Forgive the ASCII art; it looks something like the below:

- My Policy Profile
|- VM and Instance Control: Scan VMs for Compliance
|-- VM Analysis Start
|----Use the ACME Inc Analysis Profile
|
|- VM and Instance Control: Check compliance after scan
|-- VM Analysis Complete
|--- Check Host or VM Compliance

Where "ACME Inc" is the name of my Analysis profile as found under Configuration.  There are also a number of compliance checks within that same policy profile.

My "ACME Inc" analysis profile will not be applied in response to the VM Analysis Start event (have also tried VM Analysis Request, no change) *unless* I follow your comment above: re-name that policy to "default" and adjust the action for VM Analysis Start to match. Then it is applied correctly.

Interestingly, if I have one analysis profile named "default", and another named "ACME Inc", then assign ACME Inc to a VM for analysis (via the policy profile), the "default" analysis profile will still be chosen.

To try and eliminate user error, I followed the documentation steps with the same result: https://access.redhat.com/documentation/en-us/red_hat_cloudforms/4.2/html/assigning_a_custom_analysis_profile_to_a_virtual_machine/index

Comment 23 Jerry Keselman 2018-04-17 13:18:21 UTC
Imaan, I'd prefer a new bug.  I have reproduced your issue.  It is specific to profiles other than the default - it seems that the default profile is applied regardless.

Comment 24 Jerry Keselman 2018-04-17 14:28:48 UTC
Actually please hold off on that separate bug.  If you look in the policy.log, there should be entries with the name of the policy in question that explain why the policy wasn't applied - for instance if the condition evaluated to false.  Can you provide the contents of the log from the time period during your scan request to see if we can debug this further?  Thanks.

Comment 25 Adam Goossens 2018-04-19 05:00:40 UTC
Hey Jerry,

Getting the full log out might take time (disconnected environment) but I have transcribed the relevant lines from policy.log below (the surrounding lines are just noise regarding worker start/stop/restart):

Unless I am misreading, it looks like the policy is successful:

...MIQ(action-invoke) Invoking action [Use the ACME Inc Compliance Analysis Profile] for successful policy [Scan VMs for Compliance], event: [VM Analysis Start], entity name: [test-ssa-vm], entity type: [Instance (OpenStack)], sequence: [1], synchronous? [true]

...MIQ(action_assign_scan_profile): Action [Use the ACME Inc Compliance Analysis Profile], using analysis profile: [Gathers data required for ISM analysis]

What else can I get you?

Cheers
Adam

Comment 26 Jerry Keselman 2018-04-19 07:12:59 UTC
Adam,

Thanks, that's perfect.  I've gotten the same results and just wanted to make sure that was what you were seeing to.  I have figured out the issue in the code, just not the solution yet. Working on it.

Comment 27 Hui Song 2018-04-20 12:25:07 UTC
The issue is due to profiles information not saved in job options during job state transfer. Will fix this in https://bugzilla.redhat.com/show_bug.cgi?id=1553808. Mark this as duplicate.

*** This bug has been marked as a duplicate of bug 1553808 ***


Note You need to log in before you can comment on or make changes to this bug.