Bug 1529725

Summary: SSA: Datastores: Get SmartState Analysis for 1 storages complete (1 in Error) for some datastores types
Product: Red Hat CloudForms Management Engine Reporter: Ruslana Babyuk <rbabyuk>
Component: SmartState AnalysisAssignee: Adam Grare <agrare>
Status: CLOSED CURRENTRELEASE QA Contact: Ruslana Babyuk <rbabyuk>
Severity: high Docs Contact:
Priority: high    
Version: 5.8.0CC: cpelland, gblomqui, obarenbo, rbabyuk, saali, simaishi
Target Milestone: GAKeywords: TestOnly
Target Release: 5.10.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: ssa
Fixed In Version: 5.10.0.0 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1534753 (view as bug list) Environment:
Last Closed: 2018-06-21 20:47:16 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: CFME Core Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1534753    
Attachments:
Description Flags
ssa_datastores
none
log_file none

Description Ruslana Babyuk 2017-12-29 16:14:44 UTC
Description of problem:
Get SmartState Analysis for 1 storages complete (1 in Error)  for some datastores types. 

Version-Release number of selected component (if applicable):
5.8, 5.9

How reproducible:
100%

Steps to Reproduce:
1. Add infra provider(my case vsphere65)
2. Add creds for host
3. Run SSA for NFS datastore types(my case [NFS-Datastore-1])

Actual results:
Get SmartState Analysis for 1 storages complete (1 in Error)

Expected results:
SSA should run

Additional info:

Comment 4 Greg Blomquist 2018-01-03 17:20:10 UTC
Adam, can you see what's going on here?

Comment 6 Adam Grare 2018-01-03 18:34:12 UTC
I was able to scan an NFS datastore on our 6.5 vCenter:

[----] I, [2018-01-03T13:27:40.886370 #31457:2b0c9e4eaf54]  INFO -- : MIQ(Storage#smartstate_analysis) Storage [NFS Share] via Host [dell-r430-13.cloudforms.lab.eng.rdu2.redhat.com]...Starting
[----] I, [2018-01-03T13:32:25.115312 #31457:2b0c9e4eaf54]  INFO -- : MIQ(Storage#smartstate_analysis) Storage [NFS Share] via Host [dell-r430-13.cloudforms.lab.eng.rdu2.redhat.com]...Completed in [284.2289729] seconds

Comment 9 Ruslana Babyuk 2018-01-04 08:29:06 UTC
Created attachment 1376692 [details]
log_file

Comment 10 Adam Grare 2018-01-04 13:36:49 UTC
Thanks Ruslana, so the error was:

[----] E, [2018-01-04T03:13:24.492475 #13175:9d3140] ERROR -- : [MiqException::MiqVimResourceNotFound]: Could not find datastore: NFS-Datastore-1  Method:[block in method_missing]
[----] E, [2018-01-04T03:13:24.492659 #13175:9d3140] ERROR -- : /opt/rh/cfme-gemset/gems/vmware_web_service-0.2.5/lib/VMwareWebService/MiqVim.rb:181:in `block in getVimDataStore'
/opt/rh/rh-ruby23/root/usr/share/ruby/sync.rb:234:in `block in sync_synchronize'
/opt/rh/rh-ruby23/root/usr/share/ruby/sync.rb:231:in `handle_interrupt'
/opt/rh/rh-ruby23/root/usr/share/ruby/sync.rb:231:in `sync_synchronize'
/opt/rh/cfme-gemset/gems/vmware_web_service-0.2.5/lib/VMwareWebService/MiqVim.rb:180:in `getVimDataStore'

Comment 11 Adam Grare 2018-01-04 14:32:39 UTC
Okay the problem is that the name of the datastore you are trying to scan is different on the VC and the host we are trying to connect through.  On the VC it is called NFS-Datastore-1 but on the host it is called Datastore but we can confirm that they are the same storage because they have the same url:

irb(main):018:0> ems.connect.dataStores["NFS-Datastore-1"]["summary"]["url"]
=> "ds:///vmfs/volumes/af0b5a1f-764ab76d/"
irb(main):019:0> h.connect.dataStores["Datastore"]["summary"]["url"]
=> "/vmfs/volumes/af0b5a1f-764ab76d"

Comment 16 CFME Bot 2018-01-15 16:34:29 UTC
New commit detected on ManageIQ/manageiq-providers-vmware/master:
https://github.com/ManageIQ/manageiq-providers-vmware/commit/50f8330ee33f74df8ef9d68ef231ff9e34890633

commit 50f8330ee33f74df8ef9d68ef231ff9e34890633
Author:     Adam Grare <agrare>
AuthorDate: Thu Jan 4 13:53:15 2018 -0500
Commit:     Adam Grare <agrare>
CommitDate: Thu Jan 4 13:53:15 2018 -0500

    Refresh datastore files through EMS
    
    Refreshing datastore files just use the VIM SDK which can be done
    through the EMS not the Host.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1529725

 .../manageiq/providers/vmware/infra_manager.rb     | 28 ++++++++++++++++++++++
 .../providers/vmware/infra_manager/host.rb         | 27 ++-------------------
 2 files changed, 30 insertions(+), 25 deletions(-)

Comment 17 CFME Bot 2018-01-15 16:42:13 UTC
New commit detected on ManageIQ/manageiq/master:
https://github.com/ManageIQ/manageiq/commit/89297e95759bbc49f0a1f99f615ed700364514b4

commit 89297e95759bbc49f0a1f99f615ed700364514b4
Author:     Adam Grare <agrare>
AuthorDate: Thu Jan 4 13:45:52 2018 -0500
Commit:     Adam Grare <agrare>
CommitDate: Thu Jan 4 13:45:52 2018 -0500

    Scan Storage via EMS not Host
    
    Storages are currently scanned via a host which the user has added
    valid credentials for.  Storages are looked up by name which can be
    different between the vCenter (where the refresh is done) and the ESXi
    Host (where the scan is done).  If the name is different the scan will
    fail with the following error:
    `[MiqException::MiqVimResourceNotFound]: Could not find datastore: NFS-Datastore-1`
    
    Since datastore files are scanned via the VIM SDK there is no
    requirement on running the scan through the host.  We can scan the
    datastore through the EMS and remove the restriction on needing to add
    host credentials to be able to run storage smartstate.
    
    Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1529725

 app/models/storage.rb | 60 +++++++++++++++++++++++++++++----------------------
 config/settings.yml   |  2 +-
 2 files changed, 35 insertions(+), 27 deletions(-)

Comment 18 CFME Bot 2018-01-15 21:23:46 UTC
New commit detected on ManageIQ/manageiq-ui-classic/master:
https://github.com/ManageIQ/manageiq-ui-classic/commit/5dcd9e3e31b748cfd56eaa43257b5da9c806854f

commit 5dcd9e3e31b748cfd56eaa43257b5da9c806854f
Author:     Adam Grare <agrare>
AuthorDate: Thu Jan 4 13:59:05 2018 -0500
Commit:     Adam Grare <agrare>
CommitDate: Thu Jan 4 14:23:42 2018 -0500

    Scan Storage via EMS not Host
    
    Remove the requirement for hosts with valid credentials to be able to
    run storage smartstate.  The valid unsupported reasons will be moved to
    the Storage model.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1529725

 .../application_helper/button/storage_scan.rb      | 22 ---------------
 .../buttons/storage_scan_spec.rb                   | 32 ++++++++++------------
 2 files changed, 15 insertions(+), 39 deletions(-)

Comment 20 Adam Grare 2018-01-22 19:10:22 UTC
*** Bug 1438963 has been marked as a duplicate of this bug. ***

Comment 21 Adam Grare 2018-02-07 13:27:32 UTC
*** Bug 1438963 has been marked as a duplicate of this bug. ***