Bug 1387336

Hi Felix, from the description it sounds like it is mostly folders/datacenters that get cleared...do you know if all VMs and Hosts still in inventory?

Also the summary states specifically "wait until a refresh is triggered by a reconfigure", do you know if only reconfigure refreshes cause this?  Up until 5.7 reconfigure refresh is actually different code than normal refreshes.

The inventory returned from the broker is very consistent (the exact same number of VMs/Hosts/Folders/etc... for all refreshes) so this looks like an issue in either parsing, saving, or linking inventory.

There are a number of cases in the logs where link_ems_inventory shows prev_relats: {} and :ext_management_systems_to_folders tends to cycle between:
:ext_management_systems_to_folders=>{1000000000004=>[1000000000012]}
and
:ext_management_systems_to_folders=>{1000000000004=>[1000000000011]}

Looking up from an occurrence of prev_relats: {} I see:
MIQ(EmsRefresh.save_vms_inventory) EMS: [vCenter], id: [1000000000004] Duplicate unique values found: ["4231ea77-efcb-1408-9bc5-90200ab24bf4"]

Right after this the top-level folder for the EMS changes
prev_relats: {:folders_to_vms=>{1000000000121=>[1000000012301]}, :folders_to_folders=>{1000000000012=>[1000000000121], 1000000000117=>[1000000000012], 1000000000011=>[1000000000117]}, :ext_management_systems_to_folders=>{1000000000004=>[1000000000011]}}
new_relats:  {:folders_to_vms=>{1000000000121=>[1000000012301]}, :folders_to_folders=>{1000000000012=>[1000000000121]}, :ext_management_systems_to_folders=>{1000000000004=>[1000000000012]}}

I think the duplicate values are causing save_inventory to update the wrong object and getting an invalid folder tree, clearing valid folders.

Hi Adam, I added a log for the failing appliance , and linked case to this BZ . 
I see your comment 13, that log is same  like I linked ?

I think you're right about the permissions causing it, I was able to reproduce with the following setup:

1. Create a user for MIQ (miq_svc)
2. Give user admin access to vc and propagate to all children
3. Have a datacenter with two clusters
4. Set the role on a cluster to "No Access" and propagate to all children of the cluster
5. Run a full refresh and confirm 'ems_infra/1?display=ems_folders&vat=true' shows all folders
6. Run a targeted refresh of a VM in the cluster with no access (e.g. reconfigure the vm)
7. go to 'ems_infra/1?display=ems_folders&vat=true', all folders will be gone

https://github.com/ManageIQ/manageiq/pull/12222

New commit detected on ManageIQ/manageiq/master:
https://github.com/ManageIQ/manageiq/commit/39fb6ce4c02e47030423da1abb7ee47b89e77f06

commit 39fb6ce4c02e47030423da1abb7ee47b89e77f06
Author:     Adam Grare <agrare>
AuthorDate: Wed Oct 26 14:43:21 2016 -0400
Commit:     Adam Grare <agrare>
CommitDate: Wed Nov 2 09:05:55 2016 -0400

    Don't stop at a datacenter when filtering vm inv
    
    When doing targeted refresh of a VM, if the VM's host isn't in inventory
    then the folders above datacenter are not returned.  This causes
    link_ems_inventory to set a new root folder for the EMS and clears all
    other folder relationships.
    
    This can happen if the user doesn't have permission to access the
    cluster/host the VM is running on but does have access to the folder the
    VM is in.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1387336

 .../manageiq/providers/vmware/infra_manager/refresh_parser/filter.rb  | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

Customer reported the issue on 5.6 and is requesting that this be backported to 5.6.z