Bug 1465761 - Refresh failed for VMware Provider in Cloudforms 4.5
Summary: Refresh failed for VMware Provider in Cloudforms 4.5
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat CloudForms Management Engine
Classification: Red Hat
Component: Providers
Version: 5.8.0
Hardware: All
OS: Linux
high
high
Target Milestone: GA
: 5.9.0
Assignee: James Wong
QA Contact: Leo Khomenko
URL:
Whiteboard: vmware:ems_refresh
Depends On:
Blocks: 1477727 1479976
TreeView+ depends on / blocked
 
Reported: 2017-06-28 07:07 UTC by Neha Chugh
Modified: 2020-09-10 10:47 UTC (History)
13 users (show)

Fixed In Version: 5.9.0.1
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1477727 1479976 (view as bug list)
Environment:
Last Closed: 2018-03-06 15:11:26 UTC
Category: ---
Cloudforms Team: VMware
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
MOB screen shot (226.75 KB, image/png)
2017-07-05 12:27 UTC, James Wong
no flags Details
disk detail (26.98 KB, image/png)
2017-07-07 06:02 UTC, Neha Chugh
no flags Details
mob output (31.58 KB, image/png)
2017-07-07 06:02 UTC, Neha Chugh
no flags Details
disk1 detail (26.97 KB, image/png)
2017-07-07 06:04 UTC, Neha Chugh
no flags Details
inconsistent hard disk (28.63 KB, image/png)
2017-07-24 04:57 UTC, Neha Chugh
no flags Details

Description Neha Chugh 2017-06-28 07:07:10 UTC
Description of problem:

While doing the manual refresh of the Provider, it failed with below exception:

[----] E, [2017-06-26T15:28:24.832907 #29096:1311130] ERROR -- : MIQ(ManageIQ::Providers::Vmware::InfraManager::Refresher#refresh) EMS: [vCenter-DR], id: [2000000000007] Unable to perform refresh for the following targets:
[----] E, [2017-06-26T15:28:24.833036 #29096:1311130] ERROR -- : MIQ(ManageIQ::Providers::Vmware::InfraManager::Refresher#refresh)  --- ManageIQ::Providers::Vmware::InfraManager [vCenter-DR] id [2000000000007]
[----] I, [2017-06-26T15:28:24.864483 #29096:1311130]  INFO -- : MIQ(ManageIQ::Providers::Vmware::InfraManager::Refresher#refresh) Refreshing all targets...Complete
[----] E, [2017-06-26T15:28:24.869871 #29096:1311130] ERROR -- : [EmsRefresh::Refreshers::EmsRefresherMixin::PartialRefreshError]: undefined method `xsiType' for nil:NilClass  Method:[rescue in deliver]
[----] E, [2017-06-26T15:28:24.872356 #29096:1311130] ERROR -- : /var/www/miq/vmdb/app/models/ems_refresh/refreshers/ems_refresher_mixin.rb:50:in `refresh'
/var/www/miq/vmdb/app/models/manageiq/providers/base_manager/refresher.rb:9:in `refresh'
/var/www/miq/vmdb/app/models/ems_refresh.rb:98:in `block in refresh'
/var/www/miq/vmdb/app/models/ems_refresh.rb:97:in `each'
/var/www/miq/vmdb/app/models/ems_refresh.rb:97:in `refresh'

Version-Release number of selected component (if applicable):

5.8.0

Actual results:

Refresh is failing with exception undefined method `xsiType' for nil:NilClass  Method:[rescue in deliver]

Expected results:
Refresh should be done successfully.

Additional info:

Increased the memory threshold from 2GB to 4 GB of vim_broker_worker but still didn't work.
Seems like the target refresh is failing as full refresh is completed above it.

Comment 3 Adam Grare 2017-06-29 20:39:32 UTC
From the logs it looks like the error is:

[----] E, [2017-06-22T12:17:18.852473 #26538:1311130] ERROR -- : [NoMethodError]: undefined method `xsiType' for nil:NilClass  Method:[rescue in block in refresh]
[----] E, [2017-06-22T12:17:18.852641 #26538:1311130] ERROR -- : /opt/rh/cfme-gemset/bundler/gems/manageiq-providers-vmware-30e6243bf993/app/models/manageiq/providers/vmware/infra_manager/refresh_parser.rb:1074:
/opt/rh/cfme-gemset/bundler/gems/manageiq-providers-vmware-30e6243bf993/app/models/manageiq/providers/vmware/infra_manager/refresh_parser.rb:1062:in `each'
/opt/rh/cfme-gemset/bundler/gems/manageiq-providers-vmware-30e6243bf993/app/models/manageiq/providers/vmware/infra_manager/refresh_parser.rb:1062:in `vm_inv_to_disk_hashes'

This is coming from https://github.com/ManageIQ/manageiq-providers-vmware/blob/fine/app/models/manageiq/providers/vmware/infra_manager/refresh_parser.rb#L1074

It looks to me like there is a VM that has a disk assigned to a deleted controller but we will need to see the managed object browser to confirm.

Comment 11 James Wong 2017-07-05 12:27:11 UTC
Created attachment 1294568 [details]
MOB screen shot

Comment 13 Neha Chugh 2017-07-07 06:02:02 UTC
Created attachment 1295178 [details]
disk detail

Comment 14 Neha Chugh 2017-07-07 06:02:33 UTC
Created attachment 1295179 [details]
mob output

Comment 15 Neha Chugh 2017-07-07 06:04:16 UTC
Created attachment 1295180 [details]
disk1 detail

Comment 25 James Wong 2017-07-17 13:15:49 UTC
Hi Chen,

Don't know how such a vm is created. vSphere client doesn't seem to allow that to be created.  Can you request customer to inspect this vm and see how they can fix it, e.g. adding controller to the disks or removing the disks?  And then rerun the refresh.

We probably can't do much with such disks without controller key.  What we can do to fix this error is to skip them. 

regards,
James

Comment 26 Neha Chugh 2017-07-24 04:53:52 UTC
Hello James,

Customer is unable to remove the disk as the disk is in inconsistent state( as per attached). Let me know if you have any other suggestion so to rectify this issue.

Regards,
Neha Chugh

Comment 27 Neha Chugh 2017-07-24 04:57:22 UTC
Created attachment 1303436 [details]
inconsistent hard disk

Comment 28 James Wong 2017-07-24 12:55:10 UTC
Hi Neha,

The customer probably has to contact VMware support for fixing the vm.

If they are savvy enough, they may try to de-register the VM and then edit the vm file (in the datastore) to remove the disk.  But they should consult VMware support first.

James

Comment 30 James Wong 2017-07-31 18:07:12 UTC
Hi Neha,

Any update from the customer?  

What we are doing to the code is to inventory these disks as having 'unknown' controller type.

regards,
James

Comment 33 Neha Chugh 2017-08-01 13:09:33 UTC
Hello James,

We haven't received any response from customer till date whether he is able to remove the disk or not so we can lower down the severity.

In case if he is not able to delete the disk we can suggest the provided PR.

Thanks for all help.

Regards,
Neha Chugh

Comment 34 CFME Bot 2017-08-01 16:08:48 UTC
New commit detected on ManageIQ/manageiq-providers-vmware/master:
https://github.com/ManageIQ/manageiq-providers-vmware/commit/b5f1089beb71f98026f22e8352383ecf83a04593

commit b5f1089beb71f98026f22e8352383ecf83a04593
Author:     James Wong <jwong>
AuthorDate: Tue Jul 18 09:54:46 2017 -0400
Commit:     James Wong <jwong>
CommitDate: Tue Aug 1 11:47:55 2017 -0400

    dealing with disks which have no controller key
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1465761

 .../providers/vmware/infra_manager/refresh_parser.rb    | 17 +++++++++++------
 .../providers/vmware/infra_manager/refresher_spec.rb    | 15 +++++++++++++++
 .../vim_data/miq_vim_inventory/virtualMachinesByMor.yml | 10 +---------
 3 files changed, 27 insertions(+), 15 deletions(-)


Note You need to log in before you can comment on or make changes to this bug.