Bug 1284412 - v2v: Available VMs to import from VMware environment cannot be queried.
v2v: Available VMs to import from VMware environment cannot be queried.
Status: CLOSED CURRENTRELEASE
Product: ovirt-engine
Classification: oVirt
Component: BLL.Virt (Show other bugs)
3.6.0
Unspecified Unspecified
high Severity high (vote)
: ovirt-3.6.2
: 3.6.2
Assigned To: Shahar Havivi
Nisim Simsolo
:
Depends On: 1287681
Blocks: 952703 1236075
  Show dependency treegraph
 
Reported: 2015-11-23 04:56 EST by Nisim Simsolo
Modified: 2016-02-18 06:08 EST (History)
8 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: In some extreme cases we encounter that vSphere is not returning a VM by its ID when libvirt query it. This is a vSphere bug - in this case we log the VM that cannot be query and returns the rest of the VMs More information can be found in bz #1287681 Consequence: Fix: Result:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-02-18 06:08:17 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: Virt
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
rule-engine: ovirt‑3.6.z+
rule-engine: blocker+
mgoldboi: planning_ack+
michal.skrivanek: devel_ack+
mavital: testing_ack+


Attachments (Terms of Use)
engine.log (2.72 MB, text/plain)
2015-11-23 04:56 EST, Nisim Simsolo
no flags Details
vdsm.log (3.90 MB, text/plain)
2015-11-23 04:57 EST, Nisim Simsolo
no flags Details
Import page get stuck in "querying" state. (207.44 KB, image/png)
2015-11-23 04:58 EST, Nisim Simsolo
no flags Details
29th of November engine.log (3.06 MB, text/plain)
2015-11-29 07:37 EST, Nisim Simsolo
no flags Details
29th of November vdsm.log (2.66 MB, text/plain)
2015-11-29 07:38 EST, Nisim Simsolo
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
oVirt gerrit 49438 ovirt-3.6 MERGED v2v: add try/except to get_external_vms Never
oVirt gerrit 49507 master NEW v2v: add try/except to get_external_vms Never
oVirt gerrit 49583 master MERGED webadmin: improve robustness of import-VM dialog Never
oVirt gerrit 49632 ovirt-engine-3.6 MERGED webadmin: improve robustness of import-VM dialog Never
oVirt gerrit 50074 ovirt-engine-3.6.1 MERGED webadmin: improve robustness of import-VM dialog Never

  None (edit)
Description Nisim Simsolo 2015-11-23 04:56:10 EST
Description of problem:
During the very first step of import VM from VMware procedure, VDSErrorException occurs and webadmin remains in "loading" state instead of failing it with an appropriate message.

Version-Release number of selected component (if applicable):
rhevm-3.6.0.3-0.1.el6 (3.6.0-20)
libvirt-client-1.2.17-5.el7.x86_64
qemu-kvm-rhev-2.3.0-31.el7_2.1.x86_64
vdsm-4.17.10.1-0.el7ev.noarch
sanlock-3.2.4-1.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1. From webadmin, navigate to virtual machines tab -> import
2. Enter VMware environment components and credentials.
3. Click "load" button.

Actual results:
VMware available VMs are not queried at all due to VDSErrorException.
Also, webadmin remains in "loading" state.

Expected results:
Available VMs to import should be queried and listed in "virtual machines on source" pane.
In case of failure, webadmin should report it with an appropriate message.

Additional info:
engine and vdsm logs attached.
import page scrreenshot attached
Comment 1 Nisim Simsolo 2015-11-23 04:56 EST
Created attachment 1097604 [details]
engine.log
Comment 2 Nisim Simsolo 2015-11-23 04:57 EST
Created attachment 1097605 [details]
vdsm.log
Comment 3 Nisim Simsolo 2015-11-23 04:58 EST
Created attachment 1097606 [details]
Import page get stuck in "querying" state.
Comment 4 Nisim Simsolo 2015-11-23 05:01:02 EST
engine.log issue started at: 2015-11-23 11:34:22,631
vdsm.log issue started at: 2015-11-23 11:35:12,435
Comment 5 Yaniv Kaul 2015-11-24 03:24:16 EST
Nisim/Meital, did you look for the exact issue in the VDSM log?

Thread-148418::ERROR::2015-11-23 11:35:26,134::v2v::145::root::(get_external_vms) error connection to hypervisor: "internal error: HTTP response code 500 for call to 'Login'. Fault: ServerFaultCode - Cannot complete login due to an incorrect user name or password."

seems relevant.

The bug is probably that we don't treat this error correctly in the backend.
Comment 6 Nisim Simsolo 2015-11-29 06:16:53 EST
The ERROR in VDSM log retrieved when i tried to use ESXi credentials instead of vSphere credentails.

According to the next logs, connection actually succeeded:

- vdsm.log: 
Thread-185139::ERROR::2015-11-29 10:36:23,258::v2v::145::root::(get_external_vms) error connection to hypervisor: "internal error: Could not find compute resource specified in '/CFME/10.8.58.12'"

- engine.log: 
ERROR [org.ovirt.engine.core.bll.GetVmsFromExternalProviderQuery] (ajp-/127.0.0.1:8702-10) [] Exception: org.ovirt.engine.core.common.errors.EngineException: EngineE
xception: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to GetVmsFromExternalProviderVDS, error = internal error: Could not fin
d compute resource specified in '/CFME/10.8.58.12', code = 65 (Failed with error unexpected and code 16)
Comment 7 Yaniv Kaul 2015-11-29 06:43:07 EST
(In reply to Nisim Simsolo from comment #6)
> The ERROR in VDSM log retrieved when i tried to use ESXi credentials instead
> of vSphere credentails.
> 
> According to the next logs, connection actually succeeded:
> 
> - vdsm.log: 
> Thread-185139::ERROR::2015-11-29
> 10:36:23,258::v2v::145::root::(get_external_vms) error connection to
> hypervisor: "internal error: Could not find compute resource specified in
> '/CFME/10.8.58.12'"


There are no attached logs from the 29th of November.
Comment 8 Nisim Simsolo 2015-11-29 07:37 EST
Created attachment 1100175 [details]
29th of November engine.log
Comment 9 Nisim Simsolo 2015-11-29 07:38 EST
Created attachment 1100176 [details]
29th of November vdsm.log
Comment 10 Nisim Simsolo 2015-11-29 07:41:35 EST
engine issue starts at: 2015-11-29 14:26:17,613
VDSM issue starts at: 2015-11-29 14:27:06,409
Comment 11 meital avital 2015-11-30 03:50:41 EST
The main flow of this feature is not working at all and there is no webadmin workaround for it.
Comment 12 Shahar Havivi 2015-11-30 07:51:22 EST
There are several issues:

1. Some of the VMs are ileagal in libvirt point of view (there is exception in XMLDesc) - a patch will be sent to 3.6 to vdsm and will be reference in this bug.

2. The UI dialog is stuck in "waiting" mode if there is a timeout or error in the connection and not presenting the error to the user (me and Arik are working on that and will send a reference patch here).

3. The vcenter that you tested is getting timeout since you had a lot of VMs in the cluster and it took more then 5 minutes (the default timeout) to probe,
we can overcode that with a cluster with only a few VMs but in any case I am prety sure that it will fail the convert process (I did tested v2v feature with server that was based in china and client in israel with no success).
Comment 13 Shahar Havivi 2015-11-30 07:56:19 EST
(In reply to meital avital from comment #11)
> The main flow of this feature is not working at all and there is no webadmin
> workaround for it.
No, You do have a workaround!

* Open virsh to the server and see what VMs are legal (ie dumpxml <vm>)
* Add only the legal VMs to a cluster
* enter the RIGHT credentials to import-v2v dialog.
Comment 14 Red Hat Bugzilla Rules Engine 2015-12-02 07:27:07 EST
Bug tickets must have version flags set prior to targeting them to a release. Please ask maintainer to set the correct version flags and only then set the target milestone.
Comment 15 Michal Skrivanek 2015-12-02 09:50:02 EST
(In reply to Nisim Simsolo from comment #0)
> Version-Release number of selected component (if applicable):
> rhevm-3.6.0.3-0.1.el6 (3.6.0-20)
> libvirt-client-1.2.17-5.el7.x86_64

Nisim, libvirt version doesn't correspond to the one you should have installed for build 20. 
Does it happen on latest libvirt? Is it an old host? Worth upgrading
Comment 16 Red Hat Bugzilla Rules Engine 2015-12-06 03:22:20 EST
Bug tickets must have version flags set prior to targeting them to a release. Please ask maintainer to set the correct version flags and only then set the target milestone.
Comment 18 Michal Skrivanek 2015-12-09 04:54:55 EST
fix to stuck UI is in 3.6.1
we're missing the other part (recognizing the problematic VMs), but that's not blocking working VMs - hence moving bug to 3.6.2
Comment 19 Nisim Simsolo 2015-12-13 09:27:57 EST
Using latest build (rhevm-3.6.1.2-0.1.el6), It is possible to get VMware VMs list.
Still, DC/cluster parameters should be entered in data center text box (cluster text box is missing).
Comment 20 Sandro Bonazzola 2015-12-23 08:43:22 EST
oVirt 3.6.2 RC1 has been released for testing, moving to ON_QA
Comment 21 Nisim Simsolo 2016-01-25 04:00:39 EST
Verified: 
rhevm-3.6.2.6-0.1.el6
vdsm-4.17.17-0.el7ev.noarch
libvirt-client-1.2.17-13.el7_2.2.x86_64
qemu-kvm-rhev-2.3.0-31.el7_2.4.x86_64

Note You need to log in before you can comment on or make changes to this bug.