Description of problem: Migration of VM from VMware VM to RHV using the setup and migration steps described in the link [1] fails to work. Migration task fails with 'VM Transformations failed'. In addition, the percentage in the progress bar is not updated in the UI, and stays at 0%. Automation.log logs multiple errors see [2]. virt-v2v logs [3] on the conversion host keeps filling up (indicating a progress of v2v process) while MIQ reports a task failure. Source environment is VMware vSphere v6.5, source VM is in power-off state. Target environment is RHV 4.2.3 with currently one selected RHV host as migration host. Both environments makes use of iSCSI data-stores. Version-Release number of selected component (if applicable): RHV 4.2.3.2-0.1.el7 VMware vSphere 6.5 How reproducible: 100% Steps to Reproduce: From compute->migration: 1. Create infrastructure mapping between VMware to RHV: Select source/target clusters, source/target data-store, source/target networks. 2. Create a local CSV file with the following columns: <vm_name>, <migration_plan_name>, <id> 3. Run migration plan, choose the infrastructure mapping from step 1 and CSV from step 2. Actual results: 1. 'VM Transformations failed'. 2. Progress is not updated in the UI and stays at 0%. Expected results: Should succeed, and progress bar should be updated. Additional info: [1] - https://github.com/RedHatDemos/RHS-Optimize_IT-Infrastructure_Migration/blob/master/docs/00-redhat_solutions-insfrastructure_migration_v2-deployment.adoc [2] Automation.log [----] E, [2018-05-03T12:06:21.918450 #36140:cb24628] ERROR -- : <AEMethod vmchecktransformed_vmwarews2rhevm_vddk> The following error occurred during method evaluation: [----] E, [2018-05-03T12:06:21.920016 #36140:cb24628] ERROR -- : <AEMethod vmchecktransformed_vmwarews2rhevm_vddk> NoMethodError: undefined method `each' for nil:NilClass [----] E, [2018-05-03T12:06:21.925603 #36140:cb24628] ERROR -- : <AEMethod vmchecktransformed_vmwarews2rhevm_vddk> /V2V/Transformation/TransformationHosts/ovirt_host/VMCheckTransformed_vmwarews2rhevm_vddk:40:i n `main' [----] E, [2018-05-03T12:06:21.937422 #36140:cb24628] ERROR -- : Method STDERR: /V2V/Transformation/TransformationHosts/ovirt_host/VMCheckTransformed_vmwarews2rhevm_vddk:40:in `main': undefined method `each' for nil:NilClass (NoMethodError) [----] E, [2018-05-03T12:06:21.938896 #36140:cb24628] ERROR -- : Method STDERR: from /V2V/Transformation/TransformationHosts/ovirt_host/VMCheckTransformed_vmwarews2rhevm_vddk:86:in `<main>' [----] I, [2018-05-03T12:06:22.015798 #36140:b1114c] INFO -- : Q-task_id([r16_service_template_transformation_plan_task_16]) <AEMethod [/V2V/Transformation/TransformationHosts/ovirt_host/VMCheckTransformed_vmwa rews2rhevm_vddk]> Ending [----] E, [2018-05-03T12:06:22.016976 #36140:b1114c] ERROR -- : Q-task_id([r16_service_template_transformation_plan_task_16]) Aborting instantiation (unknown method return code) because [Method exited with rc=Un known RC: [1]] [----] E, [2018-05-03T12:06:22.018069 #36140:b1114c] ERROR -- : Q-task_id([r16_service_template_transformation_plan_task_16]) State=<State2> running raised exception: <Method exited with rc=Unknown RC: [1]> [3] v2v-import-date-number-wrapper.log --- tailed log lines -- 2018-05-03 09:58:56,283:DEBUG: Updated progress: 100.00 (virt-v2v-wrapper:142) 2018-05-03 10:25:28,154:INFO: virt-v2v terminated with return code 0 (virt-v2v-wrapper:323) 2018-05-03 10:25:28,158:INFO: Removing password files (virt-v2v-wrapper:460) 2018-05-03 10:25:28,159:INFO: Finished (virt-v2v-wrapper:479)
MIQ version: master.20180417121918_2dd64f8.
I see two v2v logs: v2v-import-20180430T085329-43940.log v2v-import-20180503T090617-41661.log Both are successful conversions.
E-mail conversation from KK: Hey Mor, Thanks for filing the BZ. I would like to request you to provide me access to your environment . And I would also like to request you to try migration with latest nightly appliance as there have been a lot of changes in the latest nightly. Please let me know the info of your appliance for further investigation. Thanks, KK.
(In reply to Richard W.M. Jones from comment #5) > I see two v2v logs: > > v2v-import-20180430T085329-43940.log > v2v-import-20180503T090617-41661.log > > Both are successful conversions. The logs shows success, but no VM is created on RHV, and MIQ reports migration failure.
Quick update on this ticket: I had a great session with KK regarding the v2v workflow, and the new nightly version which contains significant improvements. It has decided not to waste time on my current version, so I will try upgrade my environment to the latest nightly MIQ next week and update you with results. Meanwhile - lowering the severity to medium.
Update on latest versions: RHV 4.2.3.5-0.1.el7 MIQ master.20180504232106_a937a31 I tried to run the migration plan again, to migrate a single VM, but I still get the error: "VM Transformations failed". I'm changing the severity back to HIGH and attaching new logs.
Created attachment 1432525 [details] automation and import logs
this is still using -o rhv, not the new rhv-upload. What are you testing?
I'm testing the latest CFME from master installed using: https://github.com/ovirt/ovirt-ansible-manageiq and configured by following: https://github.com/RedHatDemos/RHS-Optimize_IT-Infrastructure_Migration/blob/master/docs/00-redhat_solutions-insfrastructure_migration_v2-deployment.adoc
relevant bug to this one Bug 1574734 - raw + sparse isn't working with iSCSI storage domain for upload (edit) https://bugzilla.redhat.com/show_bug.cgi?id=1574734
https://people.redhat.com/~rjones/virt-v2v-RHEL-7.5-rhv-preview/ virt-v2v >= -6.10.rhvpreview from the repository above removes the restriction on sparse. You can now use ‘-oa preallocated’ on the command line which should work for iSCSI & FC transfers. It doesn't (because it cannot) solve the problem that different storage domains have unknowable requirements so you have to somehow just know whether or not to pass ‘-oa preallocated’.
(In reply to Richard W.M. Jones from comment #18) > https://people.redhat.com/~rjones/virt-v2v-RHEL-7.5-rhv-preview/ > > virt-v2v >= -6.10.rhvpreview from the repository above removes the > restriction on sparse. You can now use ‘-oa preallocated’ on the > command line which should work for iSCSI & FC transfers. > > It doesn't (because it cannot) solve the problem that different > storage domains have unknowable requirements so you have to somehow > just know whether or not to pass ‘-oa preallocated’. Richard, the workaround works manually from the command line, not if triggered virt-v2v >= -6.10.rhvpreview from MIQ, right?
I believe that is correct, yes.
Testing last week was successful in migrating to iSCSI storage domains using -oa preallocated with RHV compose.
I'm failing this bug on CFME version 5.10.0.0.20180613200131_887cc81 When trying to migrate a VM from vSphere 6.5 I get "VM Transformations failed" with the following error: [----] E, [2018-06-18T23:34:23.638528 #11725:aaf5820] ERROR -- : <AEMethod vmtransform_vmwarews2rhevm_vddk> The following error occurred during method evaluation: [----] E, [2018-06-18T23:34:23.640039 #11725:aaf5820] ERROR -- : <AEMethod vmtransform_vmwarews2rhevm_vddk> Errno::ENOTTY: Inappropriate ioctl for device [----] E, [2018-06-18T23:34:23.644992 #11725:aaf5820] ERROR -- : <AEMethod vmtransform_vmwarews2rhevm_vddk> /opt/rh/cfme-gemset/gems/net-ssh-4.2.0/lib/net/ssh/prompt.rb:44:in `noecho' [----] E, [2018-06-18T23:34:23.686174 #11725:aaf5820] ERROR -- : Method STDERR: /opt/rh/cfme-gemset/gems/net-ssh-4.2.0/lib/net/ssh/prompt.rb:44:in `noecho': Inappropriate ioctl for device (Errno::ENOTTY) [----] E, [2018-06-18T23:34:23.690540 #11725:aaf5820] ERROR -- : Method STDERR: from /opt/rh/cfme-gemset/gems/net-ssh-4.2.0/lib/net/ssh/prompt.rb:44:in `ask' [----] E, [2018-06-18T23:34:23.692198 #11725:aaf5820] ERROR -- : Method STDERR: from /opt/rh/cfme-gemset/gems/net-ssh-4.2.0/lib/net/ssh/authentication/methods/password.rb:68:in `ask_password' [----] E, [2018-06-18T23:34:23.693836 #11725:aaf5820] ERROR -- : Method STDERR: from /opt/rh/cfme-gemset/gems/net-ssh-4.2.0/lib/net/ssh/authentication/methods/password.rb:21:in `authenticate' [----] E, [2018-06-18T23:34:23.695531 #11725:aaf5820] ERROR -- : Method STDERR: from /opt/rh/cfme-gemset/gems/net-ssh-4.2.0/lib/net/ssh/authentication/session.rb:80:in `block in authenticate' [----] E, [2018-06-18T23:34:23.697077 #11725:aaf5820] ERROR -- : Method STDERR: from /opt/rh/cfme-gemset/gems/net-ssh-4.2.0/lib/net/ssh/authentication/session.rb:66:in `each' [----] E, [2018-06-18T23:34:23.698736 #11725:aaf5820] ERROR -- : Method STDERR: from /opt/rh/cfme-gemset/gems/net-ssh-4.2.0/lib/net/ssh/authentication/session.rb:66:in `authenticate' [----] E, [2018-06-18T23:34:23.700160 #11725:aaf5820] ERROR -- : Method STDERR: from /opt/rh/cfme-gemset/gems/net-ssh-4.2.0/lib/net/ssh.rb:241:in `start' [----] E, [2018-06-18T23:34:23.701780 #11725:aaf5820] ERROR -- : Method STDERR: from ManageIQ/Transformation/TransformationHosts/ovirt_host/Utils:15:in `remote_command' [----] E, [2018-06-18T23:34:23.703321 #11725:aaf5820] ERROR -- : Method STDERR: from /ManageIQ/Transformation/TransformationHosts/ovirt_host/VMTransform_vmwarews2rhevm_vddk:74:in `main' [----] E, [2018-06-18T23:34:23.704753 #11725:aaf5820] ERROR -- : Method STDERR: from /ManageIQ/Transformation/TransformationHosts/ovirt_host/VMTransform_vmwarews2rhevm_vddk:103:in `<main>' [----] E, [2018-06-18T23:34:23.854573 #11725:128ef78] ERROR -- : Q-task_id([r1000000000010_service_template_transformation_plan_task_1000000000022]) Aborting instantiation (unknown method return code) because [Method exited with rc=Unknown RC: [1]] [----] E, [2018-06-18T23:34:23.855223 #11725:128ef78] ERROR -- : Q-task_id([r1000000000010_service_template_transformation_plan_task_1000000000022]) State=<State1> running raised exception: <Method exited with rc=Unknown RC: [1]> Logs will be attached.
There are no import logs generated. It fails when executing the virt-v2v wrapper. PLEASE NOTE: evm.log contains another unrelated keystone error. It's on a totally different RHV/VMWare registered environment. We are still investigating it. If you aware of this issue, please let us know. [----] E, [2018-06-19T11:00:31.733875 #23546:358f7c] ERROR -- : MIQ(ManageIQ::Providers::Redhat::NetworkManager#verify_api_credentials) Error Class=Excon::Error::Unauthorized, Message=Expected([200, 204]) <=> Actual(401 Unauthorized) excon.error.response :body => "{\n \"error\": {\n \"message\": \"Error during SSO authentication invalid_request : Missing parameter: 'client_secret'\",\n \"code\": 401,\n \"title\": \"Unauthorized\"\n }\n}\n" :cookies => [ ] :headers => { "Connection" => "close" "Content-Type" => "application/json" "Date" => "Tue, 19 Jun 2018 08:00:31 GMT" "Server" => "BaseHTTP/0.3 Python/2.7.5" } :host => "nsimsolo41.scl.lab.tlv.redhat.com" :local_address => "10.12.69.26" :local_port => 35058 :path => "/v2.0/tokens" :port => 35357 :reason_phrase => "Unauthorized" :remote_ip => "10.35.161.173" :status => 401 :status_line => "HTTP/1.0 401 Unauthorized\r\n" [----] W, [2018-06-19T11:00:31.734057 #23546:358f7c] WARN -- : MIQ(ManageIQ::Providers::Redhat::NetworkManager::EventCatcher::Runner#start_event_monitor) EMS [nsimsolo41.scl.lab.tlv.redhat.com] as [admin@internal] Login failed due to a bad username or password.
Created attachment 1452842 [details] evm and automation logs
MIQ nightly: master.20180619230249_84e9fa9 found to be working. I will downgrade the CFME from 5.10 to 5.9 and reopen the bug in case it fails.