Bug 1708359

Appliance can be accessed via https://10.8.197.176/migration#/plans

The problem is not about the conversion hosts not being there, but is due to the mapping being wrong.
The ValidateRequest method follows the relationships to verify that there is at least on conversion host available in the destination providers.
The chain is:

ServiceTemplateTransformationPlan
  -> TransformationMapping
    -> TransformationMappingItems where source_type == 'EmsCluster'
      -> EmsCluster or CloudTenant
        -> ExtManagementSystem
          -> ConversionHost

Looking into the appliance, I focused on the denied requests, i.e. those who received MIQ_ABORT.

irb(main):011:0> ServiceTemplateTransformationPlanRequest.where(approval_state: 'denied').last.validate_conversion_hosts
NoMethodError: undefined method `ext_management_system' for nil:NilClass
	from /var/www/miq/vmdb/app/models/service_template_transformation_plan_request.rb:22:in `block in validate_conversion_hosts'
	from /var/www/miq/vmdb/app/models/service_template_transformation_plan_request.rb:21:in `each'
	from /var/www/miq/vmdb/app/models/service_template_transformation_plan_request.rb:21:in `all?'
	from /var/www/miq/vmdb/app/models/service_template_transformation_plan_request.rb:21:in `validate_conversion_hosts'
	from (irb):11
	from /opt/rh/cfme-gemset/gems/railties-5.0.7.2/lib/rails/commands/console.rb:65:in `start'
	from /opt/rh/cfme-gemset/gems/railties-5.0.7.2/lib/rails/commands/console_helper.rb:9:in `start'
	from /opt/rh/cfme-gemset/gems/railties-5.0.7.2/lib/rails/commands/commands_tasks.rb:78:in `console'
	from /opt/rh/cfme-gemset/gems/railties-5.0.7.2/lib/rails/commands/commands_tasks.rb:49:in `run_command!'
	from /opt/rh/cfme-gemset/gems/railties-5.0.7.2/lib/rails/commands.rb:18:in `<top (required)>'
	from bin/rails:4:in `require'
	from bin/rails:4:in `<main>'

What this stack trace says is that the destination EmsCluster or CloudTenant is nil.
This is confirmed by the next command:

irb(main):013:0> ServiceTemplateTransformationPlanRequest.where(approval_state: 'denied').last.transformation_mapping.transformation_mapping_items.select { |i| i.source_type == 'EmsCluster' }
=> [#<TransformationMappingItem id: 48, source_id: 2, source_type: "EmsCluster", destination_id: 7, destination_type: "CloudTenant", transformation_mapping_id: 6, options: {}, created_at: "2019-05-09 18:06:40", updated_at: "2019-05-09 18:06:40">]
irb(main):014:0> CloudTenant.find(7)
ActiveRecord::RecordNotFound: Couldn't find CloudTenant with 'id'=7

I see on this same appliance that some requests went beyond the ValidateRequest method but they still failed. So, I went further into the error by checking the task:

irb(main):016:0> ServiceTemplateTransformationPlanRequest.where(approval_state: 'approved').last
=> #<ServiceTemplateTransformationPlanRequest id: 30, description: "vddk_plan", approval_state: "approved", type: "ServiceTemplateTransformationPlanRequest", created_on: "2019-05-09 19:28:43", updated_on: "2019-05-09 19:31:30", fulfilled_on: "2019-05-09 19:31:30", requester_id: 1, requester_name: "Administrator", request_type: "transformation_plan", request_state: "finished", message: "VM Transformations failed", status: "Error", options: {:dialog=>nil, :workflow_settings=>{:resource_action_id=>82}, :initiator=>nil, :src_id=>9, :request_options=>{:submit_workflow=>true, :init_defaults=>false}, :cart_state=>"ordered", :requester_group=>"EvmGroup-super_administrator", :executed_on_servers=>[1, 1], :delivered_on=>2019-05-09 19:28:59 UTC}, userid: "admin", source_id: 9, source_type: "ServiceTemplate", destination_id: nil, destination_type: nil, tenant_id: 1, service_order_id: 20, process: true, cancelation_status: nil>

irb(main):017:0> ServiceTemplateTransformationPlanTask.where(miq_request_id: 30).last.options[:virtv2v_wrapper]
=> {"wrapper_log"=>"/var/log/virt-v2v/v2v-import-20190509T153112-520-wrapper.log", "v2v_log"=>"/var/log/virt-v2v/v2v-import-20190509T153112-520.log", "state_file"=>"/tmp/v2v-import-20190509T153112-520.state", "throttling_file"=>"/tmp/v2v-import-20190509T153112-520.throttle"}

This allowed me to know the path of the logs for virt-v2v-wrapper and virt-v2v. I connect to the conversion host (using the credentials registered in CFME) and looked into the virt-v2v-wrapper log where I found:

2019-05-09 15:31:18,615:ERROR: Command exited with non-zero return code 1, output:
Failed to discover available identity versions when contacting https://10.8.58.137:13000/v3. Attempting to parse version from URL.
SSL exception connecting to https://10.8.58.137:13000/v3/auth/tokens: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:618)

So, the second issue here is that the SSL CA certificate for the OSP provider isn't present on the conversion host.

In the current implementation, we don't store the provider's CA certificate for OSP providers. The RHV provider allows setting it in the provider configuration via the UI, while it's not possible for OSP. We will file a RFE, but it won't be implemented before CFME 5.11. The conversion host enablement playbook is already able to accept a CA certificate, so we could ask the user to provide it in the wizard. The change on the backend would be minimal.

Conversion Host - Handle CA bundle from UI - https://github.com/ManageIQ/manageiq/pull/18762

Add OpenStack Trusted CA Certificates field to Conversion Host Wizard - https://github.com/ManageIQ/manageiq-v2v/pull/952

Fixed! I can see migration can proceed when we have conversion host configured at endpoint.
Verified on: 5.11.0.8.20190611155126_01e077e