Bug 1560917

Created attachment 1413637 [details]
/vdsm/import.log

Created attachment 1413638 [details]
vdsm.log

Created attachment 1413639 [details]
engine.log

The libvirt XML shows only 4 NICs in the source:

    <interface type='bridge'>
      <mac address='00:1a:4a:16:01:55'/>
      <source bridge='VM Network'/>
      <model type='vmxnet3'/>
    </interface>
    <interface type='bridge'>
      <mac address='00:1a:4a:16:21:85'/>
      <source bridge='VM Network'/>
      <model type='e1000'/>
    </interface>
    <interface type='bridge'>
      <mac address='00:1a:4a:16:21:82'/>
      <source bridge='VM Network'/>
      <model type='e1000e'/>
    </interface>
    <interface type='bridge'>
      <mac address='00:1a:4a:16:21:69'/>
      <source bridge='VM Network'/>
      <model type='vmxnet3'/>
    </interface>

It could be that:

(1) There are only 4 NICs in the source.

(2) The libvirt vpx:// driver only reports the first 4 NICs for some reason.

(3) The other NICs are of a sort which the libvirt vpx driver cannot
parse for some reason.

Can you go to the source VMware server and grab the guest .vmx file?
If you go to the guest storage the .vmx file should be next to the
.vmdk file.  Usually you can connect to the ESXi hypervisor over https,
add "/folder" (without the trailing /) and go from there.

(In reply to Richard W.M. Jones from comment #5)
> The libvirt XML shows only 4 NICs in the source:
> 
>     <interface type='bridge'>
>       <mac address='00:1a:4a:16:01:55'/>
>       <source bridge='VM Network'/>
>       <model type='vmxnet3'/>
>     </interface>
>     <interface type='bridge'>
>       <mac address='00:1a:4a:16:21:85'/>
>       <source bridge='VM Network'/>
>       <model type='e1000'/>
>     </interface>
>     <interface type='bridge'>
>       <mac address='00:1a:4a:16:21:82'/>
>       <source bridge='VM Network'/>
>       <model type='e1000e'/>
>     </interface>
>     <interface type='bridge'>
>       <mac address='00:1a:4a:16:21:69'/>
>       <source bridge='VM Network'/>
>       <model type='vmxnet3'/>
>     </interface>
> 
> It could be that:
> 
> (1) There are only 4 NICs in the source.
> 
> (2) The libvirt vpx:// driver only reports the first 4 NICs for some reason.
> 
> (3) The other NICs are of a sort which the libvirt vpx driver cannot
> parse for some reason.
> 
> Can you go to the source VMware server and grab the guest .vmx file?
> If you go to the guest storage the .vmx file should be next to the
> .vmdk file.  Usually you can connect to the ESXi hypervisor over https,
> add "/folder" (without the trailing /) and go from there.

In the source VM there are 10 NICs, also when I run it all NICs are listed in "ip a" command. 
8 of the NICs adapter type is VMXNET3, 1 NIC is E1000 and the last one is E1000E.

.vmx file attached

> .vmx file attached

Not yet? :-?

Created attachment 1413667 [details]
vmx file

This is actually a bug in libvirt, as its vmx parser (used by the esx driver) hardcodes 4 as maximum number of network interfaces. I just posted a patch series fixing this:
https://www.redhat.com/archives/libvir-list/2018-March/msg01723.html

-----

virt-v2v -i vmx is not affected, and it correctly sees all the network interfaces:
$ virt-v2v -i vmx rh1560917.vmx --print-source 
[   0.0] Opening the source -i vmx rh1560917.vmx
Source guest information (--print-source option):

    source name: RHEL7_10_NICs
hypervisor type: vmware
         memory: 2147483648 (bytes)
       nr vCPUs: 8
     CPU vendor: 
      CPU model: 
   CPU topology: sockets: 4 cores/socket: 2 threads/core: -
   CPU features: 
       firmware: bios
        display: 
          video: vmvga
          sound: 
disks:
        RHEL7_6.vmdk (vmdk) [scsi]
        RHEL7_6_1.vmdk (vmdk) [scsi]
        /vmfs/volumes/5669422e-699d77db-c144-00e0815e303e/block4/block4.vmdk (vmdk) [scsi]
removable media:

NICs:
        Network "VM Network" [vmxnet3]
        Network "VM Network" mac: 00:1a:4a:16:21:85 [e1000]
        Network "VM Network" mac: 00:1a:4a:16:21:82 [e1000e]
        Network "VM Network" mac: 00:1a:4a:16:21:69 [vmxnet3]
        Network "VM Network" mac: 00:1a:4a:16:21:80 [vmxnet3]
        Network "VM Network" mac: 00:1a:4a:16:21:a3 [vmxnet3]
        Network "VM Network" mac: 00:1a:4a:16:21:a8 [vmxnet3]
        Network "VM Network" mac: 00:1a:4a:16:21:a9 [vmxnet3]
        Network "VM Network" mac: 00:1a:4a:16:21:78 [vmxnet3]
        Network "VM Network" mac: 00:1a:4a:16:21:81 [vmxnet3]

Now pushed upstream:

commit 5c744a2d445c01fdb9d83a9c3857ba6c0091d834
Author:     Pino Toscano <ptoscano>
AuthorDate: Wed Mar 28 13:40:21 2018 +0200
Commit:     Michal Privoznik <mprivozn>
CommitDate: Sun Apr 1 12:43:42 2018 +0200

    vmx: convert any amount of NICs
    
    Scan the parsed VMX file, and gather the biggest index of the network
    interfaces there: this way, it is possible to parse all the available
    network interfaces, instead of just 4 maximum.
    
    Add the VMX file attached to RHBZ#1560917 as testcase esx-in-the-wild-8.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1560917
    
    Signed-off-by: Pino Toscano <ptoscano>

commit e508e04b108709af0ca2c4d6728440b0edc3cc4b
Author:     Pino Toscano <ptoscano>
AuthorDate: Wed Mar 28 13:40:20 2018 +0200
Commit:     Michal Privoznik <mprivozn>
CommitDate: Sun Apr 1 12:43:42 2018 +0200

    internal: add STRCASEPREFIX
    
    Simple macro to check the prefix of a string in a case-insensitive way.
    
    Signed-off-by: Pino Toscano <ptoscano>

commit 6d394989fda53b53f0a21522bebe44ea01a40de9
Author:     Pino Toscano <ptoscano>
AuthorDate: Wed Mar 28 13:40:19 2018 +0200
Commit:     Michal Privoznik <mprivozn>
CommitDate: Sun Apr 1 12:43:42 2018 +0200

    vmx: allocate space for network interfaces if needed
    
    Dynamically grow the array of network interfaces for each interface
    read, instead of using a single array of size 4.  This way, in the
    future it will be easier to not limit the number of network interfaces
    (which this patch still does not change).
    
    Signed-off-by: Pino Toscano <ptoscano>

commit 5d5430e1fc6f853f79da39fbef08e97bebffc8eb
Author:     Pino Toscano <ptoscano>
AuthorDate: Wed Mar 28 13:40:18 2018 +0200
Commit:     Michal Privoznik <mprivozn>
CommitDate: Sun Apr 1 12:43:42 2018 +0200

    vmx: check for present/enabled devices earlier
    
    When parsing filesystems, network interfaces, serial ports, and
    parallel ports, check earlier whether they are present/enabled, delaying
    the allocation of the objects.
    
    This is mostly a small optimization, with no behaviour change.
    
    Signed-off-by: Pino Toscano <ptoscano>

v4.2.0-6-g5c744a2d44

I can reproduce the bug with below builds:
libvirt-3.9.0-14.el7_5.2.x86_64
virt-v2v-1.36.10-6.el7_5.1.x86_64

Steps:
Scenario 1:
1.Prepare a guest with 10 NICs on ESXi6.5
2.Using virsh to connect Vcenter:
# virsh -c vpx://root@vcenter_IP/data/ESXi_IP/?no_verify=1
3.Dump the guest xml, but we can only find 4 NICs:
virsh # dumpxml rhel7.4-tmp-test
4.Using virt-v2v convert the guest to RHV4.2, Only 4 NICs were found after conversion.

Scenario 2:
1.Prepare a guest with 10 NICs on ESXi6.5
2.Import the VM using RHV environment
3.Only 4 NICs were found after conversion.

Scenario 3:
1.Prepare a guest with 10 NICs on ESXi6.5
2.Mount the guest metadata folder to v2v conversion server.
3.Using virt-v2v convert the guest to rhv
#virt-v2v -i vmx guest.vmx -o rhv -os IP:/home/nfs_export -of qcow2
4.After conversion, 10 NICs were found on RHV

Scenario 4:
1.Prepare a guest with 10 NICs on ESXi6.5
2.Export the guest to a OVA file.
3.Using virt-v2v convert the guest to rhv
 # virt-v2v -i ova  rhel7.4.ova -o rhv -os IP:/home/nfs_export -of qcow2
4.After conversion, 10 NICs were found on RHV


So, from comment 9, it is a bug in libvirt.
virt-v2v -i vmx and -i ova is not affected, and it correctly sees all the network interfaces

Verified:
rhvm-4.2.4-0.1.el7
libvirt-client-3.9.0-14.el7_5.5.x86_64
sanlock-3.6.0-1.el7.x86_64
qemu-kvm-rhev-2.10.0-21.el7_5.3.x86_64
vdsm-4.20.28-1.el7ev.x86_64
virt-v2v-1.36.10-6.10.rhvpreview.el7ev.x86_64

Verification scenario:
1. Import VMware VM with 10 NICs.
2. Import VMware OVA of VM with 10 NICs.

For both cases:
- Verify VM/OVA imported successfully
- Browse RHV Webadmin -> Compute -> VMs -> click on VM -> Network interfaces tab and verify 10 NICs are added to VM, each one with MAC address from RHV cluster MAC addresses pool
- Run VM and verify VM is running with 10 NICs, and each NIC have IP address from DHCP server.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:3113