Bug 1506572 - virt-v2v '-i ova' is not parsing the MAC address from the source OVF
Summary: virt-v2v '-i ova' is not parsing the MAC address from the source OVF
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: libguestfs
Version: 7.4
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: rc
: ---
Assignee: Richard W.M. Jones
QA Contact: Virtualization Bugs
Yehuda Zimmerman
URL:
Whiteboard: V2V
Depends On:
Blocks: 1503947
TreeView+ depends on / blocked
 
Reported: 2017-10-26 10:55 UTC by Nisim Simsolo
Modified: 2018-04-10 09:21 UTC (History)
9 users (show)

Fixed In Version: libguestfs-1.36.10-2.el7
Doc Type: Bug Fix
Doc Text:
When importing OVAs, *virt-v2v* now parses MAC addresses Previously, the *virt-v2v* utility did not parse the MAC addresses of network interfaces when importing Open Virtual Appliances (OVAs). Consequently, the converted guest virtual machines had network interfaces with different MAC addresses, resulting in the network setup breaking. With this release, *virt-v2v* parses the MAC addresses, if available, of network interfaces when importing OVAs. As a result, network converted guests have the same MAC addresses as specified in the OVAs and the network setup does not break.
Clone Of:
Environment:
Last Closed: 2018-04-10 09:20:40 UTC
Target Upstream Version:


Attachments (Terms of Use)
centos44.ovf (8.37 KB, application/xml)
2017-10-26 13:33 UTC, Nisim Simsolo
no flags Details
RHEL7_1_with_MAC.ovf (9.32 KB, text/plain)
2017-11-01 18:09 UTC, Richard W.M. Jones
no flags Details
Scrrenshot of VMware OVF creation - advanced options (383.31 KB, image/png)
2017-11-02 08:06 UTC, Nisim Simsolo
no flags Details
network-rhv (78.91 KB, image/png)
2017-11-08 09:05 UTC, mxie@redhat.com
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2018:0677 0 None None None 2018-04-10 09:21:36 UTC

Description Nisim Simsolo 2017-10-26 10:55:17 UTC
Description of problem:
Importing VMware OVA file using RHV 4.2.0 Webadmin failed with MAC duplication error although there is no such MAC in use.
Also importing OVA manually using 'virt-v2v -i ova' command failed with "General command validation failure"
According to https://bugzilla.redhat.com/show_bug.cgi?id=1503947#c18, 
-i ova method does not parse the MAC address from the source
VMware OVF


Version-Release number of selected component (if applicable):
virt-v2v-1.36.3-6.el7_4.3.x86_64
vdsm-4.20.3-216.git87725ec.el7.centos.x86_64
libvirt-client-3.2.0-14.el7_4.3.x86_64
qemu-kvm-rhev-2.9.0-16.el7_4.8.x86_64
ovirt-engine-4.2.0-0.0.master.20171023193344.gitc3b23f6.el7.centos

How reproducible:
100%

Steps to Reproduce:
1. See bug https://bugzilla.redhat.com/show_bug.cgi?id=1503947 steps to reproduce.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 2 Pino Toscano 2017-10-26 12:26:47 UTC
A simpler reproducer would be:

virt-v2v -i ova file.ova --print-source

If the MAC file is parsed, you should see:

NICs:
        Network "<name of the network>" mac: ab:cd:ef:01:23:45

instead of

NICs:
        Network "<name of the network>"

Also, -i disk cannot provide a MAC, since... there is no OVF to parse, in that case: all virt-v2v can use is the disk.

Can you please provide an OVA (or at least an OVF) with MAC address?

Comment 3 Nisim Simsolo 2017-10-26 13:30:41 UTC
[root@intel-vfio nisim]# virt-v2v -i ova centos44.ova --print-source
[   0.0] Opening the source -i ova centos44.ova
virt-v2v: warning: making OVA directory public readable to work around 
libvirt bug https://bugzilla.redhat.com/1045069
Source guest information (--print-source option):

    source name: centos44
hypervisor type: vmware
         memory: 2147483648 (bytes)
       nr vCPUs: 1
   CPU features: 
       firmware: bios
        display: 
          video: 
          sound: 
disks:
	json:{ "file": { "driver": "raw", "offset": 9728, "size": 360347648, "file": { "driver": "file", "filename": "/home/nisim/centos44.ova" } } } (vmdk) [scsi]
removable media:
	CD-ROM [ide] in slot 0
	Floppy in slot 0
NICs:
	Network "Network adapter 1"

[root@intel-vfio nisim]# 

Please see ovf file attached

Comment 4 Nisim Simsolo 2017-10-26 13:33:00 UTC
Created attachment 1343738 [details]
centos44.ovf

Comment 5 Pino Toscano 2017-10-26 13:44:46 UTC
(In reply to Nisim Simsolo from comment #4)
> Created attachment 1343738 [details]
> centos44.ovf

This OVF file does not have any MAC address for the network device.

Comment 6 Pino Toscano 2017-10-26 13:45:34 UTC
Comment on attachment 1343738 [details]
centos44.ovf

It does not have a MAC address.

Comment 7 Richard W.M. Jones 2017-10-27 06:18:43 UTC
I'll add here that oVirt still has to be fixed to deal with
the "no <MacAddress>" situation.  virt-v2v is not in a position
where it can make up MAC addresses if they are missing in the
metadata.  For example, we don't know the target hypervisor
administrator's policy, nor can we always choose non-conflicting
MAC addresses.

However we also need to fix virt-v2v to preserve the MAC if it
is present in the OVA metadata, which is what this bug is for.

Comment 8 Richard W.M. Jones 2017-11-01 13:46:31 UTC
(In reply to Pino Toscano from comment #6)
> It does not have a MAC address.

I had a look through my collection of OVA files generated from
VMware and none of them had a <MACAddress> field.  So I don't
know, but it strengthens my belief even more that oVirt needs
to be able to handle the case where there is no MAC address
and create one using local policy.

Anyway, it looks like if the correct XML should be:

<Item>
<rasd:ResourceType>10</rasd:ResourceType>
<rasd:ResourceSubType>3</rasd:ResourceSubType>
<rasd:MACAddress>00:11:22:33:44:55</rasd:MACAddress>
...
</Item>

Comment 9 Richard W.M. Jones 2017-11-01 13:51:01 UTC
See also bug 1378045.  Note these are two separate, independent
bugs.  virt-v2v should preserve the MAC address if it is present,
but does not have enough information to make one up.

Comment 10 Nisim Simsolo 2017-11-01 15:42:55 UTC
It took me time to figure out how to create such OVA file and in order to generate OVA with MAC address, VMware Client Integration Plugin is needed to be installed first. 
Afterward from vSphere -> select VM -> actions -> template -> export OVF template -> select advanced options check 'enable MAC address' checkbox.

Please contact me if you need such OVA file.

Comment 13 Richard W.M. Jones 2017-11-01 18:09:51 UTC
Created attachment 1346654 [details]
RHEL7_1_with_MAC.ovf

Interestingly Nisim's OVA does contain a MAC address, but
it's not using the <MACAddress/> element as we thought.  Instead
it uses the <Address/> element.  I wonder if VMware made that up?

      <Item>
        <rasd:Address>00:0c:29:72:1e:79</rasd:Address>
...
        <rasd:ResourceType>10</rasd:ResourceType>

Comment 14 Richard W.M. Jones 2017-11-01 18:16:48 UTC
Gloriously the DTMF "standard" uses <Address> ... but with a
completely different set of enclosing elements and format for the
MAC address:

<EthernetPortItem>
  <epasd:Address>00-16-8B-DB-00-5E</epasd:Address>

This is using something called "CIM_EthernetPortAllocationSettingData"
at which point my eyes glaze over.

Anyway it looks like we should be prepared to accept a wide
range of formats.

Comment 16 Nisim Simsolo 2017-11-02 08:06:58 UTC
Created attachment 1346861 [details]
Scrrenshot of VMware OVF creation - advanced options

Comment 17 Pino Toscano 2017-11-02 10:33:32 UTC
I guess we will need to handle each method as soon as we have OVAs/OVFs with them.

I just sent a simple patch for <rasd:Address>:
https://www.redhat.com/archives/libguestfs/2017-November/msg00000.html

Comment 18 Richard W.M. Jones 2017-11-02 10:42:24 UTC
(In reply to Nisim Simsolo from comment #16)
> Created attachment 1346861 [details]
> Scrrenshot of VMware OVF creation - advanced options

It would definitely be interesting to have an OVF with
everything enabled.

We don't even try to preserve the BIOS UUID at the moment,
but it would be interesting to try.

One thing that we don't have from OVF now but we would like
to preserve is CPU vendor/model.  Also we only have partial
topology information (#threads is missing).

The internal model used by virt-v2v is:

https://github.com/libguestfs/libguestfs/blob/22d5a20ed240b2d0c35a843314782b342fa1d73a/v2v/types.mli#L59-L161

The current parser for OVA/OVF is:

https://github.com/libguestfs/libguestfs/blob/master/v2v/input_ova.ml
https://github.com/libguestfs/libguestfs/blob/master/v2v/parse_ovf_from_ova.ml

Comment 19 Pino Toscano 2017-11-02 13:00:22 UTC
Patch ACKed, and pushed.

Comment 22 mxie@redhat.com 2017-11-08 06:26:09 UTC
Try to verify the bug with builds:
RHV:4.1.7.5-0.1.el7
virt-v2v-1.36.10-2.el7.x86_64
libguestfs-1.36.10-2.el7.x86_64
libvirt-3.9.0-1.el7.x86_64
qemu-kvm-rhev-2.10.0-4.el7.x86_64


Steps:
1.Prepare a ova file whose ovf has MAC address in network part
#cat esx-rhel6.8-ova/esx5.5-rhel6.8-x86_64.ovf 
.....
 <Item>
        <rasd:Address>00:0c:30:27:1c:82</rasd:Address>
        <rasd:AddressOnParent>7</rasd:AddressOnParent>
        <rasd:AutomaticAllocation>true</rasd:AutomaticAllocation>
        <rasd:Connection>VM Network</rasd:Connection>
        <rasd:Description>VmxNet3 ethernet adapter on "VM Network"</rasd:Description>

......

2.Use virt-v2v to convert this guest to rhv
# virt-v2v -i ova esx-rhel6.8-ova -o rhv -os 10.73.131.93:/home/nfs_export -of qcow2 -on add-mac-ova
[   0.0] Opening the source -i ova esx-rhel6.8-ova
virt-v2v: warning: making OVA directory public readable to work around 
libvirt bug https://bugzilla.redhat.com/1045069
[  17.1] Creating an overlay to protect the source from being modified
[  18.0] Initializing the target -o rhv -os 10.73.131.93:/home/nfs_export
[  18.7] Opening the overlay
[  48.5] Inspecting the overlay
[  61.6] Checking for sufficient free disk space in the guest
[  61.6] Estimating space required on target for each disk
[  61.6] Converting Red Hat Enterprise Linux Server release 6.8 (Santiago) to run on KVM
virt-v2v: This guest has virtio drivers installed.
[ 116.2] Mapping filesystem data to avoid copying unused and blank areas
[ 116.5] Closing the overlay
[ 117.5] Checking if the guest needs BIOS or UEFI to boot
[ 117.5] Assigning disks to buses
[ 117.5] Copying disk 1/1 to /tmp/v2v.0dK5DG/bdf9c90b-e6f0-439c-aa9f-6305fd5fad7e/images/5a802fc7-45f2-40b0-9efb-f7690c09a7d9/726d2507-4cc5-4ae4-adb6-5b3f97c8ebf4 (qcow2)
    (100.00/100%)
[ 406.0] Creating output metadata
[ 406.1] Finishing off

3.After finishing conversion, check guest's ovf file in export domain of rhv, there is mac address in network part
# cat 5aa0b5ea-c98e-4050-a127-d6768ff4527c/5aa0b5ea-c98e-4050-a127-d6768ff4527c.ovf 
....
<Item>
        <rasd:InstanceId>62e1f7d7-8a44-414b-a093-02be5c24f59b</rasd:InstanceId>
        <rasd:Caption>Ethernet adapter on Network adapter 1</rasd:Caption>
        <rasd:ResourceType>10</rasd:ResourceType>
        <rasd:ResourceSubType>3</rasd:ResourceSubType>
        <Type>interface</Type>
        <rasd:Connection>Network adapter 1</rasd:Connection>
        <rasd:Name>eth0</rasd:Name>
        <rasd:MACAddress>00:0c:30:27:1c:82</rasd:MACAddress>
      </Item>
....

4.Power on the guest and checkpoints of guest are passed

Result:
   V2V could read the MAC address of the network interfaces from the <rasd:Address>, so move the bug from ON_QA to VERIFIED

Comment 23 Pino Toscano 2017-11-08 08:52:19 UTC
Ming: did you also check that, in the running converted guest, the network interface has the proper MAC address?

Comment 24 mxie@redhat.com 2017-11-08 09:05:18 UTC
(In reply to Pino Toscano from comment #23)
> Ming: did you also check that, in the running converted guest, the network
> interface has the proper MAC address?

Do you mean check the network of guest which was imported from export domain at rhv, the network has proper MAC address which is same with ova file's MAC address, pls refer to screenshot“network-rhv”, so guest's mac address is from original ova file,not created by rhv randomly

Comment 25 mxie@redhat.com 2017-11-08 09:05:46 UTC
Created attachment 1349326 [details]
network-rhv

Comment 26 Pino Toscano 2017-11-08 12:21:35 UTC
(In reply to mxie from comment #24)
> (In reply to Pino Toscano from comment #23)
> > Ming: did you also check that, in the running converted guest, the network
> > interface has the proper MAC address?
> 
> Do you mean check the network of guest which was imported from export domain
> at rhv, the network has proper MAC address which is same with ova file's MAC
> address, pls refer to screenshot“network-rhv”, so guest's mac address is
> from original ova file,not created by rhv randomly

That is good to know -- what I meant is checking the MAC address from the guest itself, e.g. by looking at the output of `ip` for the network interface (see the "link/ether" line).

Comment 27 mxie@redhat.com 2017-11-09 06:49:21 UTC
Check network status in guest:
# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:0C:30:27:1C:82  
          inet addr:10.66.144.227  Bcast:10.66.147.255  Mask:255.255.252.0
          inet6 addr: 2620:52:0:4292:20c:30ff:fe27:1c82/64 Scope:Global
          inet6 addr: fe80::20c:30ff:fe27:1c82/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:221240 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2254 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:15662837 (14.9 MiB)  TX bytes:177785 (173.6 KiB)

Comment 28 Pino Toscano 2017-11-09 08:13:34 UTC
Sounds good, thank you for checking :)

Comment 33 errata-xmlrpc 2018-04-10 09:20:40 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:0677


Note You need to log in before you can comment on or make changes to this bug.