Bug 1257004

Summary:	use ioctl(SIOC[GS]IFHWADDR) to save/set MAC address of macvtap passthrough physdevs with an 802.1Qbh port profile
Product:	Red Hat Enterprise Linux 7	Reporter:	Laine Stump <laine>
Component:	libvirt	Assignee:	Laine Stump <laine>
Status:	CLOSED ERRATA	QA Contact:	Virtualization Bugs <virt-bugs>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	7.2	CC:	dyuan, honzhang, lmiksik, mzhan, rbalakri, sassmann
Target Milestone:	rc	Keywords:	Regression
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	libvirt-1.2.17-8.el7	Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:
Clones:	1261566 (view as bug list)		Environment:
Last Closed:	2015-11-19 06:52:45 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Laine Stump 2015-08-26 04:17:57 UTC

Before libvirt sets the MAC address of the physdev (the physical ethernet device) linked to a macvtap passthrough device, it always saves the previous MAC address to restore when the guest is finished (following a "leave nothing behind" policy). It has even done this for macvtap devices that have an 802.1Qbh port profile attached to them. It turns out that this is unnecessary, because the port profile Associate/Disassociate operations do that for us.

Beyond that, with a recent change to the way we retrieve the MAC address (new in RHEL7.2, thanks to the fix for Bug 1211758 (Bug 1113474 in RHEL6)), all attempts to start a macvtap passthrough device with an 802.1Qbh port profile attached to a Cisco VMFEX card (which uses the "enic" driver in the kernel) to fail.

Comment 2 Laine Stump 2015-08-26 05:17:22 UTC

Posted upstream for review:

https://www.redhat.com/archives/libvir-list/2015-August/msg00889.html

Comment 3 Laine Stump 2015-09-02 16:19:19 UTC

V2 is posted upstream for review:

  https://www.redhat.com/archives/libvir-list/2015-September/msg00014.html

Testing with V1 of the patch on actual hardware showed that libvirt *does* need to set the MAC address of the VF in some way (otherwise the 802.1Qbh "associate" command fails), so the patch (and the summary of this BZ) have been changed accordingly. The new patch appears to work properly both on Cisco VMFEX hardware (with an 802.1Qbh <virtualport> and on Intel 82576 hardware (an example of a more generic SRIOV implementation).

Comment 4 Laine Stump 2015-09-03 15:47:04 UTC

Fix pushed upstream:

commit 3ce08fab8477da2c76918329523a5e6a312cef06
Author: Laine Stump <laine>
Date:   Mon Aug 31 17:09:01 2015 -0400

    util: make virNetDev(Replace|Restore)MacAddress public functions
    
commit 46cf0cefa0168a0d929ca87010f59e1cba6c689b
Author: Laine Stump <laine>
Date:   Wed Aug 26 00:18:26 2015 -0400

    util: don't use netlink to save/set mac for macvtap+passthrough+802.1Qbh

Comment 7 hongming 2015-10-04 01:13:15 UTC

It can be reproduced in libvirt-1.2.17-7.el7.x86_64.

[root@cisco-b230m2-1 home]# rpm -q libvirt
libvirt-1.2.17-7.el7.x86_64

[root@cisco-b230m2-1 home]# virsh dumpxml r7.1
    <interface type='direct'>
      <source dev='enp6s0f1' mode='passthrough'/>
      <virtualport type='802.1Qbh'>
        <parameters profileid='hm-test'/>
      </virtualport>
      <model type='virtio'/>
    </interface>

[root@cisco-b230m2-1 home]# virsh list --all
 Id    Name                           State
----------------------------------------------------
 -     r7.1                           shut off

[root@cisco-b230m2-1 home]# virsh start r7.1
error: Failed to start domain r7.1
error: internal error: missing IFLA_VF_INFO in netlink response

============================================================================
It can not be reproduced in libvirt-1.2.17-8.el7.x86_64. The result is expected. Move its status to VERIFIED.


[root@cisco-b230m2-1 ~]# rpm -q libvirt qemu-kvm
libvirt-1.2.17-8.el7.x86_64
qemu-kvm-1.5.3-102.el7.x86_64


[root@cisco-b230m2-1 ~]# lspci|grep Eth
06:00.0 Ethernet controller: Cisco Systems Inc VIC Ethernet NIC (rev a2)
06:00.1 Ethernet controller: Cisco Systems Inc VIC SR-IOV VF (rev a2)
06:00.2 Ethernet controller: Cisco Systems Inc VIC SR-IOV VF (rev a2)


[root@cisco-b230m2-1 ~]# virsh nodedev-list --tree
computer
  |
  +- net_lo_00_00_00_00_00_00
  +- net_virbr0_nic_52_54_00_46_be_b8
  +- pci_0000_00_00_0
  +- pci_0000_00_01_0
  +- pci_0000_00_03_0
  |   |
  |   +- pci_0000_02_00_0
  |       |
  |       +- pci_0000_03_00_0
  |           |
  |           +- pci_0000_04_00_0
  |               |
  |               +- pci_0000_05_00_0
  |               |   |
  |               |   +- pci_0000_06_00_0
  |               |   |   |
  |               |   |   +- net_enp6s0f0_00_25_b5_00_00_00
  |               |   |    
  |               |   +- pci_0000_06_00_1
  |               |   |   |
  |               |   |   +- net_enp6s0f1_00_00_00_00_00_00
  |               |   |    
  |               |   +- pci_0000_06_00_2
  |               |       |
  |               |       +- net_enp6s0f2_00_00_00_00_00_00
  |               |        


[root@cisco-b230m2-1 ~]# virsh edit r7.1
Domain r7.1 XML configuration not changed.

    <interface type='direct'>
      <source dev='enp6s0f1' mode='passthrough'/>
      <virtualport type='802.1Qbh'>
        <parameters profileid='hm-test'/>
      </virtualport>
      <model type='virtio'/>
    </interface>

[root@cisco-b230m2-1 ~]# virsh start r7.1
Domain r7.1 started

[root@cisco-b230m2-1 ~]# virsh dumpxml r7.1
<domain type='kvm' id='4'>
  <name>r7.1</name>
   .....
    <interface type='direct'>
      <mac address='52:54:00:40:6f:1b'/>
      <source dev='enp6s0f1' mode='passthrough'/>
      <virtualport type='802.1Qbh'>
        <parameters profileid='hm-test'/>
      </virtualport>
      <target dev='macvtap0'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
   .....
</domain>

[root@cisco-b230m2-1 ~]# ip link show enp6s0f1
2: enp6s0f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000
    link/ether 52:54:00:40:6f:1b brd ff:ff:ff:ff:ff:ff

[root@cisco-b230m2-1 ~]# virsh save r7.1 r7.1.save

Domain r7.1 saved to r7.1.save

[root@cisco-b230m2-1 ~]# virsh restore r7.1.save
Domain restored from r7.1.save

[root@cisco-b230m2-1 ~]# virsh dumpxml r7.1|grep /interface -B10
    <interface type='direct'>
      <mac address='52:54:00:40:6f:1b'/>
      <source dev='enp6s0f1' mode='passthrough'/>
      <virtualport type='802.1Qbh'>
        <parameters profileid='hm-test'/>
      </virtualport>
      <target dev='macvtap0'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>


[root@cisco-b230m2-1 ~]# ip link show enp6s0f1
2: enp6s0f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000
    link/ether 52:54:00:40:6f:1b brd ff:ff:ff:ff:ff:ff

Comment 9 errata-xmlrpc 2015-11-19 06:52:45 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-2202.html