Bug 1302166

Summary: MAC address of VF is not editable even when attached to host
Product: Red Hat Enterprise Linux 7 Reporter: David Jorm <djorm>
Component: kernelAssignee: Kamal Heib <kheib>
kernel sub component: NIC Drivers QA Contact: haidong li <haili>
Status: CLOSED ERRATA Docs Contact:
Severity: low    
Priority: medium CC: aquini, atragler, bugs, ckannan, danken, ddutile, djorm, dledford, jpirko, jshortt, jturner, kheib, kzhang, laine, mburman, mleitner, network-qe, shanim, tgraf, ylavi, zguo
Version: 7.0   
Target Milestone: pre-dev-freeze   
Target Release: 7.3   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: kernel-3.10.0-449.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1401837 (view as bug list) Environment:
Last Closed: 2016-11-03 15:13:26 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Network RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1274397    

Description David Jorm 2016-01-27 01:24:41 UTC 
Description of problem:
MAC addresses are left on hypervisor VFs after a VM has been removed and the MAC returned to the pool, causing potential MAC conflicts.

Version-Release number of selected component (if applicable):
oVirt 3.6, vdsm 4.17.13-0.el7.centos

How reproducible:
Every time

Steps to Reproduce:
1. Create a VM template with a NIC that uses PCI Passthrough (SR-IOV)
2. Instantiate and run a VM from this template
3. Shutdown and destroy the VM

Actual results:
The SR-IOV VF will retain the MAC address on the hypervisor node that executed the VM (show with "ip link"), but the MAC address will be returned to the pool and used again by oVirt for a new VM. This can cause MAC address conflicts, as multiple hypervisor nodes will have VFs with the same MAC.

Expected results:
MAC address on the VF is zeroed out.
Comment 1 Dan Kenigsberg 2016-01-27 13:37:55 UTC 
mac collision can take place only if the VF is assigned to a network with an IP address, which is not the most common use case for VFs. Lowering severity.
Comment 2 Chandrasekar Kannan 2016-01-28 21:05:05 UTC 
I suspect this thread might add some color to this issue.
https://www.redhat.com/archives/libvir-list/2015-December/msg00683.html
Comment 3 Meni Yakove 2016-03-14 09:59:17 UTC 
Chandrasekar, it seems that you are right:

When taking a VM down, libvirt attempts to reset the MAC address of its VF, and fails

Mar 14 11:52:00 puma22 kernel: igbvf 0000:05:10.0: enabling device (0000 -> 0002)
Mar 14 11:52:00 puma22 kernel: igb 0000:05:00.0: VF 0 attempted to override administratively set MAC address#012Reload the VF driver to resume operations
Mar 14 11:52:00 puma22 kernel: igbvf 0000:05:10.0: Intel(R) 82576 Virtual Function
Mar 14 11:52:00 puma22 kernel: igbvf 0000:05:10.0: Address: 00:1a:4a:16:20:0e

(this has an unfortunate side effect if two VFs on the same host where used for the same VM: kernel starts complaining periodically about
Mar 14 11:50:30 puma22 kernel: igb 0000:05:00.0: Spoof event(s) detected on VF 0
)
Comment 4 Dan Kenigsberg 2016-03-14 12:03:10 UTC 
David, apparently libvirt is already attempting to do what you ask, but it can fail due to various driver limitations. Which driver and kernel version have you been using?
Comment 5 David Jorm 2016-03-15 00:28:14 UTC 
I am using Mellanox 40G cards with the OFED driver (MLNX_OFED_LINUX-3.1-1.0.3-rhel7.2-x86_64) on CentOS 7.2.1511 with kernel version 3.10.0-327.4.4.el7.x86_64.
Comment 6 Dan Kenigsberg 2016-03-15 07:02:50 UTC 
David, sorry to have not asked this earlier: do you see similar to commit 3 in your /var/log/message? could you share it?

Laine, do you know where does this issue stand? Do we need to file bugs on each and every device driver in order to allow the host to clear MAC addresses of VFs?
Comment 7 Laine Stump 2016-03-15 14:43:31 UTC 
Yes, unfortunately each sriov-capable driver does its own validation of the supplied mac address, so each will need to be fixed separately. IIRC the Cisco enic driver may already allow setting a mac address of all 0's (I may be misremembering though).

Additionally, some drivers initialize the VFs' MAC addresses to 0, and some initialize them to random numbers. Personally I think that initializing them to a random number is a pox upon the Earth, but the one advantage is that when libvirt tries to restore the original MAC, it is successful (since the original MAC is some random non-zero value). This 0 vs. random behavior is even different for the same driver in different versions (e.g. the igb driver that's in RHEL6 vs. what's in RHEL7) and for the driver for different cards by the same manufacturer (igb (Intel 82576) in RHEL6 sets random MACs to VFs but in RHEL7 sets 0's. ixgbe (Intel 82599) currently sets random MACs in both RHEL6 and RHEL7, but at some point was setting 0 MACs (and someone actually *complained* about it, and it was "fixed" :-( )

(BTW - beware of the combination of NetworkManager and VFs with random (i.e. non-zero) MAC addresses - without an intervention in the NM config, it will run a dhclient for each of them and attempt to acquire an IP address)

So in conclusion, it's a big mess.
Comment 8 Laine Stump 2016-03-15 14:47:28 UTC 
Oh, and also I forgot to mention that the method used to set the MAC address of a VF varies according to driver. In particular, while other sriov-capable driver use a netlink message to the PF to set the MAC of a VF, the Cisco enic driver doesn't support that, so we must use ioctl(SIOCSIFHWADDR) (we can't use that on, e.g. igb/ixgbe, though, because you have to set the MAC for a vfio-assign VF after the host driver has already been detached (so there will be no netdev to use the SIOCSIFHWADDR on))
Comment 9 Yaniv Lavi 2016-03-16 09:19:08 UTC 
Do you have bugs open on these issues?
Do we need to open them on libvirt?
Comment 10 Laine Stump 2016-03-25 20:56:10 UTC 
I don't recall if the report about the mlx driver and 0 mac addresses was via a BZ or email. None of this is a problem in libvirt, it's all in the kernel.
Comment 12 Shany 2016-04-12 11:07:22 UTC 
The issue is fixed in upstream, commit - 6e52242.
Kamal will take it into 7.3
Comment 13 Don Dutile (Red Hat) 2016-04-12 14:49:55 UTC 
(In reply to Shany from comment #12)
> The issue is fixed in upstream, commit - 6e52242.
> Kamal will take it into 7.3

I will end up pulling this in during the rdma-rebase for 7.3,
as I plan to go up through 4.5, then look to pick through 4.6, or take
the entire 4.6 update in.
i.e., everything under drivers/infiniband/hw/mlx*, drivers/net/ethernet/mellanox/*
Comment 14 Kamal Heib 2016-06-14 17:29:10 UTC 
RDMA rebase for RHEL-7.3 resolves this bz.
Comment 15 Rafael Aquini 2016-06-21 00:57:24 UTC 
Patch(es) committed on kernel repository and an interim kernel build is undergoing testing
Comment 17 Rafael Aquini 2016-06-23 13:08:03 UTC 
Patch(es) available on kernel-3.10.0-449.el7
Comment 19 haidong li 2016-09-27 00:17:34 UTC 
The issue is verified on the mlx4 card:

[root@hp-dl380pg8-15 ~]# uname -a
Linux hp-dl380pg8-15.rhts.eng.pek2.redhat.com 3.10.0-510.el7.x86_64 #1 SMP Wed Sep 21 14:44:14 EDT 2016 x86_64 x86_64 x86_64 GNU/Linux
[root@hp-dl380pg8-15 ~]# 
[root@hp-dl380pg8-15 ~]# ethtool -i ens5
driver: mlx4_en
version: 2.2-1 (Feb 2014)
firmware-version: 2.33.5100
expansion-rom-version: 
bus-info: 0000:24:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: yes
[root@hp-dl380pg8-15 ~]# modprobe -r mlx4_en; modprobe -r mlx4_ib;  modprobe -r mlx4_core
[root@hp-dl380pg8-15 ~]#  modprobe mlx4_core num_vfs=2 probe_vf=2
[root@hp-dl380pg8-15 ~]# ip link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000
    link/ether 28:80:23:9d:46:20 brd ff:ff:ff:ff:ff:ff
3: eno2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
    link/ether 28:80:23:9d:46:21 brd ff:ff:ff:ff:ff:ff
4: ens2f0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
    link/ether 00:90:fa:2a:65:82 brd ff:ff:ff:ff:ff:ff
5: ens4f4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
    link/ether 00:07:43:2e:04:10 brd ff:ff:ff:ff:ff:ff
6: ens4f4d1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
    link/ether 00:07:43:2e:04:18 brd ff:ff:ff:ff:ff:ff
7: ens2f1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
    link/ether 00:90:fa:2a:65:86 brd ff:ff:ff:ff:ff:ff
8: eno3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
    link/ether 28:80:23:9d:46:22 brd ff:ff:ff:ff:ff:ff
9: eno4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
    link/ether 28:80:23:9d:46:23 brd ff:ff:ff:ff:ff:ff
12: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT qlen 1000
    link/ether 52:54:00:7c:e0:1e brd ff:ff:ff:ff:ff:ff
13: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN mode DEFAULT qlen 1000
    link/ether 52:54:00:7c:e0:1e brd ff:ff:ff:ff:ff:ff
14: ens5: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
    link/ether e4:1d:2d:22:66:d0 brd ff:ff:ff:ff:ff:ff
    vf 0 MAC 00:00:00:00:00:00, vlan 4095, spoof checking off, link-state auto
    vf 1 MAC 00:00:00:00:00:00, vlan 4095, spoof checking off, link-state auto
15: ens5d1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
    link/ether e4:1d:2d:22:66:d1 brd ff:ff:ff:ff:ff:ff
    vf 0 MAC 00:00:00:00:00:00, vlan 4095, spoof checking off, link-state auto
    vf 1 MAC 00:00:00:00:00:00, vlan 4095, spoof checking off, link-state auto
16: ens5f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000
    link/ether 52:11:3b:17:7f:ad brd ff:ff:ff:ff:ff:ff
17: ens5f1d1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000
    link/ether 56:ce:d2:4d:2c:c9 brd ff:ff:ff:ff:ff:ff
18: ens5f2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000
    link/ether f2:01:11:b1:46:85 brd ff:ff:ff:ff:ff:ff
19: ens5f2d1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000
    link/ether ea:ae:6e:a0:e9:89 brd ff:ff:ff:ff:ff:ff
[root@hp-dl380pg8-15 ~]# virsh start g1
Domain g1 started
[root@hp-dl380pg8-15 ~]# ethtool -i ens5f1
driver: mlx4_en
version: 2.2-1 (Feb 2014)
firmware-version: 2.33.5100
expansion-rom-version: 
bus-info: 0000:24:00.1
supports-statistics: yes
supports-test: yes
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: yes
[root@hp-dl380pg8-15 ~]# cat 1.xml 
<interface type='hostdev' managed='yes'>

    <mac address='fa:16:3e:11:af:fe'/>

    <source>

      <address type='pci' domain='0x0000' bus='0x24' slot='0x00' function='0x1'/>

    </source>

    <vlan>

      <tag id='190'/>

    </vlan>

    <alias name='hostdev0'/>

</interface>

[root@hp-dl380pg8-15 ~]# 
[root@hp-dl380pg8-15 ~]# virsh attach-device g1 1.xml 
Device attached successfully

[root@hp-dl380pg8-15 ~]# ip link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000
    link/ether 28:80:23:9d:46:20 brd ff:ff:ff:ff:ff:ff
3: eno2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
    link/ether 28:80:23:9d:46:21 brd ff:ff:ff:ff:ff:ff
4: ens2f0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
    link/ether 00:90:fa:2a:65:82 brd ff:ff:ff:ff:ff:ff
5: ens4f4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
    link/ether 00:07:43:2e:04:10 brd ff:ff:ff:ff:ff:ff
6: ens4f4d1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
    link/ether 00:07:43:2e:04:18 brd ff:ff:ff:ff:ff:ff
7: ens2f1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
    link/ether 00:90:fa:2a:65:86 brd ff:ff:ff:ff:ff:ff
8: eno3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
    link/ether 28:80:23:9d:46:22 brd ff:ff:ff:ff:ff:ff
9: eno4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
    link/ether 28:80:23:9d:46:23 brd ff:ff:ff:ff:ff:ff
12: virbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT qlen 1000
    link/ether 52:54:00:7c:e0:1e brd ff:ff:ff:ff:ff:ff
13: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN mode DEFAULT qlen 1000
    link/ether 52:54:00:7c:e0:1e brd ff:ff:ff:ff:ff:ff
14: ens5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000
    link/ether e4:1d:2d:22:66:d0 brd ff:ff:ff:ff:ff:ff
    vf 0 MAC 00:00:00:00:00:00, vlan 4095, spoof checking off, link-state auto
    vf 1 MAC 00:00:00:00:00:00, vlan 4095, spoof checking off, link-state auto
15: ens5d1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000
    link/ether e4:1d:2d:22:66:d1 brd ff:ff:ff:ff:ff:ff
    vf 0 MAC fa:16:3e:11:af:fe, vlan 190, spoof checking off, link-state auto
    vf 1 MAC 00:00:00:00:00:00, vlan 4095, spoof checking off, link-state auto
18: ens5f2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000
    link/ether f2:01:11:b1:46:85 brd ff:ff:ff:ff:ff:ff
19: ens5f2d1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000
    link/ether ea:ae:6e:a0:e9:89 brd ff:ff:ff:ff:ff:ff
20: vnet0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master virbr0 state UNKNOWN mode DEFAULT qlen 1000
    link/ether fe:de:ad:39:00:21 brd ff:ff:ff:ff:ff:ff
[root@hp-dl380pg8-15 ~]# virsh destroy g1
Domain g1 destroyed
[root@hp-dl380pg8-15 ~]# ip link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000
    link/ether 28:80:23:9d:46:20 brd ff:ff:ff:ff:ff:ff
3: eno2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
    link/ether 28:80:23:9d:46:21 brd ff:ff:ff:ff:ff:ff
4: ens2f0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
    link/ether 00:90:fa:2a:65:82 brd ff:ff:ff:ff:ff:ff
5: ens4f4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
    link/ether 00:07:43:2e:04:10 brd ff:ff:ff:ff:ff:ff
6: ens4f4d1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
    link/ether 00:07:43:2e:04:18 brd ff:ff:ff:ff:ff:ff
7: ens2f1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
    link/ether 00:90:fa:2a:65:86 brd ff:ff:ff:ff:ff:ff
8: eno3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
    link/ether 28:80:23:9d:46:22 brd ff:ff:ff:ff:ff:ff
9: eno4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
    link/ether 28:80:23:9d:46:23 brd ff:ff:ff:ff:ff:ff
12: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT qlen 1000
    link/ether 52:54:00:7c:e0:1e brd ff:ff:ff:ff:ff:ff
13: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN mode DEFAULT qlen 1000
    link/ether 52:54:00:7c:e0:1e brd ff:ff:ff:ff:ff:ff
14: ens5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000
    link/ether e4:1d:2d:22:66:d0 brd ff:ff:ff:ff:ff:ff
    vf 0 MAC 00:00:00:00:00:00, vlan 4095, spoof checking off, link-state auto
    vf 1 MAC 00:00:00:00:00:00, vlan 4095, spoof checking off, link-state auto
15: ens5d1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000
    link/ether e4:1d:2d:22:66:d1 brd ff:ff:ff:ff:ff:ff
    vf 0 MAC 00:00:00:00:00:00, vlan 4095, spoof checking off, link-state auto
    vf 1 MAC 00:00:00:00:00:00, vlan 4095, spoof checking off, link-state auto
18: ens5f2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000
    link/ether f2:01:11:b1:46:85 brd ff:ff:ff:ff:ff:ff
19: ens5f2d1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000
    link/ether ea:ae:6e:a0:e9:89 brd ff:ff:ff:ff:ff:ff
27: ens5f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000
    link/ether 12:a8:5c:4d:fd:45 brd ff:ff:ff:ff:ff:ff
28: ens5f1d1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000
    link/ether e6:50:c6:e8:22:d2 brd ff:ff:ff:ff:ff:ff
[root@hp-dl380pg8-15 ~]# ip link show | grep vf
    vf 0 MAC 00:00:00:00:00:00, vlan 4095, spoof checking off, link-state auto
    vf 1 MAC 00:00:00:00:00:00, vlan 4095, spoof checking off, link-state auto
    vf 0 MAC 00:00:00:00:00:00, vlan 4095, spoof checking off, link-state auto
    vf 1 MAC 00:00:00:00:00:00, vlan 4095, spoof checking off, link-state auto
[root@hp-dl380pg8-15 ~]# ip link set ens5 vf 0 mac 00:00:00:00:00:01
[root@hp-dl380pg8-15 ~]# ip link show | grep vf
    vf 0 MAC 00:00:00:00:00:01, vlan 4095, spoof checking off, link-state auto
    vf 1 MAC 00:00:00:00:00:00, vlan 4095, spoof checking off, link-state auto
    vf 0 MAC 00:00:00:00:00:00, vlan 4095, spoof checking off, link-state auto
    vf 1 MAC 00:00:00:00:00:00, vlan 4095, spoof checking off, link-state auto
[root@hp-dl380pg8-15 ~]# ip link set ens5 vf 0 mac 00:00:00:00:00:00
[root@hp-dl380pg8-15 ~]# 
[root@hp-dl380pg8-15 ~]# ip link show | grep vf
    vf 0 MAC 00:00:00:00:00:00, vlan 4095, spoof checking off, link-state auto
    vf 1 MAC 00:00:00:00:00:00, vlan 4095, spoof checking off, link-state auto
    vf 0 MAC 00:00:00:00:00:00, vlan 4095, spoof checking off, link-state auto
    vf 1 MAC 00:00:00:00:00:00, vlan 4095, spoof checking off, link-state auto
[root@hp-dl380pg8-15 ~]#
Comment 20 haidong li 2016-09-27 00:21:46 UTC 
Hi Rafael,
Do these patches include other kinds of nic driver such as ixgbe and sfc?Or just for mlx? thanks!
Comment 21 haidong li 2016-09-28 10:11:19 UTC 
Hi kamal,
There are a lot of patches in comment 16. Have other kinds of nics also been fixed for the bug?I saw some patchs are for cxgb4 and enic.Should I do some regression test on them too?
Comment 24 Kamal Heib 2016-09-28 13:51:26 UTC 
I'm the Mellanox on-site @Redhat. So I'm not aware of other drivers issues.

And for some reason I don't have access to comment 16 and the fix for this issue was in the mlx4 code (Please see below upstream commit):

commit 6e5224224faa50ec4c8949dcefadf895e565f0d1
Author: Jack Morgenstein <jackm.co.il>
Date:   Wed Mar 2 17:47:46 2016 +0200

    net/mlx4_core: Allow resetting VF admin mac to zero
    
    The VF administrative mac addresses (stored in the PF driver) are
    initialized to zero when the PF driver starts up.
    
    These addresses may be modified in the PF driver through ndo calls
    initiated by iproute2 or libvirt.
    
    While we allow the PF/host to change the VF admin mac address from zero
    to a valid unicast mac, we do not allow restoring the VF admin mac to
    zero. We currently only allow changing this mac to a different unicast mac.
    
    This leads to problems when libvirt scripts are used to deal with
    VF mac addresses, and libvirt attempts to revoke the mac so this
    host will not use it anymore.
    
    Fix this by allowing resetting a VF administrative MAC back to zero.
    
    Fixes: 8f7ba3ca12f6 ('net/mlx4: Add set VF mac address support')
    Signed-off-by: Jack Morgenstein <jackm.co.il>
    Reported-by: Moshe Levi <moshele>
    Signed-off-by: David S. Miller <davem>


Thanks,
Kamal
Comment 25 haidong li 2016-09-29 02:20:08 UTC 
Hi David,
Is this bug for most kinds of nics or only for mlx?
Comment 26 Jay Turner 2016-09-29 12:59:32 UTC 
Answering for David (I'm at IIX as well, and am most familiar with the issue).  We only hit this issue with Mellanox adapters, but we have only utilized Mellanox and Intel adapters, so a more complete answer is that all we know is Intel does not have the same issue (it does have some other issues related to playing around with the MAC address, but not this one.)
Comment 27 Zhang Kexin 2016-09-29 14:51:43 UTC 
Hi Jay, Intel card you mean igb, ixgbe and i40e? You don't care about other cards, right?
Comment 28 Jay Turner 2016-09-29 15:33:10 UTC 
And fm10k.
Comment 29 Don Dutile (Red Hat) 2016-09-29 16:32:27 UTC 
(In reply to haidong li from comment #25)
> Hi David,
> Is this bug for most kinds of nics or only for mlx?

This bug is for mlx4 only.

The other nics don't have this problem, if you read between the lines in  Laine's replies in c#7 & c#8 -- they reset the VF's mac's differently for different devices in libvirt, b/c of the device idiosyncracies (set intel nic VF's back to original, random-number seed; use SIOC interface for enic; set to 0 for others -- "it's a big mess".

The patches listed in c#16 were the RDMA core rebase, which included the upstream fix for mlx4, thus the reason it's listed for this bz.
No other VF-capable driver (other than mlx5) was included in the patch list.
Comment 30 haidong li 2016-09-30 06:07:47 UTC 
The bug is verified according to comment 19 and 26
Comment 35 errata-xmlrpc 2016-11-03 15:13:26 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2016-2574.html