Bug 723026

Summary: MAC address conflict when using Channel Bonding and Cisco M81KR vNIC (Palo)
Product: Red Hat Enterprise Linux 6 Reporter: jon.benedict
Component: kernelAssignee: Andy Gospodarek <agospoda>
Status: CLOSED NOTABUG QA Contact: Red Hat Kernel QE team <kernel-qe>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 6.1CC: arozansk, peterm
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-07-22 15:26:08 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description jon.benedict 2011-07-18 19:11:11 UTC
Description of problem:
When using Channel Bonding + Cisco Palo, errors appear in /var/log/messages regarding the MAC address.  This occurs with both modes 1 and 6.  The config below is for mode 6.  Here is the actual error:

Jul 18 14:41:44 rhel-frontend-1 kernel: bonding: bond1: Warning: the permanent HWaddr of eth2 - 00:25:b5:26:a0:6f - is still in use by bond1. Set the HWaddr of eth2 to a different address to avoid conflicts.

We would expect that the driver would move the MAC address around as necessary.

---

Typical Bonding configuration:
[root@rhel-frontend-1 ~]# cat /etc/sysconfig/network-scripts/*eth2
DEVICE=eth2
MASTER=bond1
SLAVE=yes
USERCTL=no
BOOTPROTO=none
ONBOOT=yes
TYPE=Ethernet
[root@rhel-frontend-1 ~]# cat /etc/sysconfig/network-scripts/*eth3
DEVICE=eth3
MASTER=bond1
SLAVE=yes
USERCTL=no
BOOTPROTO=none
ONBOOT=yes
TYPE=Ethernet
[root@rhel-frontend-1 ~]# cat /etc/sysconfig/network-scripts/*bond1
DEVICE=bond1
BOOTPROTO=none
ONBOOT=yes
USERCTL=no
TYPE=bond
BONDING_OPTS="miimon=100 mode=6"

---

Bonding info:
[root@rhel-frontend-1 ~]# cat /proc/net/bonding/bond1
Ethernet Channel Bonding Driver: v3.6.0 (September 26, 2009)

Bonding Mode: adaptive load balancing
Primary Slave: None
Currently Active Slave: eth2
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0

Slave Interface: eth2
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:25:b5:26:a0:6f
Slave queue ID: 0

Slave Interface: eth3
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:25:b5:26:b3:6f
Slave queue ID: 0

---

Bonding as viewed by ifconfig:

[root@rhel-frontend-1 ~]# ifconfig bond1
bond1     Link encap:Ethernet  HWaddr 00:25:B5:26:A0:6F  
          inet6 addr: fe80::225:b5ff:fe26:a06f/64 Scope:Link
          UP BROADCAST RUNNING MASTER MULTICAST  MTU:1500  Metric:1
          RX packets:2702 errors:0 dropped:0 overruns:0 frame:0
          TX packets:33076 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:320020 (312.5 KiB)  TX bytes:2249606 (2.1 MiB)

[root@rhel-frontend-1 ~]# ifconfig eth2
eth2      Link encap:Ethernet  HWaddr 00:25:B5:26:A0:6F  
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:1348 errors:0 dropped:0 overruns:0 frame:0
          TX packets:16556 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:159840 (156.0 KiB)  TX bytes:1126110 (1.0 MiB)

[root@rhel-frontend-1 ~]# ifconfig eth3
eth3      Link encap:Ethernet  HWaddr 00:25:B5:26:B3:6F  
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:1354 errors:0 dropped:0 overruns:0 frame:0
          TX packets:16559 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:160180 (156.4 KiB)  TX bytes:1126148 (1.0 MiB)

---

UDEV rule:
[root@rhel-frontend-1 ~]# cat /etc/udev/rules.d/70-persistent-net.rules 
# This file was automatically generated by the /lib/udev/write_net_rules
# program, run by the persistent-net-generator.rules rules file.
#
# You can modify it, as long as you keep each rule on a single
# line, and change only the value of the NAME= key.

# PCI device 0x1137:0x0043 (enic)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:25:b5:26:a0:6f", ATTR{type}=="1", KERNEL=="eth*", NAME="eth2"

# PCI device 0x1137:0x0043 (enic)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:25:b5:26:a0:9f", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"

# PCI device 0x1137:0x0043 (enic)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:25:b5:26:b3:9f", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1"

# PCI device 0x1137:0x0043 (enic)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:25:b5:26:b3:6f", ATTR{type}=="1", KERNEL=="eth*", NAME="eth3"

---

Comment 2 Andy Gospodarek 2011-07-22 15:26:08 UTC
(In reply to comment #0)
> Description of problem:
> When using Channel Bonding + Cisco Palo, errors appear in /var/log/messages
> regarding the MAC address.  This occurs with both modes 1 and 6.  The config
> below is for mode 6.  Here is the actual error:
> 
> Jul 18 14:41:44 rhel-frontend-1 kernel: bonding: bond1: Warning: the permanent
> HWaddr of eth2 - 00:25:b5:26:a0:6f - is still in use by bond1. Set the HWaddr
> of eth2 to a different address to avoid conflicts.
> 

This message appears because eth2 was being removed from bond1.

The bond device will take-on the MAC address of the first slave that is added to it and if this slave is removed, it will not automatically change the address of the bond.  If you need to change the configuration permanently you will need to alter the MAC address of the bond so it matches the BIA of one of the slaves.

The only real way to workaround this would be in initscripts and would possibly interfere with the current link status of the bond.  I would not want this done automatically for users.  The message is there to indicate that the user may need to make a change if they permanently want the ethernet device removed from the bond and used for other purposes.