Bug 516569

Summary: Bridge got disconnected from Bond at reboot
Product: Red Hat Enterprise Linux 5 Reporter: Michael Pasternak <mpastern>
Component: initscriptsAssignee: initscripts Maintenance Team <initscripts-maint-list>
Status: CLOSED ERRATA QA Contact: qe-baseos-daemons
Severity: urgent Docs Contact:
Priority: urgent    
Version: 5.6CC: apevec, azelinka, bazulay, cpelland, danken, dpaikov, harald, jcavallaro, jscotka, liko, llim, notting, ovirt-maint, pcao, plyons, Rhev-m-bugs, riek, schlegel, vbian, ycui, ykaul
Target Milestone: rcKeywords: Reopened, ZStream
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: initscripts-8.45.32-1.el5 Doc Type: Bug Fix
Doc Text:
Due to an incorrect loading of the bonding driver module, certain systems may not have been able to activate a bonding interface. This error has been fixed, and the bonding driver is now correctly loaded when an ifcfg script contains the "BONDING_OPTS" option.
Story Points: ---
Clone Of:
: 564300 (view as bug list) Environment:
Last Closed: 2011-01-13 23:05:22 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 564300, 600484, 627924, 630936, 652250    
Attachments:
Description Flags
All Logs and Conf.files (in WinRar archive) none

Description Michael Pasternak 2009-08-10 14:01:04 UTC
Created attachment 356896 [details]
All Logs and Conf.files (in WinRar archive)

Description of problem:
Bridge got disconnected from Bond after Ovirt reboot

Version-Release number of selected component (if applicable):
2.1,34149 sp180

How reproducible:

Steps to Reproduce:
1. creaet bond with sw bridge attached to it:
(we using '/usr/share/vdsm/addNetwork' script)

nicX ->
	|
	| -> BondX -> BridgeX
	|
nicY ->

2. reboot Ovirt
3. After Ovirt come's up, at 'brctl show' you'll see that BridgeX not connected to BondX
  
Actual results:
'Device bond0 does not seem to be present, delaying initialization.'

Additional info:

see attached conf. && 'ifup-eth' output

Comment 2 Alan Pevec 2009-08-10 20:39:34 UTC
Please try to recreate the problem on RHEL 5.4 beta and reassign to RHEL/initscripts component.

> (we using '/usr/share/vdsm/addNetwork' script)
I'm reassigning to vdsm and danken to validate your usage of addNetwork - please provide complete steps to reproduce

Comment 3 Alan Pevec 2009-08-10 20:41:45 UTC
And please avoid rar, unrar is not in Fedora :)

Here are the ifcfg-* from the attached rar archive, for easier viewing:
::::::::::::::
ifcfg-bond0
::::::::::::::
DEVICE=bond0
ONBOOT=yes
BOOTPROTO=none
BRIDGE=sw2
::::::::::::::
ifcfg-eth2
::::::::::::::
DEVICE=eth2
ONBOOT=yes
BOOTPROTO=none
HWADDR=00:1D:09:68:71:4E
MASTER=bond0
SLAVE=yes
::::::::::::::
ifcfg-eth3
::::::::::::::
DEVICE=eth3
ONBOOT=yes
BOOTPROTO=none
HWADDR=00:1D:09:68:71:50
MASTER=bond0
SLAVE=yes
::::::::::::::
ifcfg-sw2
::::::::::::::
DEVICE=sw2
TYPE=Bridge
ONBOOT=yes
GATEWAYDEV=none
DELAY=0
STP=no

Comment 4 Dan Kenigsberg 2009-08-11 07:36:19 UTC
Michael has shown me the machine - the ifcfg-* scripts are well-persisted and appear after reboot.

The issue is with initscripts: on reboot, network service tries to bring bond0 up and into sw2 bridge. However, as one can see in the attached ifup-eth.output.txt, bond0 is not yet present and that operation fails. few minutes later, manual ifup bond0 succeeds.

Maybe it is worth for ifup-eth to wait a bit for bonding devices, their initialisation is slower than that of nics.

btw, the make of nicX and nicY is "Broadcom Corporation NetXtreme II BCM5708 Gigabit Ethernet (rev 12)"

Comment 6 Michael Pasternak 2009-08-11 13:36:04 UTC
Apparently IT guys configured by mistake 4 switch ports in to single bond [21,22,23,24], while should two ports in to bonds, cause two Ovirt’s where connected like this:

Ovirt A : BondA : eth2/eth3 – switch[21/22]
Ovirt B : BondA : eth2/eth3 – switch[23/24]

and its caused server’s bonds to come up a bit longer then usual

Comment 9 Michael Pasternak 2009-08-12 06:43:35 UTC
Agree, sorry forgot to remove it with my last comment

Comment 10 Vivian Bian 2009-08-20 12:59:02 UTC
I tried with the following configuration on RHEL5.4 , after reboot sw2 is still on bond0 . Details will be described as follows :

the topology is still :
nicX ->
 |
 | -> BondX -> BridgeX
 |
nicY ->

and the configure files are :
::::::::::::::
ifcfg-bond0
::::::::::::::
DEVICE=bond0
ONBOOT=yes
BOOTPROTO=none
BRIDGE=sw2
::::::::::::::
ifcfg-eth2
::::::::::::::
DEVICE=eth2
ONBOOT=yes
BOOTPROTO=none
HWADDR=00:1D:09:68:71:4E
MASTER=bond0
SLAVE=yes
::::::::::::::
ifcfg-eth3
::::::::::::::
DEVICE=eth3
ONBOOT=yes
BOOTPROTO=none
HWADDR=00:1D:09:68:71:50
MASTER=bond0
SLAVE=yes
::::::::::::::
ifcfg-sw2
::::::::::::::
DEVICE=sw2
TYPE=Bridge
ONBOOT=yes
GATEWAYDEV=none
DELAY=0
STP=no  

After editing all the files, I did :
# echo "alias bond0 bonding"  >> /etc/modprobe.conf
# modprobe bonding
# service network restart
# ifconfig
  bond0     Link encap:Ethernet  HWaddr 00:1B:21:1D:2D:3C  
          UP BROADCAST RUNNING MASTER MULTICAST  MTU:1500  Metric:1
          RX packets:3449 errors:0 dropped:0 overruns:0 frame:0
          TX packets:110 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:239418 (233.8 KiB)  TX bytes:10128 (9.8 KiB)
eth0      Link encap:Ethernet  HWaddr 00:1B:21:1D:2D:3C  
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:1725 errors:0 dropped:0 overruns:0 frame:0
          TX packets:55 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:119573 (116.7 KiB)  TX bytes:5407 (5.2 KiB)
          Base address:0x1100 Memory:d8800000-d8820000 

eth1      Link encap:Ethernet  HWaddr 00:1B:21:1D:2D:3C  
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:1724 errors:0 dropped:0 overruns:0 frame:0
          TX packets:55 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:119845 (117.0 KiB)  TX bytes:4721 (4.6 KiB)
          Base address:0x1140 Memory:d8840000-d8860000 
lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:156 errors:0 dropped:0 overruns:0 frame:0
          TX packets:156 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:18896 (18.4 KiB)  TX bytes:18896 (18.4 KiB)
sw2       Link encap:Ethernet  HWaddr 00:1B:21:1D:2D:3C  
          inet addr:192.168.1.101  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::21b:21ff:fe1d:2d3c/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1175 errors:0 dropped:0 overruns:0 frame:0
          TX packets:108 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:86428 (84.4 KiB)  TX bytes:9960 (9.7 KiB)

And then create a new Host in RHEVM , which is the same IP with sw2 . Will start the installing process. After the installation, this RHEL will reboot . 
after reboot

# brctl show 
  bridge name        bridge id             STP enabled           interface
  sw2                8000,001b211d2d3c     no                    bond0 
also , ifconfig will show the same result as above.

Comment 11 Vivian Bian 2009-08-20 13:05:55 UTC
(In reply to comment #10)
this comment is for :
 Linux 2.6.18-162.el5xen #1 SMP Tue Aug 4 16:32:26 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux

 Red Hat Enterprise Linux Server release 5.4 (Tikanga)

Comment 12 Vivian Bian 2009-08-20 13:07:45 UTC
(In reply to comment #10)
so as for me after reboot , Bridge got connected from Bond at reboot -- we can get the correct result on RHEL5u4.

Comment 13 Vivian Bian 2009-08-24 12:43:41 UTC
checked with rhev-hypervisor-5.4-2.0.99.16.2.el5rhev and RHEVM_34758.exe . Also with reported configuration files. After reboot could still show the bridge device is on the bonding device. 
# brctl show 
bridge name     bridge id               STP enabled     interfaces
rhevm           8000.001f2903867f       no              bond0

On RHEVH, you should do like this :
1.configure the file ifcfg-ethX as you reported. 
2.configure the bonding device as reported
3.configure the bridge device must called rhevm. And make the bonding device insert on it. 
4.modprobe bond0 (the bonding module must be loaded)
5.persist all the configuration files.
6.Regist to VDC
7.Reboot RHEVH
8.check with brctl show 

Actual result
rhevm bridge is still on the bonding device. 

So I think this is not a bug, or at least it works for me. Close it for now, any difference, please reopen it.

Comment 14 Dan Kenigsberg 2009-08-30 08:20:51 UTC
Seems like your verification process is missing a crucial step:
0. configure switch as described in comment 6

I wouldn't have astounded if you decided to close this bug as esoteric, but if we decide to dig into it, let us do it properly.

Comment 15 Vivian Bian 2009-08-30 09:52:10 UTC
(In reply to comment #6)
> Apparently IT guys configured by mistake 4 switch ports in to single bond
> [21,22,23,24], while should two ports in to bonds, cause two Ovirt’s where
> connected like this:
> 
> Ovirt A : BondA : eth2/eth3 – switch[21/22]
> Ovirt B : BondA : eth2/eth3 – switch[23/24]
> 
> and its caused server’s bonds to come up a bit longer then usual  
 
  Hi Michael,
  would you please show me the detail configuration on the switch? And by this comment you mean there are two oVirt on the same bond before. But should be separated now, right ? I wonder why need to prepare two oVirt ? Is it OK for only one ovirt ? And should the connect like this ?
 
  Ovirt A : BondA : eth2/eth3 – switch[21/22]
  Ovirt B : BondB : eth2/eth3 – switch[23/24]

  Or if they both in bondA, what's the difference between
 
  OvirtA |
         |--BondA - switch [21/22/23/24] ? 
  OvirtB |

Comment 16 Alan Pevec 2009-10-20 13:17:55 UTC
*** Bug 529557 has been marked as a duplicate of this bug. ***

Comment 17 Alan Pevec 2009-10-20 13:19:47 UTC
*** Bug 529714 has been marked as a duplicate of this bug. ***

Comment 18 Alan Pevec 2009-10-20 13:27:09 UTC
(In reply to comment #10)
> After editing all the files, I did :
> # echo "alias bond0 bonding"  >> /etc/modprobe.conf

This does the trick, current initscript requires bond alias to be defined in
order to load it on boot.

Bill, why is that needed, shouldn't initscript just always modprobe bonding if
BONDING_OPTS is defined?

Comment 19 Daniel Paikov 2009-10-20 15:00:04 UTC
Please note that the "alias bond0 bonding" workaround would be problematic on RHEVH, since a RHEVH user shouldn't be able to create or change files.

Comment 20 Bill Nottingham 2009-10-20 15:35:12 UTC
Does this work for you?

diff --git a/sysconfig/network-scripts/network-functions b/sysconfig/network-scripts/network-
index 1583a3a..035f5a1 100644
--- a/sysconfig/network-scripts/network-functions
+++ b/sysconfig/network-scripts/network-functions
@@ -203,7 +203,7 @@ END {
        fi
     fi
 
-    if [ ${alias} == "bonding" ]; then
+    if [ ${alias} == "bonding" -o -n "$BONDING_OPTS" ]; then
        install_bonding_driver $1
     fi

Comment 21 Bill Nottingham 2009-10-20 20:56:35 UTC
Actually, try this:

diff --git a/sysconfig/network-scripts/network-functions b/sysconfig/network-scripts/network-funct
index 1583a3a..43c8606 100644
--- a/sysconfig/network-scripts/network-functions
+++ b/sysconfig/network-scripts/network-functions
@@ -180,6 +180,8 @@ is_available ()
     LC_ALL= LANG= ip -o link | grep -q $1
     [ "$?" = "1" ] || return 0
 
+    [ -n "$BONDING_OPTS" ] && install_bonding_driver $1
+
     alias=$(modprobe -c | awk \
 'BEGIN { alias = ""; }
 $1 == "alias" && $2 == "'"$1"'" { alias = $3; }

Comment 23 Alan Pevec 2009-11-04 23:51:44 UTC
(In reply to comment #21)
> Actually, try this:
> +    [ -n "$BONDING_OPTS" ] && install_bonding_driver $1
> +

This seems to work, I got sw2 up on boot, I just got one permission error:
Bringing up loopback interface:  [  OK  ]
Bringing up interface bond0:  ./network-functions: line 397: echo: write error: Operation not permitted
[  OK  ]
Bringing up interface eth0:  [  OK  ]
Bringing up interface breth0:  
Determining IP information for breth0... done.
[  OK  ]
Bringing up interface sw2:  [  OK  ]


Line 397 is echo in install_bonding_driver() function:
install_bonding_driver ()
{
   ethtool -i $1 2>/dev/null | grep -q "driver: bonding" && return 0
   [ ! -f /sys/class/net/bonding_masters ] && ( modprobe bonding || return 1 )
   echo "+$1" > /sys/class/net/bonding_masters
   return 0
}

It's a timing issue, /sys/class/net/bonding_masters is not created yet and it's not boot-time specific, the same issue is seen with:
# rmmod bonding; service network restart

There would need to be a loop waiting for /sys/class/net/bonding_masters to show up...
But is that echo necessary? Seems bond0 is added to bonding_masters even with that echo line commented out.

Comment 24 Alan Pevec 2009-11-05 00:07:12 UTC
(In reply to comment #23)
> There would need to be a loop waiting for /sys/class/net/bonding_masters to
> show up...

I take that back, /sys/class/net/bonding_masters is present in time (verified with ls before echo line) so Permission error is something else.

Comment 25 Alan Pevec 2009-11-05 00:11:52 UTC
Ah there we go, from /var/log/messages:
Nov  5 00:00:37 nodel5 kernel: bonding: bond0 is being created...
Nov  5 00:00:37 nodel5 kernel: bonding: cannot add bond bond0; it already exists
Nov  5 00:00:37 nodel5 kernel: bonding: Bond creation failed.

So that echo line should really be dropped, bond0 is automatically added AFAICT.

Comment 26 Bill Nottingham 2009-11-06 19:23:23 UTC
No, just redirect it to /dev/null.

Comment 35 RHEL Program Management 2010-06-04 20:19:48 UTC
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux maintenance release.  Product Management has requested
further review of this request by Red Hat Engineering, for potential
inclusion in a Red Hat Enterprise Linux Update release for currently deployed
products.  This request is not yet committed for inclusion in an Update
release.

Comment 36 Harald Hoyer 2010-09-13 13:16:49 UTC
(In reply to comment #26)
> No, just redirect it to /dev/null.

redirecting the echo to /dev/null would not prevent the kernel messages :)

Comment 39 Dan Kenigsberg 2010-10-29 11:49:17 UTC
Avi, would you verify that this bug is fixed?

Comment 40 Avi Tal 2010-11-01 11:12:43 UTC
Sorry but i do not have the resources in order to check this bug!

Comment 42 Jaromir Hradilek 2010-11-12 16:03:15 UTC
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
Due to an incorrect loading of the bonding driver module, certain systems may not have been able to activate a bonding interface. This error has been fixed, and the bonding driver is now correctly loaded when an ifcfg script contains the "BONDING_OPTS" option.

Comment 43 Pengzhen Cao 2010-11-15 12:46:05 UTC
Repro the issue with rhevh build 5.6-2.2.0.1, verified the fix with latest rhevh 5.6-2.2-5.1 build. 

Manually check file "/etc/sysconfig/network-scripts/network-functions" also found that the following 2 changset present in rhevh 5.6-2.2-5.1 build :
http://git.fedorahosted.org/git/?p=initscripts.git;a=commitdiff;h=dfdb699ff30d0a5dc799d79de70150823a6942ea
http://git.fedorahosted.org/git/?p=initscripts.git;a=commitdiff;h=86bc305

logs:
[root@localhost network-scripts]# cat ifcfg-bond0 
DEVICE=bond0
USERCTL=no
BOOTPROTO=none
ONBOOT=yes
BONDING_MASTER=yes
BONDING_OPTS="mode=4 miimon=100"
BONDING_SLAVE0=eth0
BONDING_SLAVE1=eth2
BRIDGE=brbond0
[root@localhost network-scripts]# cat ifcfg-brbond0 
DEVICE=brbond0
TYPE=Bridge
BOOTPROTO=static
ONBOOT=yes
IPADDR=192.168.2.106
NETMASK=255.255.255.0
GATEWAY=192.168.2.1
DELAY=0
STP=no
[root@localhost network-scripts]# cat ifcfg-eth0    
DEVICE=eth0
HWADDR=00:1b:21:3c:f0:1c
BOOTPROTO=none
MASTER=bond0
SLAVE=yes
[root@localhost network-scripts]# cat ifcfg-eth2
DEVICE=eth2
HWADDR=00:0e:0c:b0:95:6a
BOOTPROTO=none
MASTER=bond0
SLAVE=yes
[root@localhost network-scripts]# cat ifcfg-eth1
DEVICE=eth1
BRIDGE=breth1
ONBOOT=yes
HWADDR=00:23:7d:53:ab:75
[root@localhost network-scripts]# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.4.0-1 (October 7, 2008)

Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer2 (0)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0

802.3ad info
LACP rate: slow
Active Aggregator Info:
	Aggregator ID: 1
	Number of ports: 1
	Actor Key: 17
	Partner Key: 1
	Partner Mac Address: 00:00:00:00:00:00

Slave Interface: eth0
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:1b:21:3c:f0:1c
Aggregator ID: 1

Slave Interface: eth2
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:0e:0c:b0:95:6a
Aggregator ID: 2
[root@localhost network-scripts]# ifconfig
bond0     Link encap:Ethernet  HWaddr 00:1B:21:3C:F0:1C  
          UP BROADCAST RUNNING MASTER MULTICAST  MTU:1500  Metric:1
          RX packets:1392 errors:0 dropped:0 overruns:0 frame:0
          TX packets:22 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:93066 (90.8 KiB)  TX bytes:2728 (2.6 KiB)

brbond0   Link encap:Ethernet  HWaddr 00:1B:21:3C:F0:1C  
          inet addr:192.168.2.106  Bcast:192.168.2.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:509 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:28115 (27.4 KiB)  TX bytes:252 (252.0 b)

breth1    Link encap:Ethernet  HWaddr 00:23:7D:53:AB:75  
          inet addr:10.66.72.91  Bcast:10.66.73.255  Mask:255.255.254.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1731 errors:0 dropped:0 overruns:0 frame:0
          TX packets:959 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:123008 (120.1 KiB)  TX bytes:138645 (135.3 KiB)

eth0      Link encap:Ethernet  HWaddr 00:1B:21:3C:F0:1C  
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:696 errors:0 dropped:0 overruns:0 frame:0
          TX packets:11 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:46533 (45.4 KiB)  TX bytes:1364 (1.3 KiB)

eth1      Link encap:Ethernet  HWaddr 00:23:7D:53:AB:75  
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1951 errors:0 dropped:0 overruns:0 frame:0
          TX packets:943 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:168302 (164.3 KiB)  TX bytes:139941 (136.6 KiB)
          Interrupt:201 Memory:d8900000-d8910000 

eth2      Link encap:Ethernet  HWaddr 00:1B:21:3C:F0:1C  
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:696 errors:0 dropped:0 overruns:0 frame:0
          TX packets:11 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:46533 (45.4 KiB)  TX bytes:1364 (1.3 KiB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:420 errors:0 dropped:0 overruns:0 frame:0
          TX packets:420 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:61257 (59.8 KiB)  TX bytes:61257 (59.8 KiB)


So verify the fix in rhevh 5.6 build

Comment 47 Pengzhen Cao 2010-11-16 09:03:25 UTC
also verified the fix on latest rhel 5.6 build, 
kernel is 2.6.18-231_x86_64,
initscripts rpm is 8.45-33-1.el5


[root@localhost ~]# uname -a
Linux localhost.localdomain 2.6.18-231.el5 #1 SMP Mon Nov 8 18:36:02 EST 2010 x86_64 x86_64 x86_64 GNU/Linux

[root@localhost ~]# rpm -q initscripts
initscripts-8.45.33-1.el5


[root@localhost network-scripts]# cat ifcfg-brbond0 
DEVICE=brbond0
TYPE=Bridge
BOOTPROTO=static
ONBOOT=yes
IPADDR=192.168.2.106
NETMASK=255.255.255.0
#GATEWAY=192.168.2.1
DELAY=0
STP=no
[root@localhost network-scripts]# cat ifcfg-bond0
DEVICE=bond0
USERCTL=no
BOOTPROTO=none
ONBOOT=yes
BONDING_MASTER=yes
BONDING_OPTS="mode=4 miimon=100"
BONDING_SLAVE0=eth0
BONDING_SLAVE1=eth1
BRIDGE=brbond0
[root@localhost network-scripts]# cat ifcfg-eth0
# Intel Corporation 82541PI Gigabit Ethernet Controller
DEVICE=eth0
HWADDR=00:1B:21:3C:F0:1C
BOOTPROTO=none
MASTER=bond0
SLAVE=yes
[root@localhost network-scripts]# cat ifcfg-eth1
# Intel Corporation 82541PI Gigabit Ethernet Controller
DEVICE=eth1
HWADDR=00:0E:0C:B0:95:6A
BOOTPROTO=none
MASTER=bond0
SLAVE=yes
[root@localhost network-scripts]# cat ifcfg-eth2
# Broadcom Corporation NetXtreme BCM5755 Gigabit Ethernet PCI Express
DEVICE=eth2
BOOTPROTO=dhcp
HWADDR=00:23:7D:53:AB:75
ONBOOT=yes

[root@localhost network-scripts]# cat /proc/net/bonding/bond0 
Ethernet Channel Bonding Driver: v3.4.0-1 (October 7, 2008)

Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer2 (0)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0

802.3ad info
LACP rate: slow
Active Aggregator Info:
	Aggregator ID: 5
	Number of ports: 2
	Actor Key: 17
	Partner Key: 3
	Partner Mac Address: 00:1e:49:88:f4:80

Slave Interface: eth0
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:1b:21:3c:f0:1c
Aggregator ID: 5

Slave Interface: eth1
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:0e:0c:b0:95:6a
Aggregator ID: 5



[root@localhost ~]# ifconfig
bond0     Link encap:Ethernet  HWaddr 00:1B:21:3C:F0:1C  
          UP BROADCAST RUNNING MASTER MULTICAST  MTU:1500  Metric:1
          RX packets:932 errors:0 dropped:0 overruns:0 frame:0
          TX packets:163 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:71620 (69.9 KiB)  TX bytes:17889 (17.4 KiB)

brbond0   Link encap:Ethernet  HWaddr 00:1B:21:3C:F0:1C  
          inet addr:192.168.2.106  Bcast:192.168.2.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:30 errors:0 dropped:0 overruns:0 frame:0
          TX packets:19 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:3490 (3.4 KiB)  TX bytes:2874 (2.8 KiB)

eth0      Link encap:Ethernet  HWaddr 00:1B:21:3C:F0:1C  
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:808 errors:0 dropped:0 overruns:0 frame:0
          TX packets:68 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:58099 (56.7 KiB)  TX bytes:6702 (6.5 KiB)

eth1      Link encap:Ethernet  HWaddr 00:1B:21:3C:F0:1C  
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:124 errors:0 dropped:0 overruns:0 frame:0
          TX packets:95 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:13521 (13.2 KiB)  TX bytes:11187 (10.9 KiB)

eth2      Link encap:Ethernet  HWaddr 00:23:7D:53:AB:75  
          inet addr:10.66.72.91  Bcast:10.66.73.255  Mask:255.255.254.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:84 errors:0 dropped:0 overruns:0 frame:0
          TX packets:51 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:10133 (9.8 KiB)  TX bytes:9639 (9.4 KiB)
          Interrupt:193 Memory:d8900000-d8910000 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:2085 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2085 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:3393548 (3.2 MiB)  TX bytes:3393548 (3.2 MiB)

[root@localhost ~]# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
192.168.2.0     0.0.0.0         255.255.255.0   U     0      0        0 brbond0
10.66.72.0      0.0.0.0         255.255.254.0   U     0      0        0 eth2
169.254.0.0     0.0.0.0         255.255.0.0     U     0      0        0 brbond0
0.0.0.0         10.66.73.254    0.0.0.0         UG    0      0        0 eth2

Comment 49 errata-xmlrpc 2011-01-13 23:05:22 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2011-0075.html