in 2.6.18-164.13.1.el5

An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2010-0147.html

Though my understanding is that this fix should have been included with the 2.6.18-164.15.1 kernel update (based on the ERRATA notice above), I am still experiencing this issue.

The problem that I am having is exactly as described in https://bugzilla.redhat.com/show_bug.cgi?id=487763 (Comment #3, for example)

The setup I am testing is the following:
* 4 physical interfaces (eth0 - eth3) are bonded using balance-alg.
* tagged interfaces are created from the resulting bond0 (bond0.1, bond0.2, etc)
* bridges are created for each of those tagged interfaces (xenbr0.1, xenbr0.2, etc), and the bonds are added to those bridges.

My understanding is that each of the bridges should end up having all of the physical interface MAC addresses marked local, however only the master interface on the bond is marked as local, as shown below (all 4 interfaces in the below are physically located on the machine).

[root@prickle1 ~]# uname -a
Linux prickle1 2.6.18-164.15.1.el5xen #1 SMP Wed Mar 17 12:04:23 EDT 2010 x86_64 x86_64 x86_64 GNU/Linux
[root@prickle1 ~]# brctl showmacs xenbr0.1 | grep "a4:ba:db:09:2c:"
  1     a4:ba:db:09:2c:1c       yes                0.00
  1     a4:ba:db:09:2c:1e       no               111.87
  1     a4:ba:db:09:2c:84       no               111.87
  1     a4:ba:db:09:2c:86       no               111.87


I am able to do some testing/alterations to the machines that are having this issue, if there's something that I can do to either fix this or help with testing.  

Please also let me know any other information that would be helpful.

This looks like a different issue to me (vlans are involved). Please fill up new bz for this, and put me in cc. Thanks.

The VLANs appears to be irrelevant.  

This appears to be exactly the same issue described in bug 506770 which we have closed as a WONTFIX for the time being.  I would strongly recommend switching to mode 5 when using bonding + bridging in a virt environment.

Thanks for the recommendation, Andy.  I'll look into switching to mode 5 in this environment.

Bug 506770 appears to be offlimits (I get permission denied when I attempt to view it); is there a way that I might be able to get access to it to verify that it is, in fact, the same issue?

(In reply to comment #12)
> Thanks for the recommendation, Andy.  I'll look into switching to mode 5 in
> this environment.
> 
> Bug 506770 appears to be offlimits (I get permission denied when I attempt to
> view it); is there a way that I might be able to get access to it to verify
> that it is, in fact, the same issue?    

Sorry I haven't gotten back to you on this.  I can try to open it up, but I suspect I might not be able to make that happen.  If I cannot I'll post the relevant comments here.

imarlier,

The fundamental problem described in bug 506770 is that ALB mode's receive load balancing (RLB) is designed to function as if the bond device is an end-point, not a device that will be responsible for accepting traffic that will need to be forwarded to other devices.

(This is a bit confusing and complex, so for clarity I will use 'bridge' to mean the bridge internal to the host that contains the bond and guest interfaces and I will use 'switch' to mean another device that is connected to the slaves in the bond.)

The patch added in this bug addresses the issue where traffic will be destined for a MAC address that belongs to one of the slaves in the bond, so it will not be properly received by the bridge (since the bridge will have the same MAC address as the first member of the bond).  This patch rewrites the frames as they come into the host and puts the mac address of the bond as the destination MAC.  This will resolve the case where the bridge that has an IP address will not maintain connectivity, but it will not prevent hosts connected via other members of the bridge (like veth0) to maintain connectivity (what bug 506770 was about).

Right now, the RLB code keeps track of all hosts that have sent frames on the bond recently (in the last 2 seconds) and attempts to balance the traffic by sending gratuitous ARPs with the source MAC address of the transmitting slave if balancing is needed.  All ARP responses will also have the MAC of the transmitting slave as the source MAC for ARP traffic coming out of the bond.

You can quickly see how the MAC of a guest may be overwritten when the bonding code starts controlling ARP traffic.  Once other hosts receive the slave device's MAC addresses as the MAC for the guest's IP, connectivity will be lost until the guest starts sending ARP frames again.  I created some patches that would allow the RLB code to maintain knowledge of the guest's MACs and not overwrite them with the slaves' MAC addresses when sending ARPs, but this still did not completely address the problem.  The possibility still exists that a single broadcast (unlearned or directed) frame could result in a loss of connectivity to the guest until the guest begins sending ARP requests.

The fact that there is no easy way to dynamically tell the bridge what addresses are considered local (the problem you are seeing), and there is no way to easily tell the bridge what guest MAC addresses should not be relearned on the bond0 interface in the bridge, supporting ALB mode is not possible.  It may be possible to write some ebtables rules to make it work, but that is not a dynamic or scalable solution.