Description of problem:
The OVS bonding modes are different from the Linux Bonding modes. We should document the OVS bonding modes, how to set them properly, and the fine-tuning knobs that exist for the bonds. The docs should also compare Linux Bond modes to OVS bonding modes.

Version-Release number of selected component (if applicable):
OSP 7 GA


Actual results:
Confusion and chaos

Expected results:
Proper documentation

Additional info:

Bond Mode 1, 2, and 4 are concepts that apply to Linux Bonds only. OVS bonds, on the other hand, have the following modes:

active-backup (only one link is used at a time)
balance-slb (MAC address-based load balancing, LACP optional)
balance-tcp (layer 2-4 intelligent load balancing, LACP required)

The active-backup mode is equivalent to Linux mode 1 bonds, and does not require switch support or configuration.

The balance-slb mode is compatible with "static" bonds, which means that the switch is always configured to send traffic down all bond members. This is roughly equivalent to Linux Bonding in mode 2. LACP support on the switch is not required, however one does not get the protection against unidirectional link failure or link error detection that LACP provides.

The balance-tcp mode requires LACP, but provides robust link failure detection, additional link data at the switch level, and has better load balancing characteristics. This is roughly equivalent to Linux mode 4 bonds. LACP support and configuration is required on the switch.

The default bonding options will configure the host to send LACP packets to the switch. If an LACP link cannot be established (for instance if the switch is not configured for LACP), then the link will fall back to active-backup mode. This makes the default options safe to deploy on switches that aren't yet configured for LACP bonds. If LACP is later configured on the switches, then the bonds will automatically switch from active-backup to active-active LACP bonds.