Consistent-hashing algorithm in load balancing enables ingress traffic to be distributed among backend servers according to a hash scheme that is computed based on given fields (single or compound). For example: source, destination or both IP addresses, transport protocol, requested URL, etc. OVS supports a set of OpenFlow "select" groups that could facilitate the programming of flows. Particularly, the selection_method=hash. From the OVS documentation, it can hash fields like "source and destination Ethernet address, VLAN ID, Ethernet type, IPv4/v6 source and destination address and protocol, and for TCP and SCTP only, the source and destination ports." Also interesting to explore would be the field "weight=" under bucket parameters to enable connection drainage of server backends (e.g. to stop forwarding new flows to given server backends). A more advanced use-case: active-active load balancing topology. OVN load balancers on Tier 1 (outer tier), Octavia amphora or 3rd party vendors load balancers on Tier 2 (inner tier). There's a patch open in Octavia implementing an active-active topology where the distributor (Tier 1) is backed by OpenFlow: https://review.opendev.org/#/c/427858/
OpenShift would also like slightly different hashing capability for LBs in UDP scenarios. It would like to use [srcip, dstip, dstport] (eg *not* srcport) for specific LB VIP entries, but allow other LB VIPs to use the normal 4-tuple.
Related affinity issues/bugs from the OpenShift side: https://issues.redhat.com/browse/SDN-510 https://bugzilla.redhat.com/show_bug.cgi?id=1707513
Clarification: OpenShift would like to optionally use [srcip, dstip, dstport] for both TCP and UDP for a given LB VIP. If traffic matching that 3-tuple has not been seen within X seconds the affinity can be cleared and recalculated when new traffic comes to the VIP.
OVN load balancer is using dp_hash as a selection method for choosing the bucket (which translates to a backend). Although the commit message in the ovs says, dp_hash uses 5-tuple hash, I didn't see that in my testing. For a given L4 pkt, the dp_hash changes and this results in bucket selection non deterministic. There are few issues with the current OVN implementation. More details here - https://mail.openvswitch.org/pipermail/ovs-discuss/2020-April/049940.html If we change from dp_hash to "hash", then ovs-vswitchd will use 5-tuple hash. I did some testing and I can confirm that with this for a given L4 packet, the bucket selection is deterministic i.e the same bucket is chosen all the time. Is opensift OK changing the selection_method from dp_hash to hash ? Also I think we can most likely support a hash of srcip, dstip and dst port. I'm looking into this and I'm planning to expose the hash method to CMS so that it can chose - i.e SRC_IP_PORT, OR SRC_IP_DST_IP_DST_PORT etc.
I don't think ovnkube/OpenShift care what the hash algorithm is, but I think it's useful to have both options. For example, OpenShift is OK with the more random nature of the current hash as that has randomness advantages. But for specific LB VIPs it would like to use a more deterministic algorithm like SRC_IP_DST_IP_DST_PORT, but only certain ones chosen by the cluster users.
Ok. Thanks. I'll not change from dp_hash to hash. Instead provide the option for CMS to choose with the default being dp_hash.
Fix available in 2.13.0-28
I create a env as follow script: ovn-nbctl ls-add ls1 ovn-nbctl lsp-add ls1 ls1p1 ovn-nbctl lsp-set-addresses ls1p1 00:01:02:01:01:01 ovn-nbctl lsp-add ls1 ls1p2 ovn-nbctl lsp-set-addresses ls1p2 00:01:02:01:01:02 ovn-nbctl lsp-add ls1 ls1p3 ovn-nbctl lsp-set-addresses ls1p3 00:01:02:01:01:04 ovn-nbctl ls-add ls2 ovn-nbctl lsp-add ls2 ls2p1 ovn-nbctl lsp-set-addresses ls2p1 00:01:02:01:01:03 ovs-vsctl add-port br-int vm1 -- set interface vm1 type=internal ip netns add server0 ip link set vm1 netns server0 ip netns exec server0 ip link set lo up ip netns exec server0 ip link set vm1 up ip netns exec server0 ip link set vm1 address 00:01:02:01:01:01 ip netns exec server0 ip addr add 192.168.0.1/24 dev vm1 ip netns exec server0 ip addr add 3001::1/64 dev vm1 ip netns exec server0 ip route add default via 192.168.0.254 dev vm1 ip netns exec server0 ip -6 route add default via 3001::a dev vm1 ovs-vsctl set Interface vm1 external_ids:iface-id=ls1p1 ovs-vsctl add-port br-int vm2 -- set interface vm2 type=internal ip netns add server1 ip link set vm2 netns server1 ip netns exec server1 ip link set lo up ip netns exec server1 ip link set vm2 up ip netns exec server1 ip link set vm2 address 00:01:02:01:01:02 ip netns exec server1 ip addr add 192.168.0.2/24 dev vm2 ip netns exec server1 ip addr add 3001::2/64 dev vm2 ip netns exec server1 ip route add default via 192.168.0.254 dev vm2 ip netns exec server1 ip -6 route add default via 3001::a dev vm2 ovs-vsctl set Interface vm2 external_ids:iface-id=ls1p2 ovn-nbctl lr-add lr1 ovn-nbctl lrp-add lr1 lr1ls1 00:01:02:0d:01:01 192.168.0.254/24 3001::a/64 ovn-nbctl lrp-add lr1 lr1ls2 00:01:02:0d:01:02 192.168.1.254/24 3001:1::a/64 ovn-nbctl lsp-add ls1 ls1lr1 ovn-nbctl lsp-set-type ls1lr1 router ovn-nbctl lsp-set-options ls1lr1 router-port=lr1ls1 ovn-nbctl lsp-set-addresses ls1lr1 "00:01:02:0d:01:01 192.168.0.254 3001::a" ovn-nbctl lsp-add ls2 ls2lr1 ovn-nbctl lsp-set-type ls2lr1 router ovn-nbctl lsp-set-options ls2lr1 router-port=lr1ls2 ovn-nbctl lsp-set-addresses ls2lr1 "00:01:02:0d:01:02 192.168.1.254 3001:1::a" ovn-nbctl lrp-add lr1 lr1p 00:01:02:0d:0f:01 172.16.1.254/24 2002::a/64 ovn-nbctl lb-add lb0 192.168.2.1:12345 192.168.0.1:12345,192.168.0.2:12345 ovn-nbctl lb-add lb0 [3000::100]:12345 [3001::1]:12345,[3001::2]:12345 uuid=`ovn-nbctl list Load_Balancer |grep uuid|awk '{printf $3}'` ovn-nbctl set load_balancer $uuid selection_fields="ip_src,ip_dst" ovn-nbctl show ovn-sbctl show ovn-nbctl set Logical_Router lr1 options:chassis="hv1" ovn-nbctl ls-lb-add ls1 lb0 I test selection_fields as "eth_src,eth_dst","ip_src,ip_dst","ip_src,ip_dst,tp_src,tp_dst",and lb on ls and lr. for tcp case,lb works well. so set the bug verified. version: # rpm -qa|grep ovn ovn2.13-central-2.13.0-34.el7fdp.x86_64 ovn2.13-2.13.0-34.el7fdp.x86_64 ovn2.13-host-2.13.0-34.el7fdp.x86_64 performance test will be done later. for udp and sctp case, lb works not correctly.raised bug here:https://bugzilla.redhat.com/show_bug.cgi?id=1846189
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:2941