In OpenShift we've seen a problem where when pods are being created and destroyed at a high rate, you eventually end up with a scenario where:

  - pod A is talking to pod B, which has, say, IP 10.0.1.5 and
    MAC bb:bb:bb:bb:bb:bb

      - pod A ends up with an entry 10.0.1.2 -> bb:bb:bb:bb:bb:bb in
        its ARP cache

  - pod B exits / is destroyed

  - Around 255 other pods on pod B's node are created/destroyed in a
    short amount of time, and the IP address assignment range wraps
    around back to the beginning again.

  - pod C is created and gets assigned IP 10.0.1.5 and
    MAC cc:cc:cc:cc:cc:cc

  - pod A tries to talk to pod C, finds that it already has an ARP
    cache entry for 10.0.1.5, and so tries to send packets to
    IP 10.0.1.5, MAC bb:bb:bb:bb:bb:bb

      - These packets go nowhere because nobody currently has that MAC

  - pod A's attempt to talk to pod C eventually times out. Things
    start failing

(This is not a problem in a VM-based world because of a combination of (a) VMs come and go less quickly than containers, so other VMs are less likely to still have stale ARP cache mappings when an IP gets reused again; and (b) VMs, like bare metal hosts, tend to have startup scripts that send out gratuitous ARPs when they bring up their network connection, so anyone who did have a stale ARP cache entry would get fixed.)

In OpenShift SDN, our fix for this was to just assign pods deterministic MAC addresses that were based on their IPs; specifically they get 0a:58:ww:xx:yy:zz, where ww:xx:yy:zz is the IP converted to hex. (The code for this comes from CNI and is used by some other plugins as well. I don't know who chose the prefix "0a:58" or why.)


With ovn-kubernetes, we will need to either

  1. also have deterministic IP-to-MAC mappings, OR
  2. send out ARP announcements whenever a pod is created

The latter would be possible, but is less inefficient if lots of pods are being created, especially if they are attached to logical switches that are spread across multiple hosts.


We don't handle IPv6 yet, and I'm not sure what the situation is there; in theory the kernel automatically handles the "announcement" part, so there might not be a problem. Unless the announcements get sent out before OVN is ready to forward them to other ports, which might be the case. Also, even if the announcements do get sent out, and do work, it would still be more efficient to *not* forward them, if they were known to be unnecessary.