Upstream PR https://github.com/ovn-org/ovn-kubernetes/pull/2152 fixes a bug where ForEachAddressSet didn't work.
This means that we won't delete old address sets if the deletion is missed, say, because ovn-k was restarting.
If another namespace or NetworkPolicy comes into existence with the same name, it will have spurious IPs in the address set.
We should backport this fix as appropriate.
This was pulled down to 4.8 as part of https://github.com/openshift/ovn-kubernetes/pull/496