This bug was initially created as a copy of Bug #1947176
I am copying this bug because:
Upstream PR https://github.com/ovn-org/ovn-kubernetes/pull/2152 fixes a bug where ForEachAddressSet didn't work.
This means that we won't delete old address sets if the deletion is missed, say, because ovn-k was restarting.
If another namespace or NetworkPolicy comes into existence with the same name, it will have spurious IPs in the address set.
We should backport this fix as appropriate.