Description of problem: OpenStack Neutron could have enormous amount of entities and connections to track. There is a long-standing RFE bug #1399987 requesting separate connection tracking for each tenant (group of VMs/router/networks/ports/etc). Originally bug #1399987 was reported for ML2/OVS plugin and it depended on iptables feature. It was implemented recently, but ML2/OVS plugin itself is going to be deprecated quite soon. So now same RFE should be implemented for ML2/OVN plugin. Ihar helped me to understand current status. It looks like in OVS, there's ct-set-limits and ct-del-limits CLI commands for dpctl tool, but there is nothing similar hooked into OVN code base (nor anything relevant pops up in its documentation). https://bugzilla.redhat.com/show_bug.cgi?id=1399987#c39
There is already an effort for the OvS side of things: https://patchwork.ozlabs.org/project/openvswitch/patch/20230330081718.196496-1-naveen.yerramneni@nutanix.com/ The commit message mentions that this extension would be later on used in OVN so it seems there is a community work towards this.
I'm sorry I did use this BZ id on other patch series by accident. This is still pending in OvS.
After some discussion it seems like the solution should be the following: New option for LS and LR (let's call it "ct-zone-limit") that would apply the limit to both SNAT and DNAT zones in the LR/LS. With addition to the LS which would also apply the limit to all LSP connected to that LS. For this to be more flexible the direct "ct-zone-limit" could be set to specific port which would overwrite the inherited limit from the LS. Can you please confirm that this would be sufficient for OSP needs?
Thank you very much for taking this bug Ales. I am not 100% sure if this will be consistent approach that will cover all possible conntrack entities: I know too little about OVN internals. From customer's perspective, a conntrack for LR/LS should be clear. I am not sure how beneficial the work for separate port would be because AFAIU these conntrack entries will contribute to LR/LS number of tracked connections, so it may not worth the effort. If it is impossible to implement this per tenant, then fine, but Neutron people should be asked to confirm that selected approach is fine and they will benefit from it and will be able to adopt changes in OVN and implement related features in Neutron (so setting needinfo for Slawek who helped me with original bug #1399987). P.S. In any case I wanted to ask to consider improving logging, so possible conntrack problems are logged clearly and are easy to troubleshoot...
I believe the original request was in scope of security (one tenant affecting another one). If the limit is per-port / router, then neutron would have to somehow translate it into a global limit. (Of course, CMS admin can also set a limit on the number of ports / routers created - and in this way set the upper boundary on the total number of ct entries, but it's cumbersome and doesn't consider the fact that some users may need a single large router while others may need a large number of smaller routers.) I think it's wise to ask Slawek if this per-port limit would cover the per-tenant need.
Original u/s RFE for ML2/OVS backend (https://bugs.launchpad.net/neutron/+bug/2020358) was just about separate conntrack limits between tenants. Unfortunately we can't really translate this theoretical "per tenant" setting into "per LS/LR" in OVN. So I think that having separate limit set for each LS/LR in OVN would be enough to address that use case mentioned in the LP bug ("A tenant can cause network issues for other tenants: nf_conntrack: table full, dropping packet."). I don't think we need to have it "per port" (LSP) really. The other thing is (but this is for even more distant future) is to use e.g. QoS rules to set different number of conntrack entries per network or router. If we would allow that at some point, the way how QoS policies are implemented in Neutron means that we would then probably benefit from setting it per port (per LSP/LRP).
I suspect the conundrum is that in OVN, there is no such thing as tenants, and zone IDs are not aggregated / shared either. So from OVN perspective, it may be hard or impossible to aggregate these zones for CMS. And on the other hand, CMS wants to set a limit per tenant (= per subset of ports/routers) and not care about per-resource (per-port, per-router) limits - in terms of multitenancy guarantees, it makes no difference for CMS if a single router wastes 1 mln conntrack entries, or there are 1000 routers, 1000 conntrack entries per each, if they belong to the same user. Let me know if this is helpful.