Description of problem: Currently the OVN ACL tables priority column is of type `int` yet the limit is set to max int16 size (0 to 32767) I want to have 200000 ++ priorities int32 : -2147483648 to 2147483647 int64 : -9223372036854775808 to 9223372036854775807 So depending on the system let's increase the limits? Is there is real reason for restricting this? One might ask why on earth do you need so many priorities? It's for the ANP feature: https://github.com/kubernetes/enhancements/tree/master/keps/sig-network/2091-admin-network-policy#design-details We can have upto 1000 ANPs, each can have 100 ingress and 100 egress rules and ingress and egress are an array of rules. Depending on the position of rules they get priority -> example rule0 in array is higher prio than rule1. Thus we need 200*1000 priorities {Other features EFW/NP use the lower bound of it upto 8000} I'm not sure if this limit removal will have implications on backwards support compatibility on OVN side, but this should be doable somehow? Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info: Would be good to get this support in during the same time frame as https://bugzilla.redhat.com/show_bug.cgi?id=2134138
Hi, Surya. ACL priorities are translated to OpenFlow priorities by OVN. And the problem is that maximum OpenFlow priority is 65535. And since not everything is ACL, OVN can't use the full rage for ACLS. Currently it is 1000 lower. So, in current OVN design ACLs can use at most 64K priorities, AFAICT. And I'm not sure if it's possible to increase that range.
(In reply to Ilya Maximets from comment #1) > Hi, Surya. > > ACL priorities are translated to OpenFlow priorities by OVN. > And the problem is that maximum OpenFlow priority is 65535. > And since not everything is ACL, OVN can't use the full rage > for ACLS. Currently it is 1000 lower. So, in current OVN > design ACLs can use at most 64K priorities, AFAICT. And I'm > not sure if it's possible to increase that range. uh-oh, hmm this might be a problem if we can't achieve this. :( I will bring this up in our next OCP/OVN Sync. Forgive me if this is a stupid question but can we increase OF priorities then? :D
(In reply to Surya Seetharaman from comment #2) > Forgive me if this is a stupid question but can we increase OF priorities > then? :D I'm afraid this will require changing half of OpenFlow APIs, so probably require a new version of a standard altogether.
(In reply to Ilya Maximets from comment #3) > (In reply to Surya Seetharaman from comment #2) > > Forgive me if this is a stupid question but can we increase OF priorities > > then? :D > > I'm afraid this will require changing half of OpenFlow APIs, so probably > require a new version of a standard altogether. hmm ack, meanwhile I will work on an upper limit of 250 ANPs or something to begin with. Cause let's say we give 64K, 8K is used by EGF/NP etc.. so that gives me 56K which is max 280 ANPs. Either ways I don't see us scaling well if we have over 100 ANPs and admins shouldn't really need more than 30 or 40 (MAX). So for now we can start with capping things at 250 on OVNK side But I want to leave this issue open so that we discuss this more.
(In reply to Surya Seetharaman from comment #4) > hmm ack, meanwhile I will work on an upper limit of 250 ANPs or something to > begin with. > Cause let's say we give 64K, 8K is used by EGF/NP etc.. so that gives me 56K > which is max 280 ANPs. > Either ways I don't see us scaling well if we have over 100 ANPs and admins > shouldn't really need > more than 30 or 40 (MAX). So for now we can start with capping things at 250 > on OVNK side FWIW, without schema change in OVN, the max is 32768 / 200 = 163 right now or (32768 - 8192) / 200 = 122, in case you need 8K for EGF/NP etc. So, maybe 100 is a good limit to start with after all?
(In reply to Ilya Maximets from comment #5) > (In reply to Surya Seetharaman from comment #4) > > hmm ack, meanwhile I will work on an upper limit of 250 ANPs or something to > > begin with. > > Cause let's say we give 64K, 8K is used by EGF/NP etc.. so that gives me 56K > > which is max 280 ANPs. > > Either ways I don't see us scaling well if we have over 100 ANPs and admins > > shouldn't really need > > more than 30 or 40 (MAX). So for now we can start with capping things at 250 > > on OVNK side > > FWIW, without schema change in OVN, the max is 32768 / 200 = 163 right now > or (32768 - 8192) / 200 = 122, in case you need 8K for EGF/NP etc. > > So, maybe 100 is a good limit to start with after all? YEAH, good point haha, probably 100 is a good starting point for now. Will bring this up in next Wednesday just to discuss this as a future effort. Don't want to close this issue yet until then
I also observed something a fun fact happening to the priorities actually: ACL: _uuid : 5980a099-b46f-4a36-b240-cfe4e2f8e3c8 action : drop direction : to-lport external_ids : {} label : 0 log : false match : "(ip4.src == $a14188272659559066445) && (outport == @a1093691609995682552)" meter : acl-logging name : deny-example_ANPIngress_31900 options : {} priority : 31900 severity : debug sh-5.2# ovn-sbctl lflow-list | grep 31900 sh-5.2# ovn-sbctl lflow-list | grep 32900 table=4 (ls_out_acl ), priority=32900, match=(reg0[10] == 1 && ((ip4.src == $a14188272659559066445) && (outport == @a1093691609995682552))), action=(ct_commit { ct_mark.blocked = 1; }; /* drop */) table=4 (ls_out_acl ), priority=32900, match=(reg0[9] == 1 && ((ip4.src == $a14188272659559066445) && (outport == @a1093691609995682552))), action=(/* drop */) table=4 (ls_out_acl ), priority=32900, match=(reg0[10] == 1 && ((ip4.src == $a14188272659559066445) && (outport == @a1093691609995682552))), action=(ct_commit { ct_mark.blocked = 1; }; /* drop */) table=4 (ls_out_acl ), priority=32900, match=(reg0[9] == 1 && ((ip4.src == $a14188272659559066445) && (outport == @a1093691609995682552))), action=(/* drop */) LOL, I guess OVN keeps the first 1000 to itself like you said. sh-5.2# ovs-ofctl dump-flows br-int | grep 10.244.1.4 cookie=0x135a1884, duration=601.691s, table=44, n_packets=0, n_bytes=0, idle_age=601, priority=32900,ip,reg0=0x400/0x400,reg15=0x6,metadata=0x3,nw_src=10.244.1.4 actions=ct(commit,zone=NXM_NX_REG13[0..15],nat(src),exec(load:0x1->NXM_NX_CT_MARK[0])) cookie=0x1dc49a57, duration=601.691s, table=44, n_packets=0, n_bytes=0, idle_age=601, priority=32900,ip,reg0=0x200/0x200,reg15=0x6,metadata=0x3,nw_src=10.244.1.4 actions=drop
Hey Surya, would hierarchical ACLs (https://bugzilla.redhat.com/show_bug.cgi?id=2134138) help with this? The way I implemented it, you essentially get 4x the number of priorities as you otherwise would have. So ~31000 x 4 == ~124000 priorities of ACL. And it wouldn't be a big deal to increase the number of tiers to 8 (248000) or 16 (596000).
I'm closing this as WONTFIX, since the tiered ACLs provided in ovn23.06 should address this by giving more priorities to use for ACLs.
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days