Bug 2175752

Hi, Surya.

ACL priorities are translated to OpenFlow priorities by OVN.
And the problem is that maximum OpenFlow priority is 65535.
And since not everything is ACL, OVN can't use the full rage
for ACLS.  Currently it is 1000 lower.  So, in current OVN
design ACLs can use at most 64K priorities, AFAICT.  And I'm
not sure if it's possible to increase that range.

(In reply to Ilya Maximets from comment #1)
> Hi, Surya.
> 
> ACL priorities are translated to OpenFlow priorities by OVN.
> And the problem is that maximum OpenFlow priority is 65535.
> And since not everything is ACL, OVN can't use the full rage
> for ACLS.  Currently it is 1000 lower.  So, in current OVN
> design ACLs can use at most 64K priorities, AFAICT.  And I'm
> not sure if it's possible to increase that range.

uh-oh, hmm this might be a problem if we can't achieve this. :( I will bring this up in our next OCP/OVN Sync.
Forgive me if this is a stupid question but can we increase OF priorities then? :D

(In reply to Surya Seetharaman from comment #2)
> Forgive me if this is a stupid question but can we increase OF priorities
> then? :D

I'm afraid this will require changing half of OpenFlow APIs, so probably
require a new version of a standard altogether.

(In reply to Ilya Maximets from comment #3)
> (In reply to Surya Seetharaman from comment #2)
> > Forgive me if this is a stupid question but can we increase OF priorities
> > then? :D
> 
> I'm afraid this will require changing half of OpenFlow APIs, so probably
> require a new version of a standard altogether.

hmm ack, meanwhile I will work on an upper limit of 250 ANPs or something to begin with.
Cause let's say we give 64K, 8K is used by EGF/NP etc.. so that gives me 56K which is max 280 ANPs.
Either ways I don't see us scaling well if we have over 100 ANPs and admins shouldn't really need
more than 30 or 40 (MAX). So for now we can start with capping things at 250 on OVNK side

But I want to leave this issue open so that we discuss this more.

(In reply to Surya Seetharaman from comment #4)
> hmm ack, meanwhile I will work on an upper limit of 250 ANPs or something to
> begin with.
> Cause let's say we give 64K, 8K is used by EGF/NP etc.. so that gives me 56K
> which is max 280 ANPs.
> Either ways I don't see us scaling well if we have over 100 ANPs and admins
> shouldn't really need
> more than 30 or 40 (MAX). So for now we can start with capping things at 250
> on OVNK side

FWIW, without schema change in OVN, the max is 32768 / 200 = 163 right now
or (32768 - 8192) / 200 = 122, in case you need 8K for EGF/NP etc.

So, maybe 100 is a good limit to start with after all?

(In reply to Ilya Maximets from comment #5)
> (In reply to Surya Seetharaman from comment #4)
> > hmm ack, meanwhile I will work on an upper limit of 250 ANPs or something to
> > begin with.
> > Cause let's say we give 64K, 8K is used by EGF/NP etc.. so that gives me 56K
> > which is max 280 ANPs.
> > Either ways I don't see us scaling well if we have over 100 ANPs and admins
> > shouldn't really need
> > more than 30 or 40 (MAX). So for now we can start with capping things at 250
> > on OVNK side
> 
> FWIW, without schema change in OVN, the max is 32768 / 200 = 163 right now
> or (32768 - 8192) / 200 = 122, in case you need 8K for EGF/NP etc.
> 
> So, maybe 100 is a good limit to start with after all?

YEAH, good point haha, probably 100 is a good starting point for now.
Will bring this up in next Wednesday just to discuss this as a future effort. Don't want to close this issue yet until then

I also observed something a fun fact happening to the priorities actually:

ACL:

_uuid               : 5980a099-b46f-4a36-b240-cfe4e2f8e3c8
action              : drop
direction           : to-lport
external_ids        : {}
label               : 0
log                 : false
match               : "(ip4.src == $a14188272659559066445) && (outport == @a1093691609995682552)"
meter               : acl-logging
name                : deny-example_ANPIngress_31900
options             : {}
priority            : 31900
severity            : debug

sh-5.2# ovn-sbctl lflow-list | grep 31900
sh-5.2# ovn-sbctl lflow-list | grep 32900
  table=4 (ls_out_acl         ), priority=32900, match=(reg0[10] == 1 && ((ip4.src == $a14188272659559066445) && (outport == @a1093691609995682552))), action=(ct_commit { ct_mark.blocked = 1; }; /* drop */)
  table=4 (ls_out_acl         ), priority=32900, match=(reg0[9] == 1 && ((ip4.src == $a14188272659559066445) && (outport == @a1093691609995682552))), action=(/* drop */)
  table=4 (ls_out_acl         ), priority=32900, match=(reg0[10] == 1 && ((ip4.src == $a14188272659559066445) && (outport == @a1093691609995682552))), action=(ct_commit { ct_mark.blocked = 1; }; /* drop */)
  table=4 (ls_out_acl         ), priority=32900, match=(reg0[9] == 1 && ((ip4.src == $a14188272659559066445) && (outport == @a1093691609995682552))), action=(/* drop */)

LOL, I guess OVN keeps the first 1000 to itself like you said.

sh-5.2#  ovs-ofctl dump-flows br-int | grep 10.244.1.4
 cookie=0x135a1884, duration=601.691s, table=44, n_packets=0, n_bytes=0, idle_age=601, priority=32900,ip,reg0=0x400/0x400,reg15=0x6,metadata=0x3,nw_src=10.244.1.4 actions=ct(commit,zone=NXM_NX_REG13[0..15],nat(src),exec(load:0x1->NXM_NX_CT_MARK[0]))
 cookie=0x1dc49a57, duration=601.691s, table=44, n_packets=0, n_bytes=0, idle_age=601, priority=32900,ip,reg0=0x200/0x200,reg15=0x6,metadata=0x3,nw_src=10.244.1.4 actions=drop

Hey Surya, would hierarchical ACLs (https://bugzilla.redhat.com/show_bug.cgi?id=2134138) help with this? The way I implemented it, you essentially get 4x the number of priorities as you otherwise would have. So ~31000 x 4 == ~124000 priorities of ACL. And it wouldn't be a big deal to increase the number of tiers to 8 (248000) or 16 (596000).

I'm closing this as WONTFIX, since the tiered ACLs provided in ovn23.06 should address this by giving more priorities to use for ACLs.

The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days