Created attachment 1396044 [details]
Creating 800 ports performance

From some performance tests in networking-ovn, looks like creating ports is slower using networking-ovn as a mechanism driver than it is with ML2/OVS.
After digging deeper, it looks like, it won't scale as creating a port in OVN takes longer and longer as the number of existing ports increases. See attached image to this bug.

The way that I got those numbers is using a TripleO deployment using 7a57d99b10f5872520e1a395214f80c9cfd1a65 DLRN hash. From the undercloud I simply ran a script which creates 800 ports sequentially in a /16 subnet and all of them in a security group which allows ICMP and SSH.

It can be seen that the time for creating a port in OVN grows linearly with the number of ports existing in the system. There's some discussion in this email thread [0].

It is suspected that ACLs are the main culprit since every time a new port is added, 8 new ACLs are added to OVN. So, after the test, 800 ports exist in the Logical Switch and 6400 ACLs, which are also referenced in the LS table.

Also, every time a new port is added, ovsdb-server will send this update2 message to every connection it has (In my deployment there were 10 simultaneous connections to OVN NB):


{"id":null,"method":"update2","params":["03db3b66-1107-11e8-825c-006d3685605b",{

"Address_Set":{"7992e538-1ff1-4c31-a813-93bad7fb16e3":{"modify":{"addresses":"10.1.1.15"}}},

"Logical_Switch":{"e5b7cfd1-fa0f-425c-aa93-b5be1f09fcea":{"modify":{"ports":["uuid","19c7b4f2-62ff-4a62-aadd-0612fd920bfe"],"acls":["set",[["uuid","0406dc4b-caf6-4cf9-930c-39438978ddde"],["uuid","062ebd77-4eeb-464a-b783-65eda13b22e9"],["uuid","0babd48d-b618-4b63-95d0-789b1de68b3c"],["uuid","32fdcb44-f698-4818-a784-46c33a9eee1a"],["uuid","7008064e-7019-4e64-b794-73ccb3d4f1b9"],["uuid","764ac474-9caf-4e75-8ed6-4e2352140db9"],["uuid","765c6abe-8dc4-42cb-b0b1-ffb75e80503b"],["uuid","addbfbbe-437b-4de6-b52f-778c4954632d"]]]}}},

"ACL":{"764ac474-9caf-4e75-8ed6-4e2352140db9":{"insert":{"match":"inport == \"d484e712-c8c5-465a-80cc-2837732a6d9b\" && ip","action":"drop","priority":1001,"external_ids":["map",[["neutron:lport","d484e712-c8c5-465a-80cc-2837732a6d9b"]]],"direction":"from-lport"}},"32fdcb44-f698-4818-a784-46c33a9eee1a":{"insert":{"match":"inport == \"d484e712-c8c5-465a-80cc-2837732a6d9b\" && ip4 && ip4.dst == 0.0.0.0/0 && tcp","action":"allow-related","priority":1002,"external_ids":["map",[["neutron:lport","d484e712-c8c5-465a-80cc-2837732a6d9b"],["neutron:security_group_rule_id","8a7ef6fb-cf23-416c-9b72-35971b51ffde"]]],"direction":"from-lport"}},"0406dc4b-caf6-4cf9-930c-39438978ddde":{"insert":{"match":"outport == \"d484e712-c8c5-465a-80cc-2837732a6d9b\" && ip","action":"drop","priority":1001,"external_ids":["map",[["neutron:lport","d484e712-c8c5-465a-80cc-2837732a6d9b"]]],"direction":"to-lport"}},"7008064e-7019-4e64-b794-73ccb3d4f1b9":{"insert":{"match":"inport == \"d484e712-c8c5-465a-80cc-2837732a6d9b\" && ip4","action":"allow-related","priority":1002,"external_ids":["map",[["neutron:lport","d484e712-c8c5-465a-80cc-2837732a6d9b"],["neutron:security_group_rule_id","04a69c27-489c-4b22-aca1-7f773ceef250"]]],"direction":"from-lport"}},"0babd48d-b618-4b63-95d0-789b1de68b3c":{"insert":{"match":"inport == \"d484e712-c8c5-465a-80cc-2837732a6d9b\" && ip6","action":"allow-related","priority":1002,"external_ids":["map",[["neutron:lport","d484e712-c8c5-465a-80cc-2837732a6d9b"],["neutron:security_group_rule_id","57063d03-8ca3-47a6-9013-56f2894930fe"]]],"direction":"from-lport"}},"765c6abe-8dc4-42cb-b0b1-ffb75e80503b":{"insert":{"match":"inport == \"d484e712-c8c5-465a-80cc-2837732a6d9b\" && ip4 && ip4.dst == {255.255.255.255, 10.1.0.0/16} && udp && udp.src == 68 && udp.dst == 67","action":"allow","priority":1002,"external_ids":["map",[["neutron:lport","d484e712-c8c5-465a-80cc-2837732a6d9b"]]],"direction":"from-lport"}},"addbfbbe-437b-4de6-b52f-778c4954632d":{"insert":{"match":"outport == \"d484e712-c8c5-465a-80cc-2837732a6d9b\" && ip4 && ip4.src == 0.0.0.0/0 && icmp4","action":"allow-related","priority":1002,"external_ids":["map",[["neutron:lport","d484e712-c8c5-465a-80cc-2837732a6d9b"],["neutron:security_group_rule_id","fc87b952-c867-4044-bb9a-8e7836de6d09"]]],"direction":"to-lport"}},"062ebd77-4eeb-464a-b783-65eda13b22e9":{"insert":{"match":"outport == \"d484e712-c8c5-465a-80cc-2837732a6d9b\" && ip4 && ip4.src == 0.0.0.0/0 && tcp && tcp.dst == 22","action":"allow-related","priority":1002,"external_ids":["map",[["neutron:lport","d484e712-c8c5-465a-80cc-2837732a6d9b"],["neutron:security_group_rule_id","89ae1a13-e6e3-4f02-bc2c-50b2302f6d9e"]]],"direction":"to-lport"}}},

"Logical_Switch_Port":{"19c7b4f2-62ff-4a62-aadd-0612fd920bfe":{"insert":{"name":"d484e712-c8c5-465a-80cc-2837732a6d9b","port_security":"fa:16:3e:04:e7:ef 10.1.1.15","addresses":"fa:16:3e:04:e7:ef 10.1.1.15","dhcpv4_options":["uuid","9cd1cf76-cb38-4e33-bffe-0505288001c6"],"options":["map",[["requested-chassis",""]]],"external_ids":["map",[["neutron:cidrs","10.1.1.15/16"],["neutron:device_id",""],["neutron:device_owner",""],["neutron:network_name","neutron-db68a2f7-a845-46aa-a9ca-58bc8724dc57"],["neutron:port_name","port2"],["neutron:project_id","073a1dacd81b4dbba7f48358fb404706"],["neutron:revision_number","4"],["neutron:security_group_ids","b4bd8a71-a2f1-4172-8d99-a6cc48508823"]]],"enabled":true}}}}]}


[0] https://mail.openvswitch.org/pipermail/ovs-discuss/2018-February/046149.html