+++ This bug was initially created as a clone of Bug #1720653 +++ Additional info: We are interested in the time that it takes ovn-controller to resync to a different southbound database with substantially the same content. So for example after creating a lot of logical switches and switch ports like in the scale tests, copy the southbound database file (or create an entirely new one with different switches and ports) and start a new ovsdb-server on a different port on the server. The change ovn- remote in the local vswitch and see how long it takes for ovn- controller to update the flows in OVS based on the new database. Girish said that the buffers that ovn-controller used to download the new database were pretty small and when they bumped the size from IIRC 4k -> 64k the download itself went quite a bit faster. But in the end, it is not clear what the delay from (a) change ovn- remote to (b) OVS flows are synchronized, is and if we need to look into optimizing that.
steps: server: systemctl start ovn-northd ovn-nbctl set-connection ptcp:6641 ovn-sbctl set-connection ptcp:6642 cp ~/ovnnb_db.txt /var/lib/ovn/ovnnb_db.db -f systemctl restart ovn-northd systemctl start openvswitch ovs-vsctl set open . external_ids:system-id=hv0 external_ids:ovn-remote=tcp:20.0.30.25:6642 external_ids:ovn-encap-type=geneve external_ids:ovn-encap-ip=20.0.30.25 systemctl restart ovn-controller ovn-nbctl clear logical_switch_port 253a3b8b-76c6-4af0-ad12-d1513cf0c6a3 options ovn-nbctl clear logical_switch_port 4cdaae8b-b472-4ae4-8f26-fd0e2a375bc1 options ovn-nbctl clear logical_switch_port 9c3ce12f-0373-4004-9a12-7c3b34ce0ae4 options ovn-nbctl clear logical_switch_port 063a7db6-2f19-45c2-a3b2-8414b4c40d38 options ovn-nbctl clear logical_switch_port 173224b0-9e29-4c49-8bdf-85a0f4297bff options ovn-nbctl clear logical_switch_port 2403f99f-9be3-427b-9937-71776bc44b5d options wait a while for sb generating on server after setup client: systemctl start openvswitch ovs-vsctl set open . external_ids:system-id=hv1 external_ids:ovn-remote=tcp:20.0.30.25:6642 external_ids:ovn-encap-type=geneve external_ids:ovn-encap-ip=20.0.30.26 systemctl restart ovn-controller sleep 10 ovs-vsctl add-port br-int vm0 -- set interface vm0 type=internal ovs-vsctl set Interface vm0 external_ids:iface-id=253a3b8b-76c6-4af0-ad12-d1513cf0c6a3 ovs-vsctl add-port br-int vm1 -- set interface vm1 type=internal ovs-vsctl set Interface vm1 external_ids:iface-id=4cdaae8b-b472-4ae4-8f26-fd0e2a375bc1 ovs-vsctl add-port br-int vm2 -- set interface vm2 type=internal ovs-vsctl set Interface vm2 external_ids:iface-id=9c3ce12f-0373-4004-9a12-7c3b34ce0ae4 ovs-vsctl add-port br-int vm3 -- set interface vm3 type=internal ovs-vsctl set Interface vm3 external_ids:iface-id=063a7db6-2f19-45c2-a3b2-8414b4c40d38 ovs-vsctl add-port br-int vm4 -- set interface vm4 type=internal ovs-vsctl set Interface vm4 external_ids:iface-id=173224b0-9e29-4c49-8bdf-85a0f4297bff ovs-vsctl add-port br-int vm5 -- set interface vm5 type=internal ovs-vsctl set Interface vm5 external_ids:iface-id=2403f99f-9be3-427b-9937-71776bc44b5d for i in {1..60} do ovs-ofctl dump-flows br-int | wc -l sleep 1 done ovs-ofctl dump-flows br-int | wc -l num_flow=$(ovs-ofctl dump-flows br-int | wc -l) ovs-vsctl set open . external_ids:ovn-remote=tcp:20.0.30.11:6642 ovs-ofctl del-flows br-int systemctl restart ovn-controller ovs-ofctl dump-flows br-int | wc -l ovs-vsctl set open . external_ids:ovn-remote=tcp:20.0.30.25:6642 systemctl restart ovn-controller date > $(date +%H-%M).log while : do ovs-ofctl dump-flows br-int | wc -l num2_flow=$(ovs-ofctl dump-flows br-int | wc -l) date [ $num_flow -le $num2_flow ] && break sleep 1 done on openvswitch2.12.0-8: start time: Thu Jan 9 07:19:34 EST 2020 flow finish time: Thu Jan 9 07:20:45 EST 2020 [root@ibm-x3650m4-01 bz1720653]# rpm -qa | grep -E "openvswitch|ovn" kernel-kernel-networking-openvswitch-ovn-common-1.0-7.noarch ovn2.12-2.12.0-19.el8fdp.x86_64 ovn2.12-host-2.12.0-19.el8fdp.x86_64 openvswitch-selinux-extra-policy-1.0-19.el8fdp.noarch openvswitch2.12-2.12.0-8.el8fdp.x86_64 ovn2.12-central-2.12.0-19.el8fdp.x86_64 on openvswitch2.12.0-12: start time: Thu Jan 9 07:34:24 EST 2020 flow finish time: Thu Jan 9 07:35:29 EST 2020 it seems that there no much improvement form 8 to 12.
Created attachment 1650959 [details] nb db file
hi Jianlin Shi, 50-100ms delta time is aligned to what is expected since most of the time will be spent parsing/creating the json file
set VERIFIED per comment 4
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2020:0172