Bug 2028581

commit e90e06a8186220c61d9c3018e06cc82a7152b04f
Merge: 77a249d38 1d8e0f861
Author: Open vSwitch CI <ovs-ci>
Date:   Thu Nov 18 15:43:12 2021 -0500

    Merging upstream branch-2.16
    
    Commit list:
    1d8e0f861f ofproto-dpif-xlate: Fix check_pkt_larger incomplete translation.

commit 77a249d38b77555f83672e2612cbb659df8e2427
Merge: 0cd586753 f8f2f7c9c
Author: Open vSwitch CI <ovs-ci>
Date:   Mon Nov 15 04:05:19 2021 -0500

    Merging upstream branch-2.16
    
    Commit list:
    f8f2f7c9cb datapath-windows: Reset flow key after Ipv4 fragments are reassembled

commit 0cd5867531e270d21e1ef386adae168121533b3c
Author: Timothy Redaelli <tredaelli>
Date:   Sat Oct 30 01:12:14 2021 +0200

    python: Replace pyOpenSSL with ssl.
    
    Currently, pyOpenSSL is half-deprecated upstream and so it's removed on
    some distributions (for example on CentOS Stream 9,
    https://issues.redhat.com/browse/CS-336), but since OVS only
    supports Python 3 it's possible to replace pyOpenSSL with "import ssl"
    included in base Python 3.
    
    Stream recv and send had to be splitted as _recv and _send, since SSLError
    is a subclass of socket.error and so it was not possible to except for
    SSLWantReadError and SSLWantWriteError in recv and send of SSLStream.
    
    TCPstream._open cannot be used in SSLStream, since Python ssl module
    requires the SSL socket to be created before connecting it, so
    SSLStream._open needs to create the socket, create SSL socket and then
    connect the SSL socket.
    
    Reported-by: Timothy Redaelli <tredaelli>
    Reported-at: https://bugzilla.redhat.com/1988429
    Signed-off-by: Timothy Redaelli <tredaelli>
    Acked-by: Terry Wilson <twilson>
    Tested-by: Terry Wilson <twilson>
    Signed-off-by: Ilya Maximets <i.maximets>
    Signed-off-by: Timothy Redaelli <tredaelli>

commit 2e704b371c074077fc3e0bc4ec2379c4fe44b3ad
Author: Timothy Redaelli <tredaelli>
Date:   Mon Oct 25 15:45:42 2021 +0200

    python: socket-util: Split inet_open_active function and use connect_ex.
    
    In an upcoming patch, PyOpenSSL will be replaced with Python ssl module,
    but in order to do an async connection with Python ssl module the ssl
    socket must be created when the socket is created, but before the
    socket is connected.
    
    So, inet_open_active function is splitted in 3 parts:
    - inet_create_socket_active: creates the socket and returns the family and
      the socket, or (error, None) if some error needs to be returned.
    - inet_connect_active: connect the socket and returns the errno (it
      returns 0 if errno is EINPROGRESS or EWOULDBLOCK).
    
    connect is replaced by connect_ex, since Python suggest to use it for
    asynchronous connects and it's also cleaner since inet_connect_active
    returns errno that connect_ex already returns, moreover due to a Python
    limitation connect cannot not be used with ssl module.
    
    inet_open_active function is changed in order to use the new functions
    inet_create_socket_active and inet_connect_active.
    
    Signed-off-by: Timothy Redaelli <tredaelli>
    Acked-by: Terry Wilson <twilson>
    Tested-by: Terry Wilson <twilson>
    Signed-off-by: Ilya Maximets <i.maximets>
    Signed-off-by: Timothy Redaelli <tredaelli>

commit 4c846afd24899bbea014b29f5772b7b625693119
Author: Timothy Redaelli <tredaelli>
Date:   Tue Sep 28 11:13:20 2021 +0200

    redhat: remove mlx4 support
    
    Resolves: #1998122

commit 93d1fa0bdf8a5754840ed8e127111e140c488e12
Author: Ilya Maximets <i.maximets>
Date:   Tue Sep 28 13:17:21 2021 +0200

    ovsdb: Don't let transaction history grow larger than the database.
    
    commit 317b1bfd7dd315e241c158e6d4095002ff391ee3
    Author: Ilya Maximets <i.maximets>
    Date:   Tue Sep 28 13:17:21 2021 +0200
    
        ovsdb: Don't let transaction history grow larger than the database.
    
        If user frequently changes a lot of rows in a database, transaction
        history could grow way larger than the database itself.  This wastes
        a lot of memory and also makes monitor_cond_since slower than
        usual monotor_cond if the transaction id is old enough, because
        re-construction of the changes from a history is slower than just
        creation of initial database snapshot.  This is also the case if
        user deleted a lot of data, so transaction history still holds all of
        it while the database itself doesn't.
    
        In case of current lb-per-service model in ovn-kubernetes, each
        load-balancer is added to every logical switch/router.  Such a
        transaction touches more than a half of a OVN_Northbound database.
        And each of these transactions is added to the transaction history.
        Since transaction history depth is 100, in worst case scenario,
        it will hold 100 copies of a database increasing memory consumption
        dramatically.  In tests with 3000 LBs and 120 LSs, memory goes up
        to 3 GB, while holding at 30 MB if transaction history disabled in
        the code.
    
        Fixing that by keeping count of the number of ovsdb_atom's in the
        database and not allowing the total number of atoms in transaction
        history to grow larger than this value.  Counting atoms is fairly
        cheap because we don't need to iterate over them, so it doesn't have
        significant performance impact.  It would be ideal to measure the
        size of individual atoms, but that will hit the performance.
        Counting cells instead of atoms is not sufficient, because OVN
        users are adding hundreds or thousands of atoms to a single cell,
        so they are largely different in size.
    
        Signed-off-by: Ilya Maximets <i.maximets>
        Acked-by: Han Zhou <hzhou>
        Acked-by: Dumitru Ceara <dceara>
    
    Reported-at: https://bugzilla.redhat.com/2012949
    Signed-off-by: Ilya Maximets <i.maximets>

commit e8a363db4904a9d6e8381ebca26a744a0729ff12
Author: Ilya Maximets <i.maximets>
Date:   Sat Oct 16 03:20:23 2021 +0200

    ovsdb: transaction: Incremental reassessment of weak refs.
    
    commit 4dbff9f0a68579241ac1a040726be3906afb8fe9
    Author: Ilya Maximets <i.maximets>
    Date:   Sat Oct 16 03:20:23 2021 +0200
    
        ovsdb: transaction: Incremental reassessment of weak refs.
    
        The main idea is to not store list of weak references in the source
        row, so they all don't need to be re-checked/updated on every
        modification of that source row.  The point is that source row already
        knows UUIDs of all destination rows stored in the data, so there is no
        much profit in storing this information somewhere else.  If needed,
        destination row can be looked up and reference can be looked up in the
        destination row.  For the fast lookup, destination row now stores
        references in a hash map.
    
        Weak reference structure now contains the table and uuid of a source
        row instead of a direct pointer.  This allows to replace/update the
        source row without breaking any weak references stored in destination
        rows.
    
        Structure also now contains the key-value pair of atoms that triggered
        creation of this reference.  These atoms can be used to quickly
        subtract removed references from a source row.  During reassessment,
        ovsdb now only needs to care about new added or removed atoms, and
        atoms that got removed due to removal of the destination rows, but
        these are marked for reassessment by the destination row.
    
        ovsdb_datum_subtract() is used to remove atoms that points to removed
        or incorrect rows, so there is no need to re-sort datum in the end.
    
        Results of an OVN load-balancer benchmark that adds 3K load-balancers
        to each of 120 logical switches and 120 logical routers in the OVN
        sandbox with clustered Northbound database and then removes them:
    
        Before:
    
          %CPU  CPU Time  CMD
          86.8  00:16:05  ovsdb-server nb1.db
          44.1  00:08:11  ovsdb-server nb2.db
          43.2  00:08:00  ovsdb-server nb3.db
    
        After:
    
          %CPU  CPU Time  CMD
          54.9  00:02:58  ovsdb-server nb1.db
          33.3  00:01:48  ovsdb-server nb2.db
          32.2  00:01:44  ovsdb-server nb3.db
    
        So, on a cluster leader the processing time dropped by 5.4x, on
        followers - by 4.5x.  More load-balancers - larger the performance
        difference.  There is a slight increase of memory usage, because new
        reference structure is larger, but the difference is not significant.
    
        Signed-off-by: Ilya Maximets <i.maximets>
        Acked-by: Dumitru Ceara <dceara>
    
    Reported-at: https://bugzilla.redhat.com/2005958
    Signed-off-by: Ilya Maximets <i.maximets>

commit f5366890c5d5d1a805173cbaabfe2a2d0e4ffac1
Merge: 4682b7669 c221c8e61
Author: Open vSwitch CI <ovs-ci>
Date:   Wed Oct 27 23:49:57 2021 -0400

    Merging upstream branch-2.16
    
    Commit list:
    c221c8e613 datapath-windows:Reset PseudoChecksum value only for TX direction offload case

commit 4682b766949ab42610269b2bcf01d963273fd0ba
Merge: cce913794 b79f0369f
Author: Open vSwitch CI <ovs-ci>
Date:   Wed Oct 27 05:34:42 2021 -0400

    Merging upstream branch-2.16
    
    Commit list:
    b79f0369f2 ci: Make linux-prepare trust system installs.

commit cce913794eae37ef615f402b6657e933f0680043
Merge: 29f01c4fd 2a4c87f30
Author: Open vSwitch CI <ovs-ci>
Date:   Mon Oct 25 12:07:10 2021 -0400

    Merging upstream branch-2.16
    
    Commit list:
    2a4c87f300 Prepare for 2.16.2.
    aaa1439b8e Set release date for 2.16.1.

commit 29f01c4fdb0046138469ff70c1204071be31a96b
Merge: 2546fa964 108176ab5
Author: Open vSwitch CI <ovs-ci>
Date:   Thu Oct 21 17:03:37 2021 -0400

    Merging upstream branch-2.16
    
    Commit list:
    108176ab5a github: Stick to python 3.9.

commit 2546fa9646400b441aec53307d837d1771d3d2a8
Merge: d572c95f6 5c5e34603
Author: Open vSwitch CI <ovs-ci>
Date:   Tue Oct 19 16:04:54 2021 -0400

    Merging upstream branch-2.16
    
    Commit list:
    5c5e34603b datapath-windows: add layers when adding the deferred actions

commit d572c95f696e32bc2b56ed322de044f814f9e427
Merge: 557ca689f 458a4f75f
Author: Open vSwitch CI <ovs-ci>
Date:   Thu Oct 14 06:05:09 2021 -0400

    Merging upstream branch-2.16
    
    Commit list:
    458a4f75f3 ofproto-dpif-xlate: Fix zone set from non-frozen-metadata fields.

commit 557ca689f7066856e4987d712cdb08dd93af5412
Merge: a6c477039 6d8190584
Author: Open vSwitch CI <ovs-ci>
Date:   Wed Oct 13 10:05:02 2021 -0400

    Merging upstream branch-2.16
    
    Commit list:
    6d8190584a dpif-netdev: Fix use-after-free on PACKET_OUT of IP fragments.
    44a66cc1d0 tunnel-push-pop.at: Mask source port in tunnel header.

commit a6c4770398f836044d78af3e6e6fbc2efe5241fa
Merge: c1145b523 27a5848a3
Author: Open vSwitch CI <ovs-ci>
Date:   Tue Oct 12 17:53:40 2021 -0400

    Merging upstream branch-2.16
    
    Commit list:
    27a5848a33 ovs-ctl: Add missing description for --ovs-vswitchd-options and --ovsdb-server-options to usage().
    0300d0c0c2 dpdk-stub: Change the ERR log to DBG.
    cdd6dd821d dpif-netlink: Fix feature negotiation for older kernels.
    c2682c42cb dpif-netdev: Fix pmd thread comments to include SMC.
    9377f4a465 python: idl: Avoid sending transactions when the DB is not synced up.

commit c1145b523661442fddd34ef25f91809d39ce93df
Merge: 5c0513317 0fd17fbb0
Author: Open vSwitch CI <ovs-ci>
Date:   Tue Oct 12 11:49:23 2021 -0400

    Merging upstream branch-2.16
    
    Commit list:
    0fd17fbb09 ipf: release unhandled packets from the batch