Bug 2228975

Summary: ovsdb-server doesn't limit transaction history size on initial database file read
Product: Red Hat Enterprise Linux Fast Datapath Reporter: ovs-bugzilla <ovs-bugzilla>
Component: openvswitch2.17Assignee: Ilya Maximets <i.maximets>
Status: MODIFIED --- QA Contact: Jianlin Shi <jishi>
Severity: high Docs Contact:
Priority: unspecified    
Version: FDP 23.KCC: ctrautma, jhsiao, ralongi
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: openvswitch2.17-2.17.0-110.el8fdp Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description ovs-bugzilla 2023-08-03 19:13:11 UTC 
+++ This bug was initially created as a clone of Bug #2228464 +++

In a clustered database mode, after restart, ovsdb-server may consume
a lot of memory during initial database file read:

  |00004|memory|INFO|95650400 kB peak resident set size after 96.9 seconds
  |00005|memory|INFO|atoms:3083346 cells:1838767 monitors:0 raft-log:123309 txn-history:123307 txn-history-atoms:1647022868

In the example above, it's a Northbound OVN database with 123K small transaction
records in the file.  The total file size is just 150 MB.  Fully compacted - 80 MB.
During the initial database file read, ovsdb-server allocates 95 GB of RAM to
store these transactions in a history.  History will be drained right after the
initial read is completed, but the memory may not be returned to the system
until the next compaction (glibc behavior).  The process may just be killed
before finishing the read if there is no enough memory in the system.

How to reproduce:

1. Create OVN setup with clustered databases.
2. Execute 100K small Northbound database updates in a short time, but make
   sure that ovsdb-server didn't compact it.
3. Re-start ovsdb-server.

Expected results:

Just restarted ovsdb-server process should not consume significantly more
memory than it consumed before the restart.
Comment 1 ovs-bugzilla 2023-08-03 19:13:15 UTC 
* Thu Aug 03 2023 Open vSwitch CI <ovs-ci> - 2.17.0-110
- Merging upstream branch-2.17 [RH git: 71b356438e]
    Commit list:
    fe98b0c1f9 ovsdb-tool: Fix json leak while showing clustered log.
    44722bbda3 ovsdb-server: Fix excessive memory usage on DB open. (#2228464)
    9db221fcd8 tests: Add ovsdb execution cases for set size constraints.
    3cfe388cb8 ovsdb: relay: Fix handling of XOR updates with size constraints.
    f4d15497f8 ovsdb: file: Fix diff application to a default column value.
    7864ed557e ovsdb: file: Fix inability to read diffs that violate type size.
    97d91ad2d6 ovs-tcpdump: Clear auto-assigned ipv6 address of mirror port.