Bug 1705732

Summary:	RFE: nftables: speed up rule listing when only a single table is requested
Product:	Red Hat Enterprise Linux 9	Reporter:	Florian Westphal <fwestpha>
Component:	nftables	Assignee:	Phil Sutter <psutter>
Status:	CLOSED ERRATA	QA Contact:	Tomas Dolezal <todoleza>
Severity:	low	Docs Contact:
Priority:	low
Version:	9.0	CC:	psutter, qe-baseos-daemons, rkhan, todoleza
Target Milestone:	rc	Keywords:	FutureFeature, Reopened, Triaged
Target Release:	9.1	Flags:	pm-rhel: mirror+
Hardware:	All
OS:	Linux
Whiteboard:
Fixed In Version:	nftables-1.0.4-1.el9	Doc Type:	No Doc Update
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2022-11-15 11:22:16 UTC	Type:	Enhancement
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Deadline:	2022-08-01

Description Florian Westphal 2019-05-02 20:19:10 UTC

Description of problem:
 
a command like "nft list table foo"

might take several seconds, even if this table doesn't contain
a lot of rules and only small sets.

This is because nft populates its cache on startup by dumping entire ruleset,
so every table is processed even if it won't be used.

nft could instead only query the kernel for the requested table.

The caveat is that the command line might read:

nft "list table foo;list table bar"

so its not enough to just restrict fetch to first table it sees.

We also might have a mix of unrelated commands/operations
on different objects in different tables.

One way to fix this would be to maintain cache per table, and
populate it on demand only. This would also need proper invalidation
when tables get altered in mean time.

How reproducible: 100%.

Steps to Reproduce:
1. add new empty table
2. observe speed of "nft list table newtable"
3. add another new table containing e.g.a large set  (e.g. 200k entries)

Actual results:

"nft list table newtable" takes noticably longer to complete

Expected results:

"nft list table newtable" should not take longer to complete

Comment 5 RHEL Program Management 2021-02-01 07:40:40 UTC

After evaluating this issue, there are no plans to address it further or fix it in an upcoming release.  Therefore, it is being closed.  If plans change such that this issue will be fixed in an upcoming release, then the bug can be reopened.

Comment 6 Phil Sutter 2021-02-01 14:00:40 UTC

Maybe implicitly fixed by a future rebase, but worth keeping this in place to have it tested for.

Comment 9 RHEL Program Management 2021-08-01 07:27:05 UTC

After evaluating this issue, there are no plans to address it further or fix it in an upcoming release.  Therefore, it is being closed.  If plans change such that this issue will be fixed in an upcoming release, then the bug can be reopened.

Comment 10 Phil Sutter 2021-08-30 13:48:35 UTC

Still relevant. Also, a quick test with current upstream nftables shows that adding chains to one table slows down listing of another table.

Comment 12 Phil Sutter 2021-12-22 13:58:25 UTC

Support for filtering data on kernel side has been completed recently in upstream nftables. There are ~40 commits that went into src/cache.c in between those commits and RHEL8. Given that this is an RFE and there's no explicit customer interest, I hereby postpone to RHEL9.

Comment 15 RHEL Program Management 2022-03-02 07:27:21 UTC

After evaluating this issue, there are no plans to address it further or fix it in an upcoming release.  Therefore, it is being closed.  If plans change such that this issue will be fixed in an upcoming release, then the bug can be reopened.

Comment 16 Phil Sutter 2022-03-02 17:19:42 UTC

Still relevant.

Comment 18 Phil Sutter 2022-06-09 16:25:38 UTC

Likely fixed implicitly by package rebase (bug 1917398), mark as TestOnly?

Comment 20 Phil Sutter 2022-06-16 10:51:33 UTC

Here's a simple test:

| #!/bin/bash
| 
| {
| 	cat <<EOF
| flush ruleset
| table empty_table {
| }
| table full_table {
| 	chain c {
| EOF
| 	for ((i = 0; i < 50000; i++)); do
| 		echo "iifname eth0 tcp dport 22 counter accept"
| 	done
| 	cat <<EOF
| 	}
| }
| EOF
| } | nft -f -
| 
| TIMEFORMAT='%R'
| time nft list table full_table >/dev/null
| time nft list table empty_table >/dev/null

The second value printed must be significantly lower than the first one.

Comment 27 errata-xmlrpc 2022-11-15 11:22:16 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (nftables bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:8381