RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1917398 - Rebase nftables to current upstream release
Summary: Rebase nftables to current upstream release
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 9
Classification: Red Hat
Component: nftables
Version: 9.1
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: 9.1
Assignee: Phil Sutter
QA Contact: Tomas Dolezal
URL:
Whiteboard:
Depends On: 1917399
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-01-18 12:33 UTC by Phil Sutter
Modified: 2023-01-16 10:12 UTC (History)
2 users (show)

Fixed In Version: nftables-1.0.4-2.el9
Doc Type: Rebase: Bug Fixes and Enhancements
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-11-15 11:22:16 UTC
Type: Bug
Target Upstream Version:
Embargoed:
pm-rhel: mirror+


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2022:8381 0 None None None 2022-11-15 11:22:20 UTC

Description Phil Sutter 2021-01-18 12:33:11 UTC
There are over 300 upstream commits between the base version for RHEL8 (v0.9.3)
and the most recent upstream release (v0.9.8). Despite the 38 commits which
have been selectively backported, RHEL certainly misses some features and fixes
from upstream. Among those are:

- ct id match support
- protocol context tracking fixes with nat statements
- frag-needed reject response
- multi-statement support for set elements
- proper icmp header matching (intra-header dependencies, correct
  deserialization)
- nat maps containing both address and port
- implicit chain bindings ("anonymous chains")
- support variables in log prefix string
- support variables in 'device' arguments of chains and flowtables
- 'typeof' keyword to simplify set declarations
- inet family ingress hook support
- fix for stale table objects in cache
- detect uid != 0 and complain
- reduce overhead when printing errors with large input
- optimize expression handler lookup via hashtable to speedup ruleset
  delinearization
- try to expand only "add" commands (others don't need this)
- support libedit as an alternative to readline which may have licensing issues.
- improved help output by grouping options
- support slave device matching (meta sdif/sdifname)
- support for extended netlink error reporting
- comment support for chains, objects, tables and set declarations
- support flowtables without device as well as adding and removing a device
  from them
- support intervals in nat maps

Comment 3 Phil Sutter 2022-06-10 12:07:25 UTC
I looked ad the (non-gating) failures in CI for the rebase:

Sanity/upstream-testsuite:
- from tests/py, kernel lacks:
  - support for netfilter egress hook
  - support for TCP option removal in exthdr expression
  - support for inner header matching in payload expression
- in tests/shell:
  - tests/shell/testcases/chains/0021prio_0 uses egress hook
  - tests/shell/testcases/maps/typeof_integer_0 uses inner header expression
    (actually, inner header offset in raw payload expression)
  - tests/shell/testcases/maps/typeof_raw_0 and
    tests/shell/testcases/sets/typeof_raw_0 also use inner header

Regression/rhel-8-1-rebase-patches:
- "segtree debug crashing" test checks segtree debug output for an anonymous
  set with ranges which is empty - segtree code got a rewrite, probably debug
  output is less verbose now
- There is a warning in dmesg output[1] (net/core/flow_dissector.c:985) - this
  needs investigation, but is probably a kernel bug

Regression/Fix-for-poor-performance-of-JSON-API-echo-support:
- This was written for bug 1835300, the fix claims reduction in restore
  run-time from 27s down to 1.4s.
- Same restore takes 1m35s on a 1minutetip VM even without --echo.
- According to perf[2], most of the time is spent sorting the set elements.
- So we're regressing, but for a different bug.


[1] http://artifacts.osci.redhat.com/baseos-ci/brew-build/45/87/87/45878743/https___baseos-jenkins.rhev-ci-vms.eng.rdu2.redhat.com-ci-artemis/28541/tmp1fnuUn.01/recipes/1/tasks/13/results/1654794774/logs/resultoutputfile.log
[2] FTR, here's the top of perf report:
  23.16%  nft      libnftables.so.1.1.0  [.] expr_msort_value
  16.59%  nft      libnftables.so.1.1.0  [.] list_expr_sort
  14.97%  nft      libgmp.so.10.4.1      [.] __gmpn_copyi_x86_64
   7.27%  nft      libnftables.so.1.1.0  [.] set_to_range
   6.26%  nft      libc.so.6             [.] _int_free
   4.68%  nft      libgmp.so.10.4.1      [.] __gmpz_set
   4.33%  nft      libnftables.so.1.1.0  [.] interval_set_eval.lto_priv.0
   3.76%  nft      libc.so.6             [.] malloc
   3.15%  nft      libnftables.so.1.1.0  [.] range_expr_value_high
   2.59%  nft      libc.so.6             [.] cfree.5
   1.82%  nft      libgmp.so.10.4.1      [.] __gmpz_cmp

Comment 7 Phil Sutter 2022-06-24 11:35:48 UTC
(In reply to Phil Sutter from comment #3)
> Regression/Fix-for-poor-performance-of-JSON-API-echo-support:
> - This was written for bug 1835300, the fix claims reduction in restore
>   run-time from 27s down to 1.4s.
> - Same restore takes 1m35s on a 1minutetip VM even without --echo.
> - According to perf[2], most of the time is spent sorting the set elements.
> - So we're regressing, but for a different bug.

I reported the issue upstream. It is caused by new overlap detection and
element auto-merging code which causes significant overhead for each individual
'add element' command. If these commands are combined into a single one adding
all elements at once, the overhead is not noticeable.

To avoid the problem, Pablo implemented automatic command coalescing for 'add
element' commands. Hence I will backport these commits:

8fafe4e6b5b30 ("tests: shell: runtime set element automerge")
498a5f0c219d8 ("rule: collapse set element commands")
87ba510fc704f ("intervals: do not report exact overlaps for new elements")

There are two related fixes to be backported as well:

d434de8b50dcf ("intervals: do not empty cache for maps")
59e3a59221fb8 ("intervals: Do not sort cached set elements over and over again")

Comment 18 errata-xmlrpc 2022-11-15 11:22:16 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (nftables bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:8381


Note You need to log in before you can comment on or make changes to this bug.