Bug 2011536

+++ This bug was initially created as a clone of Bug #2005195 +++

Description of problem:
running sosreport creates iptables built-in tables in nftables

When generating sosreport, these commands are executed in networking plugin:
...
2021-09-06 11:43:15,893 INFO: [plugin:networking] collecting output of 'ip6tables -t nat -nvL'
2021-09-06 11:43:15,908 INFO: [plugin:networking] collecting output of 'ip6tables -t mangle -nvL'
2021-09-06 11:43:15,923 INFO: [plugin:networking] collecting output of 'ip6tables -t filter -nvL'
2021-09-06 11:43:16,225 INFO: [plugin:networking] collecting output of 'iptables -vnxL'
2021-09-06 11:43:16,242 INFO: [plugin:networking] collecting output of 'ip6tables -vnxL'

The problem is that these calls create iptables compatibility built-in tables in nft, which may not be desired. 

For example:
# nft list ruleset | grep table
table ip v4 {
table ip6 v6 {
table ip6 nat {
# ip6tables -t mangle -nvL > /dev/null
# nft list ruleset | grep table
table ip v4 {
table ip6 v6 {
table ip6 nat {
table ip6 mangle { // << NOTE

// the compat tables are added by iptables code:
int nft_rule_list(struct nft_handle *h, const char *chain, const char *table,
          int rulenum, unsigned int format)
{
    const struct nft_family_ops *ops = h->ops;
    struct nftnl_chain_list *list;
    struct nftnl_chain_list_iter *iter;
    struct nftnl_chain *c;
    bool found = false;

    nft_xt_builtin_init(h, table);
    nft_assert_table_compatible(h, table, chain);
...

static int nft_xt_builtin_init(struct nft_handle *h, const char *table)
{
    const struct builtin_table *t;
    t = nft_table_builtin_find(h, table);
    if (t == NULL)
        return -1;
    if (nft_table_initialized(h, t->type))
        return 0;
    if (nft_table_builtin_add(h, t) < 0)
        return -1;
    nft_chain_builtin_init(h, t);
    h->cache->table[t->type].initialized = true;
    return 0;
}

// so no way to bypass that - iptables/ip6tables must not be called to avoid creation of compat tables

// looking at sosreport upstream
https://github.com/sosreport/sos/blame/main/sos/report/plugins/networking.py

// uses: /usr/sbin/iptables-save (which doesn't create compat tables), the rest is in ufw plugin
// sos-4.1 removes all the iptables calls (except for iptable-save) from networking and adds ufw.py

// in 4.0:
// https://github.com/sosreport/sos/blob/4.0/sos/report/plugins/networking.py
        # collect the same for ip6tables
        try:
            ip_tables_names = open("/proc/net/ip6_tables_names").read()
        except IOError:
            ip_tables_names = "nat\nmangle\nfilter\n"
        for table in ip_tables_names.splitlines():
            self.collect_ip6table(table)


// the collects have predicates on ip*table_* kernel module
    def collect_iptable(self, tablename):
        """ Collecting iptables rules for a table loads either kernel module
        of the table name (for kernel <= 3), or nf_tables (for kernel >= 4).
        If neither module is present, the rules must be empty."""

        modname = "iptable_" + tablename
        cmd = "iptables -t " + tablename + " -nvL"
        self.add_cmd_output(
            cmd,
            pred=SoSPredicate(self, kmods=[modname, 'nf_tables']))

    def collect_ip6table(self, tablename):
        """ Same as function above, but for ipv6 """

        modname = "ip6table_" + tablename
        cmd = "ip6tables -t " + tablename + " -nvL"
        self.add_cmd_output(
            cmd,
            pred=SoSPredicate(self, kmods=[modname, 'nf_tables']))

// the default predicate is 'any' so we run the command if ip[6]table_xyz OR nf_tables is installed

Customer monitors nftables and doesn't want the compat tables. They get security alert on every sosreport run. Plus compat tables may have (little) performance impact (as they connect some chain to hooks and have some counters too).
https://wiki.nftables.org/wiki-nftables/index.php/Main_differences_with_iptables

I believe sosreport should avoid calling iptables command if iptables are not really in use. 

We do have possible workaround - skip networking plugin or replace iptables binary (removing is not an option due to dependencies). But the current behavior is incorrect and should be addressed properly.

--- Additional comment from Christian Horn on 2021-09-17 05:14:45 UTC ---

I looked at freshly deployed rhel8 without nft rules,
after running "sosreport":

- rhel8.1 and 8.2: still have no nft tables ("nft list tables" has no output)
- rhel8.3 and 8.3: have nft tables.

We have customers who monitor their rules, so this is not desired.
One of the rules of sosreport is also "leave no trace", IIRC.

--- Additional comment from Pavel Moravec on 2021-09-17 16:15:28 UTC ---

(I expect the root cause here is *different* than in https://bugzilla.redhat.com/show_bug.cgi?id=2001096#c10 ?)

Be aware, firewall_tables in upstream has separated the iptables commands from networking plugin. I expect the same applies there?

> I believe sosreport should avoid calling iptables command if iptables are not really in use. 

How to identify that "iptables are not really in use"? By some nft command output ("nft list ruleset | grep table ip")? Such that we should extend the predicate of an ip[|6]tables command by *also* that command output match?

--- Additional comment from Christian Horn on 2021-09-21 07:59:18 UTC ---

Doh, how could I oversee bz2001096 ?  It's in the same area indeed..

> How to identify that "iptables are not really in use"? By some nft
> command output ("nft list ruleset | grep table ip")? Such that we
> should extend the predicate of an ip[|6]tables command by *also*
> that command output match?

So my idea was to check as specific as possible before
each of the commands, like this in pseudo code:

if ( nft list ruleset | grep -q 'table ip6 mangle' )
  ip6tables -t mangle -nvL
if ( nft list ruleset | grep -q 'table ip6 net' )
  ip6tables -t nat -nvL
if ( nft list ruleset | grep -q 'table ip6 filter' )
  ip6tables -t filter -nvL
if ( nft list ruleset | grep -q 'table ip filter' )
  iptables -t filter -nvL
if ( nft list ruleset | grep -q 'table ip6 filter' )
  ip6tables -t filter -nvL

Our partner points out that grep -q is sometimes not
nice, so this here is more direct and might be less
error prone for checking (i.e. when table filter2
also exists, and we do not want to use regex:

[root@rhel8 ~]# nft -nt list table ip mangle
table ip mangle {
...
[root@rhel8 ~]# echo $?
0
[root@rhel8 ~]# nft delete table ip mangle
[root@rhel8 ~]# nft -nt list table ip mangle
Error: No such file or directory; did you mean table ‘mangle’ in family ip6?
list table ip mangle
              ^^^^^^
[root@rhel8 ~]# echo $?
1

We can then redirect STDOUT and STDERR from the
"nft -nt list table" to /dev/null, as we just need the
return code.