Bug 2052886

Upstream PR:
https://github.com/redhat-performance/tuned/pull/413

On the 4.18.0-364.el8.x86_64 it seems the /sys/module/nf_conntrack/parameters/hashsize changes are not propagated to the /proc/sys/net/netfilter/nf_conntrack_buckets:

# cat /sys/module/nf_conntrack/parameters/hashsize
16384

# cat /proc/sys/net/netfilter/nf_conntrack_buckets
16384

# echo 1048576 > /sys/module/nf_conntrack/parameters/hashsize
# cat /sys/module/nf_conntrack/parameters/hashsize
1048576

# cat /proc/sys/net/netfilter/nf_conntrack_buckets
16384

On the other hand on the 5.16.5-100.fc34.x86_64:

# cat /sys/module/nf_conntrack/parameters/hashsize
16384

# cat /proc/sys/net/netfilter/nf_conntrack_buckets
16384

# echo 1048576 > /sys/module/nf_conntrack/parameters/hashsize
# cat /sys/module/nf_conntrack/parameters/hashsize
1048576

# cat /proc/sys/net/netfilter/nf_conntrack_buckets
1048576

Is it kernel bug? I haven't checked the kernel code yet.

Yup, it seems like RHEL-8.6.0 kernel bug:
/* size the user *wants to set */
 static unsigned int nf_conntrack_htable_size_user __read_mostly;
 
 static int
 nf_conntrack_hash_sysctl(struct ctl_table *table, int write,
-			 void __user *buffer, size_t *lenp, loff_t *ppos)
+			 void *buffer, size_t *lenp, loff_t *ppos)
 {
 	int ret;
 
+	/* module_param hashsize could have changed value */
+	nf_conntrack_htable_size_user = nf_conntrack_htable_size;
+
 	ret = proc_dointvec(table, write, buffer, lenp, ppos);
 	if (ret < 0 || !write)
 		return ret;

So the question is @Paolo Abeni does the TuneD fix work for you and can we just change the test (i.e. to check the sysfs, not the sysctl)?

(In reply to Jaroslav Škarvada from comment #16)
> Yup, it seems like RHEL-8.6.0 kernel bug:
> /* size the user *wants to set */
>  static unsigned int nf_conntrack_htable_size_user __read_mostly;
>  
>  static int
>  nf_conntrack_hash_sysctl(struct ctl_table *table, int write,
> -			 void __user *buffer, size_t *lenp, loff_t *ppos)
> +			 void *buffer, size_t *lenp, loff_t *ppos)
>  {
>  	int ret;
>  
> +	/* module_param hashsize could have changed value */
> +	nf_conntrack_htable_size_user = nf_conntrack_htable_size;
> +
>  	ret = proc_dointvec(table, write, buffer, lenp, ppos);
>  	if (ret < 0 || !write)
>  		return ret;
> 
> So the question is @Paolo Abeni does the TuneD fix work for you 

Do you mean tuned mr/413? it WFM, but I tested it on a rhel-9 kernel, but we need it to be effective on rhel-8 too.

> and can we
> just change the test (i.e. to check the sysfs, not the sysctl)?

uhmm... then the data exposed by the kernel will be confusing. Userspace usually check only sysctl.

I think we need to use 'sys.net.netfilter.nf_conntrack_buckets' in tuned profiles instead of 'nf_conntrack_hashsize' or to patch the rhel-8 kernel. The latter can take quite a long timeframe and will need and explicit exception at this point. I think the first option is more doable.

(In reply to Paolo Abeni from comment #17)
> (In reply to Jaroslav Škarvada from comment #16)
> > Yup, it seems like RHEL-8.6.0 kernel bug:
> > /* size the user *wants to set */
> >  static unsigned int nf_conntrack_htable_size_user __read_mostly;
> >  
> >  static int
> >  nf_conntrack_hash_sysctl(struct ctl_table *table, int write,
> > -			 void __user *buffer, size_t *lenp, loff_t *ppos)
> > +			 void *buffer, size_t *lenp, loff_t *ppos)
> >  {
> >  	int ret;
> >  
> > +	/* module_param hashsize could have changed value */
> > +	nf_conntrack_htable_size_user = nf_conntrack_htable_size;
> > +
> >  	ret = proc_dointvec(table, write, buffer, lenp, ppos);
> >  	if (ret < 0 || !write)
> >  		return ret;
> > 
> > So the question is @Paolo Abeni does the TuneD fix work for you 
> 
> Do you mean tuned mr/413? it WFM, but I tested it on a rhel-9 kernel, but we
> need it to be effective on rhel-8 too.
> 
> > and can we
> > just change the test (i.e. to check the sysfs, not the sysctl)?
> 
> uhmm... then the data exposed by the kernel will be confusing. Userspace
> usually check only sysctl.
> 
> I think we need to use 'sys.net.netfilter.nf_conntrack_buckets' in tuned
> profiles instead of 'nf_conntrack_hashsize' or to patch the rhel-8 kernel.
> The latter can take quite a long timeframe and will need and explicit
> exception at this point. I think the first option is more doable.

Thinking again about it, this scenario is not different from what we had before (e.g. tuned behavior did not change). So we could just update the test to check '/sys/module/nf_conntrack/parameters/hashsize'  instead of net.netfilter.nf_conntrack_buckets. In the longer run, I think it would be better additionally fix the kernel, so that the system view is consistent.

@Robin: could you please modify the test as described above?

Thanks for filling the kernel bug 2053628.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (tuned bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:2106