Bug 634735

Summary:	net.bridge.bridge-nf-call-iptables setting in sysctl.conf doesn't match setting in kernel
Product:	Red Hat Enterprise Linux 6	Reporter:	Laine Stump <laine>
Component:	kernel	Assignee:	Neil Horman <nhorman>
Status:	CLOSED NOTABUG	QA Contact:	Red Hat Kernel QE team <kernel-qe>
Severity:	medium	Docs Contact:
Priority:	low
Version:	6.0	CC:	jolsa, nhorman, pknirsch, rhughes, tgraf
Target Milestone:	rc
Target Release:	---
Hardware:	All
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:
Clones:	634736 (view as bug list)		Environment:
Last Closed:	2011-01-04 18:22:09 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Laine Stump 2010-09-16 20:29:00 UTC

Description of problem:

On a system that has no bridges defined in /etc/sysconfig/network-scripts/ifcfg-*, but does have virtual networks defined in libvirtd, when system boot is complete the setting of net.bridge.bridge-nf-call-iptables is always "1", regardless of what is set in /etc/sysctl.conf.

Version-Release number of selected component (if applicable):

bridge-utils-=1.2-0.el6.x86_64

How reproducible:

100%

Steps to Reproduce:
1. install a new RHEL6/F13 system with the Virtualization package included. (or just make sure that your current system has no bridges defined in /etc/sysconfig/network-scripts, and does have at least one virtual network defined in libvirt (the basic libvirt install has a virtual network, "default", defined)

2. Make sure that /etc/sysctl.conf contains:

   net.bridge.bridge-nf-call-iptables = 0

(this is the default on a new RHEL6 install).

3. After the system has completed booting, run:

   sysctl net.bridge.bridge-nf-call-iptables

Actual results:

   net.bridge.bridge-nf-call-iptables = 1

Expected results:

   net.bridge.bridge-nf-call-iptables = 0

Additional info:

Bug 629206 and Bug 633346 suffer some of the blow-by from this - although their root cause is different, the obvious workaround (setting net.bridge.bridge-nf-call-iptables = 0) doesn't always work, because that setting doesn't always make it into the kernel.

Another related item is given in the description of a patch to the netcf library:

  https://fedorahosted.org/pipermail/netcf-devel/2010-September/000459.html

The cause of the problem is:


1) Before the bridge module is loaded, those variables don't yet exist, so any sysctl.conf reload has no effect on them

2) The network service reloads /etc/sysctl.conf, and happens to do that after it has caused the bridge module to load, but it only loads the bridge module if there are bridge interfaces to ifup.

3) the default for this setting (as well as *-ip6tables and *-arptables) in the bridge module is 1

4) no other init script happens to reload sysctl.conf after the network service.

5) if libvirt has any virtual networks, those require a bridge interface to be created, and one of the involved commands (brctl?) causes the bridge module to be loaded, but doesn't reload sysctl.conf, so we're left with the module's default (1) rather than the configured default in sysctl.conf.

I *think* the fix is that either the initialization of the bridge module itself, or whatever utility does the modprobe of the bridge module, needs to run "sysctl -a -p" to cause the configured defaults to be loaded as soon as possible after the bridge module is loaded. It's not correct to rely on someone else (eg, the network service) to know that this must be done.

Comment 1 Laine Stump 2010-09-16 20:33:03 UTC

Cloned for F13 as Bug 634735

Comment 3 Laine Stump 2010-09-23 17:48:35 UTC

Actually, the cloned bug is Bug 634736

Comment 4 Phil Knirsch 2010-12-17 15:13:54 UTC

I'm pretty sure this should be properly handled in the kernel module as thats the point of origin of the problem.

Moving to kernel component.

Thanks & regards, Phil

Comment 5 Neil Horman 2011-01-03 15:14:49 UTC

Triage assignment.  If you feel this bug doesn't belong to you, or that it cannot be handled in a timely fashion, please contact me for re-assignment

Comment 6 Neil Horman 2011-01-04 18:11:17 UTC

This can't be solved in the kernel.  User space events can trigger the need for module load which the kernel implements by forking a copy of modprobe directly.  The kernel has no way of knowing if it should run sysctl commands post module install or not, nor what teh settings should be as it doesn't have access to the /etc/sysctl.conf file.

What could be done is a modification to the modprobe utility to get it to optionally rescan /etc/sysctl conf and apply changes after a module install.  I'l lsee if I can whip together a patch for that.

Comment 7 Neil Horman 2011-01-04 18:22:09 UTC

actually, scratch what I just said.  Turns out there is already a way to do this administratively, so theres no need for change.  All you need to do is use the install command in modprobe.conf (or any of the modprobe conf files).  the install command lets you run any command that you would like when loading a module.  So if you want to guarantee that a module has its sysctl settings applied when its not guaranteed that the module will be loaded at rc.sysinit time, you only need to add this line to your modprobe config:

install bridge /sbin/modprobe --ignore-install bridge && sysctl -p

That will reapply the /etc/sysctl file once the bridge module is installed and set your bridge sysctl values properly.

Comment 8 Laine Stump 2011-01-04 18:45:57 UTC

Which package should be creating this modprobe config?

AFAIK, libvirt purposefully avoids having anything to do with loading modules (especially something like the bridge module - libvirt doesn't even have any idea that such a thing exists; it just calls the ioctls() to create a bridge), and I would consider setting up a command to be run at module load time to be in that category.

Also, this isn't a libvirt-specific problem - it's endemic to the way that /etc/sysctl.conf interacts with all kernel modules, which may be used by any number of applications, and those applications may or may not even know that they've done something that causes a module to be loaded. It's very likely this same problem is playing out (or will play out) for some other set of sysctl values/modules/applications.

But back to the specific case - should this be added to dist.conf (which yum says is a part of inittools)?