Red Hat Bugzilla – Full Text Bug Listing
|Summary:||Linux Bridge NAT doesn't work due to net.bridge.bridge-nf-call-iptables=1|
|Product:||Red Hat OpenStack||Reporter:||Etsuji Nakai <enakai>|
|Component:||openstack-quantum||Assignee:||Terry Wilson <twilson>|
|Status:||CLOSED ERRATA||QA Contact:||Ofer Blaut <oblaut>|
|Version:||2.0 (Folsom)||CC:||gkotton, prmarino1, rkukura|
|Fixed In Version:||2012.2.1-7.el6ost||Doc Type:||Bug Fix|
|Doc Text:||Story Points:||---|
|Last Closed:||2013-03-21 15:03:20 EDT||Type:||Bug|
|oVirt Team:||---||RHEL 7.3 requirements from Atomic Host:|
Description Etsuji Nakai 2012-11-18 01:20:23 EST
Description of problem: When using Linux Bridge plugin, NAT function of the virtual router doesn't work, probably, due to "net.bridge.bridge-nf-call-iptables=1". I'm not sure where this is set. My dirty workaround is to place the following code in /etc/rc.local. ----------- rc=1 while [[ $rc -ne 0 ]]; do sysctl net.bridge.bridge-nf-call-iptables | grep "1" rc=$? sleep 5 done sysctl -w net.bridge.bridge-nf-call-iptables=0 sysctl -w net.bridge.bridge-nf-call-ip6tables=0 sysctl -w net.bridge.bridge-nf-call-arptables=0 ----------- My setup is: # rpm -qa | grep quantum openstack-quantum-linuxbridge-2012.2-2.1.el6.noarch python-quantumclient-2.1.1-0.el6.noarch python-quantum-2012.2-2.1.el6.noarch openstack-quantum-2012.2-2.1.el6.noarch
Comment 1 Etsuji Nakai 2012-11-18 01:30:40 EST
My guess about the reason of this problem is, in the iptables's nat chain, before packets to be SNAT-ed reach the "quantum-l3-agent-snat" chain, it is ACCEPT-ed at the "quantum-l3-agent-POSTROUTING"'s following rule (if net.bridge.bridge-nf-call-iptables=1). Chain quantum-l3-agent-POSTROUTING (1 references) pkts bytes target prot opt in out source destination 10 600 ACCEPT all -- !qg-18ac7fca-0a !qg-18ac7fca-0a 0.0.0.0/0 0.0.0.0/0 ! ctstate DNAT The relevant parts of the nat tables is as below. -------------- Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination 10 600 quantum-l3-agent-POSTROUTING all -- * * 0.0.0.0/0 0.0.0.0/0 0 0 quantum-postrouting-bottom all -- * * 0.0.0.0/0 0.0.0.0/0 0 0 nova-compute-POSTROUTING all -- * * 0.0.0.0/0 0.0.0.0/0 0 0 nova-api-POSTROUTING all -- * * 0.0.0.0/0 0.0.0.0/0 0 0 nova-postrouting-bottom all -- * * 0.0.0.0/0 0.0.0.0/0 Chain quantum-l3-agent-POSTROUTING (1 references) pkts bytes target prot opt in out source destination 10 600 ACCEPT all -- !qg-18ac7fca-0a !qg-18ac7fca-0a 0.0.0.0/0 0.0.0.0/0 ! ctstate DNAT Chain quantum-postrouting-bottom (1 references) pkts bytes target prot opt in out source destination 0 0 quantum-l3-agent-snat all -- * * 0.0.0.0/0 0.0.0.0/0 Chain quantum-l3-agent-snat (1 references) pkts bytes target prot opt in out source destination 0 0 quantum-l3-agent-float-snat all -- * * 0.0.0.0/0 0.0.0.0/0 0 0 SNAT all -- * * 192.168.101.0/24 0.0.0.0/0 to:10.64.201.1 -----------
Comment 3 Alan Pevec 2012-11-19 06:30:43 EST
This is set in default sysctl.conf # grep bridge-nf /etc/sysctl.conf net.bridge.bridge-nf-call-ip6tables = 0 net.bridge.bridge-nf-call-iptables = 0 net.bridge.bridge-nf-call-arptables = 0 # rpm -qf /etc/sysctl.conf initscripts-9.03.31-2.el6.x86_64 So this shouldn't be an issue on default RHEL install, how do you install your machine?
Comment 4 Alan Pevec 2012-11-19 07:25:04 EST
*** Bug 877703 has been marked as a duplicate of this bug. ***
Comment 5 Etsuji Nakai 2012-11-19 09:09:23 EST
I'm using the default RHEL6.3 installation. However, at the boot time of RHEL6.3, because the bridge kernel module is not loaded, net.bridge.bridge-nf-call-* in sysctl.conf is simply ignored as below. # sysctl -p error: "net.bridge.bridge-nf-call-ip6tables" is an unknown key error: "net.bridge.bridge-nf-call-iptables" is an unknown key error: "net.bridge.bridge-nf-call-arptables" is an unknown key I suspect that once Quantum L2 agent has started, it loads the bridge module and set net.bridge.bridge-nf-call-* = 1. I'm not sure why the agent requires them to be 1.
Comment 6 Etsuji Nakai 2012-11-19 09:17:50 EST
Or probably, when the bridge module is loaded by the L2 agent, net.bridge.bridge-nf-call-* becomes 1 by default. If so, the L2 agent should explicitly set net.bridge.bridge-nf-call-* = 0 at that timing.
Comment 7 Etsuji Nakai 2012-11-19 09:38:59 EST
I confirmed my guess by putting the following block in /etc/rc.local. (This is called after sysctl.conf is applied, and befer L2 agent starts.) ------ modprobe bridge sysctl -w net.bridge.bridge-nf-call-iptables=0 sysctl -w net.bridge.bridge-nf-call-ip6tables=0 sysctl -w net.bridge.bridge-nf-call-arptables=0 ------ And the result is: # sysctl -a | grep bridge net.bridge.bridge-nf-call-arptables = 0 net.bridge.bridge-nf-call-iptables = 0 net.bridge.bridge-nf-call-ip6tables = 0 net.bridge.bridge-nf-filter-vlan-tagged = 0 net.bridge.bridge-nf-filter-pppoe-tagged = 0 So the problem is that net.bridge.bridge-nf-call-* in sysctl.conf is ignored if the "brdige" module is not loaded at boot time.
Comment 8 Alan Pevec 2012-11-19 13:33:11 EST
> So the problem is that net.bridge.bridge-nf-call-* in sysctl.conf is ignored if the "brdige" module is not loaded at boot time. Indeed and I see the same on F17 too. Packaging workaround in openstack-quantum-linuxbridge could be to include /etc/sysconfig/modules/openstack-quantum-linuxbridge.modules script which modprobes bridge.
Comment 9 Etsuji Nakai 2012-11-19 18:21:11 EST
I see. The same workaround worked well for me, thanks. # cat /etc/sysconfig/modules/openstack-quantum-linuxbridge.modules #!/bin/sh modprobe -b bridge >/dev/null 2>&1 exit 0
Comment 10 Paul Robert Marino 2012-12-18 12:37:27 EST
why not add a "/sbin/sysctl -p" to the end of init script for quantum? or even better quantum should be setting theses options based on need along with others.
Comment 11 Etsuji Nakai 2013-01-03 21:29:11 EST
By the way, I confirmed that this affects only when netns support is disabled. When netns support is enabled, nat table is placed under a different namespace which doesn't contain the bridge interface inside it. So the value of net.bridge.bridge-nf-call-iptables doesn't affect the rule matching. The bottom line is the requirement for net.bridge.bridge-nf-call-iptables could be different based on the configuration to be used by the customer. So hardcoding the parameter in Quantum's source code may be a little bit risky. I think the best way to deal with it is: - Use the package workaround of /etc/sysconfig/modules/openstack-quantum-linuxbridge.modules. - Describe in the Deployment Guide what parameters in /etc/sysctl.conf should be set according to the intended configuration.
Comment 12 Terry Wilson 2013-01-31 19:27:23 EST
Created attachment 691296 [details] Add sysctl modules script for linuxbridge apevec: does this patch look like a good start?
Comment 15 Ofer Blaut 2013-03-19 06:18:43 EDT
I have tested floating ip on quantum with linux bridge , it works also setting is as expected using RHEL 6.4 [root@puma34 ~(keystone_admin_tenant1)]$ lsmod | grep bridge bridge 79078 0 stp 2218 2 garp,bridge llc 5546 3 garp,bridge,stp [root@puma34 ~(keystone_admin_tenant1)]$ sysctl -a | grep bridge net.bridge.bridge-nf-call-arptables = 0 net.bridge.bridge-nf-call-iptables = 0 net.bridge.bridge-nf-call-ip6tables = 0 net.bridge.bridge-nf-filter-vlan-tagged = 0 net.bridge.bridge-nf-filter-pppoe-tagged = 0 [root@puma34 ~(keystone_admin_tenant1)]$ rpm -qa | grep quantum openstack-quantum-2012.2.3-8.el6ost.noarch
Comment 17 errata-xmlrpc 2013-03-21 15:03:20 EDT
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHBA-2013-0672.html