Bug 877704
Summary: | Linux Bridge NAT doesn't work due to net.bridge.bridge-nf-call-iptables=1 | ||||||
---|---|---|---|---|---|---|---|
Product: | Red Hat OpenStack | Reporter: | Etsuji Nakai <enakai> | ||||
Component: | openstack-quantum | Assignee: | Terry Wilson <twilson> | ||||
Status: | CLOSED ERRATA | QA Contact: | Ofer Blaut <oblaut> | ||||
Severity: | medium | Docs Contact: | |||||
Priority: | medium | ||||||
Version: | 2.0 (Folsom) | CC: | gkotton, prmarino1, rkukura | ||||
Target Milestone: | snapshot3 | Keywords: | Triaged | ||||
Target Release: | 2.1 | ||||||
Hardware: | Unspecified | ||||||
OS: | Unspecified | ||||||
Whiteboard: | |||||||
Fixed In Version: | 2012.2.1-7.el6ost | Doc Type: | Bug Fix | ||||
Doc Text: | Story Points: | --- | |||||
Clone Of: | Environment: | ||||||
Last Closed: | 2013-03-21 19:03:20 UTC | Type: | Bug | ||||
Regression: | --- | Mount Type: | --- | ||||
Documentation: | --- | CRM: | |||||
Verified Versions: | Category: | --- | |||||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
Cloudforms Team: | --- | Target Upstream Version: | |||||
Embargoed: | |||||||
Attachments: |
|
Description
Etsuji Nakai
2012-11-18 06:20:23 UTC
My guess about the reason of this problem is, in the iptables's nat chain, before packets to be SNAT-ed reach the "quantum-l3-agent-snat" chain, it is ACCEPT-ed at the "quantum-l3-agent-POSTROUTING"'s following rule (if net.bridge.bridge-nf-call-iptables=1). Chain quantum-l3-agent-POSTROUTING (1 references) pkts bytes target prot opt in out source destination 10 600 ACCEPT all -- !qg-18ac7fca-0a !qg-18ac7fca-0a 0.0.0.0/0 0.0.0.0/0 ! ctstate DNAT The relevant parts of the nat tables is as below. -------------- Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination 10 600 quantum-l3-agent-POSTROUTING all -- * * 0.0.0.0/0 0.0.0.0/0 0 0 quantum-postrouting-bottom all -- * * 0.0.0.0/0 0.0.0.0/0 0 0 nova-compute-POSTROUTING all -- * * 0.0.0.0/0 0.0.0.0/0 0 0 nova-api-POSTROUTING all -- * * 0.0.0.0/0 0.0.0.0/0 0 0 nova-postrouting-bottom all -- * * 0.0.0.0/0 0.0.0.0/0 Chain quantum-l3-agent-POSTROUTING (1 references) pkts bytes target prot opt in out source destination 10 600 ACCEPT all -- !qg-18ac7fca-0a !qg-18ac7fca-0a 0.0.0.0/0 0.0.0.0/0 ! ctstate DNAT Chain quantum-postrouting-bottom (1 references) pkts bytes target prot opt in out source destination 0 0 quantum-l3-agent-snat all -- * * 0.0.0.0/0 0.0.0.0/0 Chain quantum-l3-agent-snat (1 references) pkts bytes target prot opt in out source destination 0 0 quantum-l3-agent-float-snat all -- * * 0.0.0.0/0 0.0.0.0/0 0 0 SNAT all -- * * 192.168.101.0/24 0.0.0.0/0 to:10.64.201.1 ----------- This is set in default sysctl.conf # grep bridge-nf /etc/sysctl.conf net.bridge.bridge-nf-call-ip6tables = 0 net.bridge.bridge-nf-call-iptables = 0 net.bridge.bridge-nf-call-arptables = 0 # rpm -qf /etc/sysctl.conf initscripts-9.03.31-2.el6.x86_64 So this shouldn't be an issue on default RHEL install, how do you install your machine? *** Bug 877703 has been marked as a duplicate of this bug. *** I'm using the default RHEL6.3 installation. However, at the boot time of RHEL6.3, because the bridge kernel module is not loaded, net.bridge.bridge-nf-call-* in sysctl.conf is simply ignored as below. # sysctl -p error: "net.bridge.bridge-nf-call-ip6tables" is an unknown key error: "net.bridge.bridge-nf-call-iptables" is an unknown key error: "net.bridge.bridge-nf-call-arptables" is an unknown key I suspect that once Quantum L2 agent has started, it loads the bridge module and set net.bridge.bridge-nf-call-* = 1. I'm not sure why the agent requires them to be 1. Or probably, when the bridge module is loaded by the L2 agent, net.bridge.bridge-nf-call-* becomes 1 by default. If so, the L2 agent should explicitly set net.bridge.bridge-nf-call-* = 0 at that timing. I confirmed my guess by putting the following block in /etc/rc.local. (This is called after sysctl.conf is applied, and befer L2 agent starts.) ------ modprobe bridge sysctl -w net.bridge.bridge-nf-call-iptables=0 sysctl -w net.bridge.bridge-nf-call-ip6tables=0 sysctl -w net.bridge.bridge-nf-call-arptables=0 ------ And the result is: # sysctl -a | grep bridge net.bridge.bridge-nf-call-arptables = 0 net.bridge.bridge-nf-call-iptables = 0 net.bridge.bridge-nf-call-ip6tables = 0 net.bridge.bridge-nf-filter-vlan-tagged = 0 net.bridge.bridge-nf-filter-pppoe-tagged = 0 So the problem is that net.bridge.bridge-nf-call-* in sysctl.conf is ignored if the "brdige" module is not loaded at boot time. > So the problem is that net.bridge.bridge-nf-call-* in sysctl.conf is ignored if the "brdige" module is not loaded at boot time.
Indeed and I see the same on F17 too.
Packaging workaround in openstack-quantum-linuxbridge could be to include /etc/sysconfig/modules/openstack-quantum-linuxbridge.modules script which modprobes bridge.
I see. The same workaround worked well for me, thanks. # cat /etc/sysconfig/modules/openstack-quantum-linuxbridge.modules #!/bin/sh modprobe -b bridge >/dev/null 2>&1 exit 0 why not add a "/sbin/sysctl -p" to the end of init script for quantum? or even better quantum should be setting theses options based on need along with others. By the way, I confirmed that this affects only when netns support is disabled. When netns support is enabled, nat table is placed under a different namespace which doesn't contain the bridge interface inside it. So the value of net.bridge.bridge-nf-call-iptables doesn't affect the rule matching. The bottom line is the requirement for net.bridge.bridge-nf-call-iptables could be different based on the configuration to be used by the customer. So hardcoding the parameter in Quantum's source code may be a little bit risky. I think the best way to deal with it is: - Use the package workaround of /etc/sysconfig/modules/openstack-quantum-linuxbridge.modules. - Describe in the Deployment Guide what parameters in /etc/sysctl.conf should be set according to the intended configuration. Created attachment 691296 [details]
Add sysctl modules script for linuxbridge
apevec: does this patch look like a good start?
I have tested floating ip on quantum with linux bridge , it works also setting is as expected using RHEL 6.4 [root@puma34 ~(keystone_admin_tenant1)]$ lsmod | grep bridge bridge 79078 0 stp 2218 2 garp,bridge llc 5546 3 garp,bridge,stp [root@puma34 ~(keystone_admin_tenant1)]$ sysctl -a | grep bridge net.bridge.bridge-nf-call-arptables = 0 net.bridge.bridge-nf-call-iptables = 0 net.bridge.bridge-nf-call-ip6tables = 0 net.bridge.bridge-nf-filter-vlan-tagged = 0 net.bridge.bridge-nf-filter-pppoe-tagged = 0 [root@puma34 ~(keystone_admin_tenant1)]$ rpm -qa | grep quantum openstack-quantum-2012.2.3-8.el6ost.noarch Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHBA-2013-0672.html |