Bug 887652 - iptables + bridge: NAT does not work
Summary: iptables + bridge: NAT does not work
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Fedora
Classification: Fedora
Component: kernel
Version: 17
Hardware: x86_64
OS: Linux
unspecified
low
Target Milestone: ---
Assignee: Kernel Maintainer List
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2012-12-16 22:42 UTC by Rolf Fokkens
Modified: 2013-05-01 18:12 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2013-05-01 18:12:57 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Rolf Fokkens 2012-12-16 22:42:31 UTC
Description of problem:
I'm trying to use a single NIC NAT configuration; this allows me to implement TC without changing the network topology (except for the default gateway). This apparently only works when tcpdump is attached to the interface, without tcpdump the returned packets (reverse NAT) apparently get lost.


Version-Release number of selected component (if applicable):
3.6.10-2.fc17.x86_64

How reproducible:
100% on my system. Haven't tried other configs.

Steps to Reproduce:
1. Build a bridge interface br0 (might be relevant) on an ethernet interface eth0.3 (VLAN 3).
2. Give br0 1 private IP address (IP1) and a public IP address (IP2)
3. Make sure that this routing device has a default route using IP2 to the internet
4. Apply this POSTROUTING NAT rule: SNAT all  * br0 0.0.0.0/0 0.0.0.0/0 to:<IP2>
5. Configure a user-device in the <IP1> network with address <IPU> and a default route to <IP1>.
6. Configure a mirror/span port to sniff the network packets on eth0
7. Now try to access the internet from the user-device
8. Repeat this with tcpdump running on eth0
  
Actual results:
The user-device has no useful internet connection. tcpdump on the mirror port shows:

19:31:56.853553 18:34:51:0b:5b:df > 00:16:3e:01:02:01, ethertype 802.1Q (0x8100), length 82: vlan 3, p 0, ethertype IPv4, 1<IPU>.57195 > 81.18.1.2.http: Flags [S], seq 3102102224, win 65535, options [mss 1460,nop,wscale 4,nop,nop,TS val 405697868 ecr 0,sackOK,eol], length 0
19:31:56.853968 00:16:3e:01:02:01 > 00:26:44:02:0b:7d, ethertype 802.1Q (0x8100), length 82: vlan 3, p 0, ethertype IPv4, <IP2>.57195 > 81.18.1.2.http: Flags [S], seq 3102102224, win 65535, options [mss 1460,nop,wscale 4,nop,nop,TS val 405697868 ecr 0,sackOK,eol], length 0
19:31:56.870809 00:26:44:02:0b:7d > 00:16:3e:01:02:01, ethertype 802.1Q (0x8100), length 78: vlan 3, p 0, ethertype IPv4, 81.18.1.2.http > <IP2>.57195: Flags [S.], seq 3405796337, ack 3102102225, win 5792, options [mss 1460,sackOK,TS val 1582061421 ecr 405697868,nop,wscale 7], length 0
19:31:58.880061 18:34:51:0b:5b:df > 00:16:3e:01:02:01, ethertype 802.1Q (0x8100), length 82: vlan 3, p 0, ethertype IPv4, <IPU>.57195 > 81.18.1.2.http: Flags [S], seq 3102102224, win 65535, options [mss 1460,nop,wscale 4,nop,nop,TS val 405699876 ecr 0,sackOK,eol], length 0
19:31:58.880447 00:16:3e:01:02:01 > 00:26:44:02:0b:7d, ethertype 802.1Q (0x8100), length 82: vlan 3, p 0, ethertype IPv4, <IP2>.57195 > 81.18.1.2.http: Flags [S], seq 3102102224, win 65535, options [mss 1460,nop,wscale 4,nop,nop,TS val 405699876 ecr 0,sackOK,eol], length 0
19:31:58.896174 00:26:44:02:0b:7d > 00:16:3e:01:02:01, ethertype 802.1Q (0x8100), length 78: vlan 3, p 0, ethertype IPv4, 81.18.1.2.http > <IP2>.57195: Flags [S.], seq 3405796337, ack 3102102225, win 5792, options [mss 1460,sackOK,TS val 1582063447 ecr 405697868,nop,wscale 7], length 0

Note that the reverse NATted packet is never seen.

Expected results:
When running tcpdump on eth0, one gets the expected results.  tcpdump on the mirror port shows:
19:30:54.844079 18:34:51:0b:5b:df > 00:16:3e:01:02:01, ethertype 802.1Q (0x8100), length 82: vlan 3, p 0, ethertype IPv4, <IPU>.57192 > 81.18.1.2.http: Flags [S], seq 3382660252, win 65535, options [mss 1460,nop,wscale 4,nop,nop,TS val 405635976 ecr 0,sackOK,eol], length 0
19:30:54.844372 00:16:3e:01:02:01 > 00:26:44:02:0b:7d, ethertype 802.1Q (0x8100), length 82: vlan 3, p 0, ethertype IPv4, <IP2>.57192 > 81.18.1.2.http: Flags [S], seq 3382660252, win 65535, options [mss 1460,nop,wscale 4,nop,nop,TS val 405635976 ecr 0,sackOK,eol], length 0
19:30:54.861440 00:26:44:02:0b:7d > 00:16:3e:01:02:01, ethertype 802.1Q (0x8100), length 78: vlan 3, p 0, ethertype IPv4, 81.18.1.2.http > <IP2>.57192: Flags [S.], seq 3392746235, ack 3382660253, win 5792, options [mss 1460,sackOK,TS val 1581999403 ecr 405635976,nop,wscale 7], length 0
19:30:54.861689 00:16:3e:01:02:01 > 18:34:51:0b:5b:df, ethertype 802.1Q (0x8100), length 78: vlan 3, p 0, ethertype IPv4, 81.18.1.2.http > <IPU>.57192: Flags [S.], seq 3392746235, ack 3382660253, win 5792, options [mss 1460,sackOK,TS val 1581999403 ecr 405635976,nop,wscale 7], length 0

Additional info:
Actual IP addresses are anonymised, full details can be shared by email when requested.

The usage of VLAN 3 and the bridge are related to local network specifics.

Comment 1 Rolf Fokkens 2013-01-01 01:02:29 UTC
The solution appears to be to add in sysctl.conf:

# Disable netfilter on bridges.
net.bridge.bridge-nf-call-ip6tables = 0
net.bridge.bridge-nf-call-iptables = 0
net.bridge.bridge-nf-call-arptables = 0

So the problem apparently was the combination of NAT and a bridge. Is this a bug?

Comment 2 Neil Horman 2013-05-01 18:12:57 UTC
I don't think so.  It sounds like your rules are causing your application to use IP1 as a source address, but hosts are responding to your IP2 addrss (beacuse of your SNAT rule in POSTROUTING, so when frames come back, theres no listening socket (as its bound to IP1).  By disabling the above sysctls, you've effectively suppressed the iptables translation so everything works (although you're not going to have the translation you want). You might want to consider a DNAT rule to flip inbound addresses back to the origional IP1.


Note You need to log in before you can comment on or make changes to this bug.