Bug 161792 - iptables conntrack breaks when Xen creates bridge
iptables conntrack breaks when Xen creates bridge
Status: CLOSED CURRENTRELEASE
Product: Fedora
Classification: Fedora
Component: kernel (Show other bugs)
4
i686 Linux
medium Severity high
: ---
: ---
Assigned To: Dave Jones
Brian Brock
http://lists.xensource.com/archives/h...
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2005-06-27 09:39 EDT by Paul Jakma
Modified: 2015-01-04 17:20 EST (History)
5 users (show)

See Also:
Fixed In Version: 2.6.12-1.1447_FC4xen0
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2005-10-06 11:20:21 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Paul Jakma 2005-06-27 09:39:49 EDT
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.7.8) Gecko/20050513 Galeon/1.3.21

Description of problem:
Outbound locally initiated connections appear to cease functioning when Xen
is used. It appears to be because conntrack fails to enter locally initiated
connections which go out via the bridge device into its state table.

Here's a tcpdump of xen-br0 while a connection to a remote host is initiated locally:

# tcpdump -i xen-br0 host remote-host
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on xen-br0, link-type EN10MB (Ethernet), capture size 96 bytes
15:24:38.732971 IP local-dom0.39517 > remote-host.smtp: S
3759604004:3759604004(0) win 5840 <mss 1460,sackOK,timestamp 205822273
0,nop,wscale 2>
15:24:38.865317 IP remote-host.smtp > local-dom0.39517: S
3168259509:3168259509(0) ack 3759604005 win 5792 <mss 1460,sackOK,timestamp
1357649216 205822273,nop,wscale 0>
15:24:38.865353 IP local-dom0 > remote-host: icmp 68: host
local-dom0 unreachable - admin prohibited
15:24:41.732419 IP local-dom0.39517 > remote-host.smtp: S
3759604004:3759604004(0) win 5840 <mss 1460,sackOK,timestamp 205822573
0,nop,wscale 2>
15:24:41.865057 IP remote-host.smtp > local-dom0.39517: S
3168259509:3168259509(0) ack 3759604005 win 5792 <mss 1460,sackOK,timestamp
1357649516 205822273,nop,wscale 0>
15:24:41.865089 IP local-dom0 > remote-host: icmp 68: host
local-dom0 unreachable - admin prohibited
15:24:47.732251 IP local-dom0.39517 > remote-host.smtp: S
3759604004:3759604004(0) win 5840 <mss 1460,sackOK,timestamp 205823173
0,nop,wscale 2>
15:24:47.864284 IP remote-host.smtp > local-dom0.39517: S
3177258495:3177258495(0) ack 3759604005 win 5792 <mss 1460,sackOK,timestamp
1357650116 205823173,nop,wscale 0>
15:24:47.864312 IP local-dom0 > remote-host: icmp 68: host
local-dom0 unreachable - admin prohibited
15:24:59.731919 IP local-dom0.39517 > remote-host.smtp: S
3759604004:3759604004(0) win 5840 <mss 1460,sackOK,timestamp 205824373
0,nop,wscale 2>
15:24:59.863863 IP remote-host.smtp > local-dom0.39517: S
3189258249:3189258249(0) ack 3759604005 win 5792 <mss 1460,sackOK,timestamp
1357651316 205824373,nop,wscale 0>
15:24:59.863896 IP local-dom0 > remote-host: icmp 68: host
local-dom0 unreachable - admin prohibited

12 packets captured
12 packets received by filter
0 packets dropped by kernel
# grep <ip of remote host> /proc/net/ip_conntrack
#

The firewalling rules were configured with the system-config-securitylevel-tui tool included with FC4. Here are the rules:

# iptables -L -v
Chain INPUT (policy ACCEPT 223K packets, 19M bytes)
 pkts bytes target     prot opt in     out     source
destination         
 121K 9494K RH-Firewall-1-INPUT  all  --  xen-br+ any     anywhere
anywhere            
    0     0 RH-Firewall-1-INPUT  all  --  vif+   any     anywhere
anywhere            
    1    73 RH-Firewall-1-INPUT  all  --  eth0   any     anywhere
anywhere          

Chain FORWARD (policy ACCEPT 4352K packets, 385M bytes)
 pkts bytes target     prot opt in     out     source
destination         

Chain OUTPUT (policy ACCEPT 357K packets, 36M bytes)
 pkts bytes target     prot opt in     out     source
destination         

Chain RH-Firewall-1-INPUT (3 references)
 pkts bytes target     prot opt in     out     source
destination         
   33  2485 ACCEPT     all  --  lo     any     anywhere             anywhere            
     264 17502 ACCEPT     icmp --  any    any     anywhere
anywhere            icmp any 
    0     0 ACCEPT     ipv6-crypt--  any    any     anywhere
anywhere            
    0     0 ACCEPT     ipv6-auth--  any    any     anywhere
anywhere            
70878 6221K ACCEPT     all  --  any    any     anywhere             anywhere
state RELATED,ESTABLISHED 
12798 1260K ACCEPT     udp  --  any    any     anywhere             anywhere
state NEW udp dpt:domain 
    0     0 ACCEPT     tcp  --  any    any     anywhere             anywhere
state NEW tcp dpt:domain 
    0     0 ACCEPT     tcp  --  any    any     anywhere             anywhere
state NEW tcp dpt:ldaps 
    0     0 ACCEPT     udp  --  any    any     anywhere             anywhere
state NEW udp dpt:kerberos 
    0     0 ACCEPT     tcp  --  any    any     anywhere             anywhere
state NEW tcp dpt:kerberos 
    0     0 ACCEPT     tcp  --  any    any     anywhere             anywhere
state NEW tcp dpt:kerberos-adm 
 4081  245K ACCEPT     tcp  --  any    any     anywhere             anywhere
state NEW tcp dpt:ssh 
    8   432 ACCEPT     tcp  --  any    any     anywhere             anywhere
state NEW tcp dpt:https 
33072 1766K REJECT     all  --  any    any     anywhere             anywhere
reject-with icmp-host-prohibited 

The problem appears to be due to conntrack failing to register  the initial outbound connection on xen-br0 and hence the RELATED,ESTABLISHED rule failing to match the remote replies.


Version-Release number of selected component (if applicable):
2.6.11-1.1366_FC4xen0

How reproducible:
Always

Steps to Reproduce:
1. Setup firewalling with system-config-securitylevel-tui
2. Setup a XenU host, using bridging, as per default
3. Try initiate outbound connection on dom0 to a remote host. Observe that,
   local DNS fails too (per default FC4 configuration).
  

Actual Results:  Connection is never established and times out.


Expected Results:  Connection should have established. The remote reply should have been matched
by conntrack.


Additional info:

This is quite critical. FC4 Xen can not be used with firewalling + bridging, which
happens to be the default configuration in FC4. For most people, FC4 Xen will be
unuseable.
Comment 1 Michael Paesold 2005-06-27 11:12:06 EDT
I can confirm this problem here. I have disabled firewalling in dom0 since using 
xen. I am asking myself if this is a problem with the source snapshot used for 
the FC4 package and would be fixed in newer revisions in the upstream source 
tree.
Comment 2 jonny.fahrenheit451 2005-06-27 13:15:21 EDT
Hi Paul, I'll add my info and delete my bug filing.
Confirmed here, this time from point of view of ssh client to dom0 host.
Problem:-
New install of fedora core 4 with xen kernel running. Iptables rules that under
the regular kernel work fine stop working when in bridge mode under xen in dom0.
This stops the conntrack system working on the xen host machine and i can't then
log in via ssh.
It seems that the conntrack system is failing to match already accepted
connections. The initial packet seems to get accepted by the INPUT rule, then
the reply packet slips past the ESTABLISHED,RELATED rule and gets logged then
dropped by the default OUTPUT policy.
The ip_conntrack, state modules and all the other modules are loaded.

This is the packet that gets logged:-
xen kernel: OUTPUT IN= OUT=xen-br0 PHYSOUT=eth0 SRC=192.168.0.45
DST=192.168.0.39 LEN=48 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF PROTO=TCP SPT=22
DPT=1152 WINDOW=5840 RES=0x00 ACK SYN URGP=0

This happens whether i start a guest os up or not.
This was reproduced on another machine at work with a Fedora Core 4 install.

xen host machine address:192.168.0.45
ssh client address:192.168.0.39

rules:-
Chain FORWARD (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0  
        state RELATED,ESTABLISHED
    0     0 LOG        all  --  *      *       0.0.0.0/0            0.0.0.0/0  
        LOG flags 0 level 4 prefix `FORWARD '

Chain INPUT (policy DROP 54 packets, 7483 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     all  --  lo     *       0.0.0.0/0            0.0.0.0/0
  304 21532 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0  
        state RELATED,ESTABLISHED
    1    48 ACCEPT     tcp  --  *      *       192.168.0.39         192.168.0.45
      tcp spts:1024:65535 dpt:22 state NEW
   54  7483 LOG        all  --  *      *       0.0.0.0/0            0.0.0.0/0  
        LOG flags 0 level 4 prefix `INPUT '

Chain OUTPUT (policy DROP 8 packets, 384 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     all  --  *      lo      0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0  
        state RELATED,ESTABLISHED
    0     0 ACCEPT     udp  --  *      *       192.168.0.45         192.168.0.19
      udp spts:1024:65535 dpt:53
    0     0 ACCEPT     tcp  --  *      *       192.168.0.45         0.0.0.0/0  
        tcp spts:1024:65535 dpt:80
    0     0 ACCEPT     icmp --  *      *       192.168.0.45         0.0.0.0/0
    0     0 LOG        all  --  *      *       0.0.0.0/0            0.0.0.0/0  
        LOG flags 0 level 4 prefix `OUTPUT '

interfaces:-
eth0      Link encap:Ethernet  HWaddr 00:08:74:EE:50:ED
          inet addr:192.168.0.45  Bcast:192.168.0.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:24684 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4406 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:1992235 (1.8 MiB)  TX bytes:631910 (617.0 KiB)
          Base address:0xecc0 Memory:ff8e0000-ff900000

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

xen-br0   Link encap:Ethernet  HWaddr 00:08:74:EE:50:ED
          inet addr:192.168.0.45  Bcast:192.168.0.255  Mask:255.255.255.255
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:24682 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4451 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:1538495 (1.4 MiB)  TX bytes:618890 (604.3 KiB)

routes:-
192.168.0.0     0.0.0.0         255.255.255.0   U     0      0        0 xen-br0
169.254.0.0     0.0.0.0         255.255.0.0     U     0      0        0 xen-br0
0.0.0.0         192.168.0.250   0.0.0.0         UG    0      0        0 xen-br0

operating system:-
Fedora Core 4

kernel version:-
2.6.11-1.1369_FC4xen0

iptables version:-
iptables v1.3.0

xen version:-
xen-2-20050522

network driver:-
e1000

Had everything working under fedora core 3 before with iptables and 5 virtual
machines with conntrack behaving as it should.
Comment 3 Dave Jones 2005-07-15 17:09:55 EDT
[This comment has been added as a mass update for all FC4 kernel bugs.
 If you have migrated this bug from an FC3 bug today, ignore this comment.]

Please retest your problem with todays 2.6.12-1.1398_FC4 update.

If your problem involved being unable to boot, or some hardware not being
detected correctly, please make sure your /etc/modprobe.conf is correct *BEFORE*
installing any kernel updates.
If in doubt, you can recreate this file using..

mv /etc/sysconfig/hwconf /etc/sysconfig/hwconf.bak
mv /etc/modprobe.conf /etc/modprobe.conf.bak
kudzu


Thank you.
Comment 4 SteveB 2005-07-17 00:49:31 EDT
2.6.12-1.1398_FC4 fixed this problem for me.
Netfilter no longer flags packets coming in on the bridged 
interface as  invalid:
IPTABLES -A INPUT -m state --state INVALID -j DROP

Thanks,
Steve
Comment 5 Jon Dowland 2005-07-26 12:02:02 EDT
Problem appears to have resolved for me with 2.6.12-1.1398_FC4 also. Cheers!
Comment 6 Paul Jakma 2005-08-09 14:32:29 EDT
This problem still occurs for me with 2.6.12-1.1390_FC4xen0, with Xen bridging
setup as per the stock FC4 setup scripts, with the firewalling rules as
configured by system-config-securitylevel:

Chain RH-Firewall-1-INPUT (3 references)
target     prot opt source               destination         
ACCEPT     all  --  anywhere             anywhere            
ACCEPT     icmp --  anywhere             anywhere            icmp any 
ACCEPT     ipv6-crypt--  anywhere             anywhere            
ACCEPT     ipv6-auth--  anywhere             anywhere            
ACCEPT     all  --  anywhere             anywhere            state
RELATED,ESTABLISHED 
ACCEPT     udp  --  anywhere             anywhere            state NEW udp
dpt:domain 
ACCEPT     tcp  --  anywhere             anywhere            state NEW tcp
dpt:domain 
ACCEPT     tcp  --  anywhere             anywhere            state NEW tcp
dpt:ldaps 
ACCEPT     udp  --  anywhere             anywhere            state NEW udp
dpt:kerberos 
ACCEPT     tcp  --  anywhere             anywhere            state NEW tcp
dpt:kerberos 
ACCEPT     tcp  --  anywhere             anywhere            state NEW tcp
dpt:kerberos-adm 
ACCEPT     tcp  --  anywhere             anywhere            state NEW tcp dpt:ssh 
ACCEPT     tcp  --  anywhere             anywhere            state NEW tcp
dpt:https 
REJECT     all  --  anywhere             anywhere            reject-with
icmp-host-prohibited 
[root@master sources.list.d]# telnet slashdot.org 80
Trying 66.35.250.150...

[root@master sources.list.d]# uname -a
Linux master.quagga.net 2.6.12-1.1390_FC4xen0 #1 SMP Tue Jul 5 20:36:07 EDT 2005
i686 i686 i386 GNU/Linux
[root@master sources.list.d]# grep 66.35.250.150 /proc/net//ip_conntrack*
[root@master sources.list.d]# 
Comment 7 Jon Dowland 2005-08-11 10:24:23 EDT
Alas, I was too hasty, the problem remains for me also.

With default RH firewall on the domain 0 machine, I get the following on tcpdump
-i vif2.0 ip (where vif2.0 is the virtual interface on dom0 for the dom1 machine):

15:24:56.852201 IP domain0-machine > domain1-machine: icmp 67: host dns-machine
unreachable - admin prohibited

After 'service iptables stop'

15:25:43.229595 IP domain1-machine.32789 > dns-machine.domain:  27274+ A?
halfcoded.net. (31)
15:25:43.230193 IP dns-machine.domain > domain1-machine.32789:  27274 1/3/2 A
alcopop.org (151)

domain0 kernel: 2.6.12-1.1398_FC4xen0
domain1 kernel: 2.6.12-1.1398_FC4xenU 

iptables script:

# iptables-save
# Generated by iptables-save v1.3.0 on Thu Aug 11 15:26:57 2005
*filter
:FORWARD ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [36:3864]
:RH-Firewall-1-INPUT - [0:0]
-A FORWARD -j RH-Firewall-1-INPUT
-A INPUT -j RH-Firewall-1-INPUT
-A RH-Firewall-1-INPUT -i lo -j ACCEPT
-A RH-Firewall-1-INPUT -p icmp -m icmp --icmp-type any -j ACCEPT
-A RH-Firewall-1-INPUT -p ipv6-crypt -j ACCEPT
-A RH-Firewall-1-INPUT -p ipv6-auth -j ACCEPT
-A RH-Firewall-1-INPUT -d 224.0.0.251 -p udp -m udp --dport 5353 -j ACCEPT
-A RH-Firewall-1-INPUT -p udp -m udp --dport 631 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
-A RH-Firewall-1-INPUT -j REJECT --reject-with icmp-host-prohibited
COMMIT
# Completed on Thu Aug 11 15:26:57 2005
Comment 8 Jon Dowland 2005-08-11 10:25:43 EDT
Erm yes, ok. To clarify the previous note, all references to dom1 should be dom2
instead (hence, vif2.0). If I rebooted the host and started the guest domain
just once, it would be dom1 and vif1.0. Sorry for any confusion.
Comment 9 Paul Jakma 2005-09-13 11:10:15 EDT
This problem appears, so far, to be fixed in 2.6.12-1.1447_FC4xen0
Comment 10 Dave Jones 2005-09-30 02:15:59 EDT
Mass update to all FC4 bugs:

An update has been released (2.6.13-1.1526_FC4) which rebases to a new upstream
kernel (2.6.13.2). As there were ~3500 changes upstream between this and the
previous kernel, it's possible your bug has been fixed already.

Please retest with this update, and update this bug if necessary.

Thanks.
Comment 11 Paul Jakma 2005-10-06 11:20:21 EDT
It's fixed, since 2.6.12-1.1447_FC4xen0.
Comment 12 Jon Dowland 2005-10-10 10:02:51 EDT
Problem remains for me with 2.6.12-1.1456_FC4xen0

Note You need to log in before you can comment on or make changes to this bug.