Bug 156628 - Router based on FC3 do not forward tcp packets with SACK set
Router based on FC3 do not forward tcp packets with SACK set
Status: CLOSED CANTFIX
Product: Fedora
Classification: Fedora
Component: kernel (Show other bugs)
3
i586 Linux
medium Severity high
: ---
: ---
Assigned To: David Miller
Brian Brock
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2005-05-02 14:13 EDT by Peter
Modified: 2007-11-30 17:11 EST (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2005-10-02 21:06:41 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Peter 2005-05-02 14:13:58 EDT
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.7) Gecko/20050414 Firefox/1.0.3

Description of problem:
Small LAN behind FC3 firewall/NAT (two NICs). SSH (CVS over ssh) connection from LAN to inet are unexpectedly closing. About 20-50% of sessions dropped (due to network load).
tcpdump on both firewall interfaces showing a small differences between traffic on LAN and inet interfaces. 

Internal NIC:
18:31:19.755341 IP 192.168.0.200.4825 > $SSH_SERVER.ssh: S 358490661:358490661(0) win 5840 <mss 1460,sackOK,timestamp 563771714 0,nop,wscale 0>
18:31:19.910782 IP $SSH_SERVER.ssh > 192.168.0.200.4825: S 2833214570:2833214570(0) ack 358490662 win 5792 <mss 1460,sackOK,timestamp 31927719 563771714,nop,wscale 0>
18:31:19.910911 IP 192.168.0.200.4825 > $SSH_SERVER.ssh: . ack 1 win 5840 <nop,nop,timestamp 563771730 31927719>
18:31:20.203011 IP $SSH_SERVER.ssh > 192.168.0.200.4825: P 1:24(23) ack 1 win 5792 <nop,nop,timestamp 31927748 563771730>
18:31:20.203149 IP 192.168.0.200.4825 > $SSH_SERVER.ssh: . ack 24 win 5840 <nop,nop,timestamp 563771759 31927748>
18:31:20.203324 IP 192.168.0.200.4825 > $SSH_SERVER.ssh: P 1:23(22) ack 24 win 5840 <nop,nop,timestamp 563771759 31927748>
18:31:20.360973 IP $SSH_SERVER.ssh > 192.168.0.200.4825: . ack 23 win 5792 <nop,nop,timestamp 31927764 563771759>
18:31:20.365652 IP $SSH_SERVER.ssh > 192.168.0.200.4825: P 24:300(276) ack 23 win 5792 <nop,nop,timestamp 31927764 563771759>
18:31:20.366643 IP 192.168.0.200.4825 > $SSH_SERVER.ssh: P 23:179(156) ack 300 win 6432 <nop,nop,timestamp 563771775 31927764>
18:31:20.837225 IP 192.168.0.200.4825 > $SSH_SERVER.ssh: P 23:179(156) ack 300 win 6432 <nop,nop,timestamp 563771823 31927764>
18:31:21.012613 IP $SSH_SERVER.ssh > 192.168.0.200.4825: P 300:312(12) ack 179 win 5792 <nop,nop,timestamp 31927829 563771823>
18:31:21.012880 IP 192.168.0.200.4825 > $SSH_SERVER.ssh: P 179:207(28) ack 312 win 6432 <nop,nop,timestamp 563771840 31927829>
18:31:21.168937 IP $SSH_SERVER.ssh > 192.168.0.200.4825: R 2833214882:2833214882(0) win 0

External NIC:
18:31:19.755520 IP $EXTERNAL_IP.4825 > $SSH_SERVER.ssh: S 358490661:358490661(0) win 5840 <mss 1460,sackOK,timestamp 563771714 0,nop,wscale 0>
18:31:19.910662 IP $SSH_SERVER.ssh > $EXTERNAL_IP.4825: S 2833214570:2833214570(0) ack 358490662 win 5792 <mss 1460,sackOK,timestamp 31927719 563771714,nop,wscale 0>
18:31:19.911034 IP $EXTERNAL_IP.4825 > $SSH_SERVER.ssh: . ack 1 win 5840 <nop,nop,timestamp 563771730 31927719>
18:31:20.202914 IP $SSH_SERVER.ssh > $EXTERNAL_IP.4825: P 1:24(23) ack 1 win 5792 <nop,nop,timestamp 31927748 563771730>
18:31:20.203267 IP $EXTERNAL_IP.4825 > $SSH_SERVER.ssh: . ack 24 win 5840 <nop,nop,timestamp 563771759 31927748>
18:31:20.203415 IP $EXTERNAL_IP.4825 > $SSH_SERVER.ssh: P 1:23(22) ack 24 win 5840 <nop,nop,timestamp 563771759 31927748>
18:31:20.360863 IP $SSH_SERVER.ssh > $EXTERNAL_IP.4825: . ack 23 win 5792 <nop,nop,timestamp 31927764 563771759>
18:31:20.365500 IP $SSH_SERVER.ssh > $EXTERNAL_IP.4825: P 24:300(276) ack 23 win 5792 <nop,nop,timestamp 31927764 563771759>
18:31:20.366798 IP $EXTERNAL_IP.4825 > $SSH_SERVER.ssh: P 23:179(156) ack 300 win 6432 <nop,nop,timestamp 563771775 31927764>
18:31:20.837359 IP $EXTERNAL_IP.4825 > $SSH_SERVER.ssh: P 23:179(156) ack 300 win 6432 <nop,nop,timestamp 563771823 31927764>
18:31:21.011792 IP $SSH_SERVER.ssh > $EXTERNAL_IP.4825: . ack 179 win 5792 <nop,nop,timestamp 31927828 563771823,nop,nop,sack sack 1 {1880160731:1880160887} >
18:31:21.012130 IP $EXTERNAL_IP.4825 > $SSH_SERVER.ssh: R 358490840:358490840(0) win 0
18:31:21.012527 IP $SSH_SERVER.ssh > $EXTERNAL_IP.4825: P 300:312(12) ack 179 win 5792 <nop,nop,timestamp 31927829 563771823>
18:31:21.013001 IP $EXTERNAL_IP.4825 > $SSH_SERVER.ssh: P 179:207(28) ack 312 win 6432 <nop,nop,timestamp 563771840 31927829>
18:31:21.168823 IP $SSH_SERVER.ssh > $EXTERNAL_IP.4825: R 2833214882:2833214882(0) win 0

Version-Release number of selected component (if applicable):
kernel-2.6.11-1.14_FC3

How reproducible:
Always

Steps to Reproduce:
1. FC3 default server install
2. Setup ip_forwarding and SNAT
3. Establish a tcp session from LAN with SACK OK.
  

Actual Results:  On SACK packet router send back RST and do not forward the packet.

Expected Results:  Router should forward "sack sack 1" packet to client.

Additional info:

Completely solved by disabling SACK either on ssh client or on ssh server.
echo "0" > /proc/sys/net/ipv4/tcp_sack

But how to make FC3 router forward tcp packets with SACK set?
Comment 1 David Miller 2005-05-02 15:42:43 EDT
The SACK blocks get stripped by netfilter.  What netfilter modules
exactly do you have loaded?

The SACK blocks get stripped so that netfilter does not have to
recompute the sequence numbers inside of them when it munges
the packets, f.e. for doing FTP NAT.

But that should not be relevant here.  I bet it's some bug in
TCP connection tracking.
Comment 2 Peter 2005-05-03 01:59:20 EDT
# lsmod
Module                  Size  Used by
iptable_filter          2881  0
cls_u32                 8517  2
sch_sfq                 5825  8
sch_htb                19137  2
iptable_mangle          2753  0
iptable_nat            22301  1
ip_conntrack           41369  1 iptable_nat
ip_tables              20417  3 iptable_filter,iptable_mangle,iptable_nat
ip_gre                 13153  0
md5                     4289  1
ipv6                  258689  10
tun                    11457  1
uhci_hcd               32857  0
e100                   44993  0
pcnet32                33733  0
8139too                28609  0
mii                     5057  3 e100,pcnet32,8139too
floppy                 63345  0
dm_snapshot            17925  0
dm_zero                 2497  0
dm_mirror              24877  0
ext3                  130761  3
jbd                    76889  1 ext3
dm_mod                 59989  7 dm_snapshot,dm_zero,dm_mirror
Comment 3 Peter 2005-05-03 02:28:57 EDT
BTW
That SACK (sack sack 1) packet was actually sent by SSH_SERVER as "sack sack 1
{23:179}" but received as "sack sack 1 {1880160731:1880160887}"

Is it OK? Or packet was corrupted somewhere between $EXTERNAL_IP and $SSH_SERVER.

SSH_SERVER dump
11:29:24.788445 IP $EXTERNAL_IP.4825 > $SSH_SERVER.ssh: S
2238651369:2238651369(0) win 5840 <mss 1460,sackOK,timestamp 563771714
0,nop,wscale 0>
11:29:24.788514 IP $SSH_SERVER.ssh > $EXTERNAL_IP.4825: S
1629206029:1629206029(0) ack 2238651370 win 5792 <mss 1460,sackOK,timestamp
31927719 563771714,nop,wscale 0>
11:29:24.944168 IP $EXTERNAL_IP.4825 > $SSH_SERVER.ssh: . ack 1 win 5840
<nop,nop,timestamp 563771730 31927719>
11:29:25.075030 IP $SSH_SERVER.ssh > $EXTERNAL_IP.4825: P 1:24(23) ack 1 win
5792 <nop,nop,timestamp 31927748 563771730>
11:29:25.235772 IP $EXTERNAL_IP.4825 > $SSH_SERVER.ssh: . ack 24 win 5840
<nop,nop,timestamp 563771759 31927748>
11:29:25.237590 IP $EXTERNAL_IP.4825 > $SSH_SERVER.ssh: P 1:23(22) ack 24 win
5840 <nop,nop,timestamp 563771759 31927748>
11:29:25.237660 IP $SSH_SERVER.ssh > $EXTERNAL_IP.4825: . ack 23 win 5792
<nop,nop,timestamp 31927764 563771759>
11:29:25.238419 IP $SSH_SERVER.ssh > $EXTERNAL_IP.4825: P 24:300(276) ack 23 win
5792 <nop,nop,timestamp 31927764 563771759>
11:29:25.402294 IP $EXTERNAL_IP.4825 > $SSH_SERVER.ssh: P 23:179(156) ack 300
win 6432 <nop,nop,timestamp 563771775 31927764>
11:29:25.429814 IP $SSH_SERVER.ssh > $EXTERNAL_IP.4825: P 300:312(12) ack 179
win 5792 <nop,nop,timestamp 31927784 563771775>
11:29:25.871950 IP $EXTERNAL_IP.4825 > $SSH_SERVER.ssh: P 23:179(156) ack 300
win 6432 <nop,nop,timestamp 563771823 31927764>
11:29:25.871997 IP $SSH_SERVER.ssh > $EXTERNAL_IP.4825: . ack 179 win 5792
<nop,nop,timestamp 31927828 563771823,nop,nop,sack sack 1 {23:179} >
11:29:25.879466 IP $SSH_SERVER.ssh > $EXTERNAL_IP.4825: P 300:312(12) ack 179
win 5792 <nop,nop,timestamp 31927829 563771823>
11:29:26.044612 IP $EXTERNAL_IP.4825 > $SSH_SERVER.ssh: R
2238651548:2238651548(0) win 0
11:29:26.046737 IP $EXTERNAL_IP.4825 > $SSH_SERVER.ssh: P 179:207(28) ack 312
win 6432 <nop,nop,timestamp 563771840 31927829>
11:29:26.046790 IP $SSH_SERVER.ssh > $EXTERNAL_IP.4825: R
1629206341:1629206341(0) win 0
Comment 4 Dave Jones 2005-07-15 14:51:16 EDT
An update has been released for Fedora Core 3 (kernel-2.6.12-1.1372_FC3) which
may contain a fix for your problem.   Please update to this new kernel, and
report whether or not it fixes your problem.

If you have updated to Fedora Core 4 since this bug was opened, and the problem
still occurs with the latest updates for that release, please change the version
field of this bug to 'fc4'.

Thank you.
Comment 5 Dave Jones 2005-10-02 21:06:41 EDT
This bug has been automatically closed as part of a mass update.
It had been in NEEDINFO state since July 2005.
If this bug still exists in current errata kernels, please reopen this bug.

There are a large number of inactive bugs in the database, and this is the only
way to purge them.

Thank you.

Note You need to log in before you can comment on or make changes to this bug.