Bug 145659 - Socket option SO_BINDTODEVICE problems with SCTP listening socket.
Summary: Socket option SO_BINDTODEVICE problems with SCTP listening socket.
Alias: None
Product: Red Hat Enterprise Linux 4
Classification: Red Hat
Component: kernel   
(Show other bugs)
Version: 4.0
Hardware: All Linux
Target Milestone: ---
: ---
Assignee: Neil Horman
QA Contact: Brian Brock
Depends On:
Blocks: 154907 156323
TreeView+ depends on / blocked
Reported: 2005-01-20 14:42 UTC by Jere Leppanen
Modified: 2007-11-30 22:07 UTC (History)
6 users (show)

Fixed In Version: RHSA-2005-514
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2005-10-05 12:41:50 UTC
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
Patch to discard packets from wrong interface (447 bytes, patch)
2005-05-03 09:28 UTC, Jere Leppanen
no flags Details | Diff
new version of patch to treat packets from unbound interfaces as OOTB (1.51 KB, patch)
2005-05-18 15:06 UTC, Neil Horman
no flags Details | Diff
updated version of BINDTODEVICE/OOTB patch (918 bytes, patch)
2005-05-19 01:41 UTC, Neil Horman
no flags Details | Diff
new version of bindtodevice/ootb patch (951 bytes, patch)
2005-05-19 20:06 UTC, Neil Horman
no flags Details | Diff
again a new version of the ootb/bindtodevice patch (1.83 KB, patch)
2005-05-20 16:06 UTC, Neil Horman
no flags Details | Diff
new approach to handling bindtodevice as ootb (816 bytes, patch)
2005-05-24 20:23 UTC, Neil Horman
no flags Details | Diff
update to previous patch (861 bytes, patch)
2005-05-26 19:40 UTC, Neil Horman
no flags Details | Diff
final version of the patch which has upstream acceptance. (2.27 KB, patch)
2005-06-02 18:54 UTC, Neil Horman
no flags Details | Diff

External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2005:514 qe-ready SHIPPED_LIVE Important: Updated kernel packages available for Red Hat Enterprise Linux 4 Update 2 2005-10-05 04:00:00 UTC

Description Jere Leppanen 2005-01-20 14:42:37 UTC
Description of problem:
SCTP accepts chunks from any interface even when the socket has been 
bound to a specific interface with socket option SO_BINDTODEVICE. 
This causes unnecessary blocking in connect() and successful 
connections from another interface than the one that the listening 
socket is bound to.

Version-Release number of selected component (if applicable):

How reproducible:
Every time.

Steps to Reproduce:
1. /usr/sbin/tethereal -i eth0 ip proto 132
2. Server: socket(PF_INET, SOCK_STREAM, 0x84 /* IPPROTO_??? */) = 3
3. Server: setsockopt(3, SOL_SOCKET, SO_BINDTODEVICE, "eth0\0", 5) = 0
4. Server: bind(3, {sa_family=AF_INET, sin_port=htons(10000), 4. 
sin_addr=inet_addr("")}, 16) = 0
5. Server: listen(3, 16)                           = 0
6. Server: accept(3,
7. Client: socket(PF_INET, SOCK_STREAM, 0x84 /* IPPROTO_??? */) = 3
8. Client: bind(3, {sa_family=AF_INET, sin_port=htons(0), 
sin_addr=inet_addr("")}, 16) = 0
9. Client: connect(3, {sa_family=AF_INET, sin_port=htons(10000), 
sin_addr=inet_addr("")}, 16) = -1 ETIMEDOUT (Connection 
timed out)

Actual results:
Tethereal says:
Capturing on eth0
  0.000000 ->    SCTP INIT_ACK
  2.997677 ->    SCTP INIT_ACK
  8.996764 ->    SCTP INIT_ACK
 20.994947 ->    SCTP INIT_ACK
 44.991297 ->    SCTP INIT_ACK
 92.984011 ->    SCTP INIT_ACK
152.974881 ->    SCTP INIT_ACK
212.965761 ->    SCTP INIT_ACK

The client connects to and the INIT goes via the loopback 
interface. That INIT is accepted by the server, even though the 
server is bound to interface eth0. The server sends back an INIT_ACK 
via the eth0 interface. The INIT_ACK never reaches the client. The 
client does the usual number of INIT retransmissions and the server 
accepts and responds to all. Finally the client gives up and connect
() returns error with ETIMEDOUT. If the client connects to instead of, the entire four-way handshake 
goes through and an association is established.

Expected results:
When the server receives an INIT - or any other chunk(?) - from the 
wrong interface, it should reply with an ABORT. The ABORT should be 
sent via the same interface that the offending chunk was received 
from. The client should then return from connect() immediately 
with "connection refused".

Additional info:

Comment 2 Patrick C. F. Ernzer 2005-01-26 14:41:20 UTC
These have now been reproduced under RHEL 4 RC by jere.

Comment 5 Jere Leppanen 2005-05-03 09:28:54 UTC
Created attachment 113956 [details]
Patch to discard packets from wrong interface

This patch discards incoming packets if the socket is bound to a specific
interface and the packet is received from another interface. This tackles the
problem of connections being formed via the wrong interface. Whether something
else should be done in addition to discarding the packets, I'm not sure.

Comment 7 Neil Horman 2005-05-17 20:31:57 UTC
Dang, I wish I had seen Jeres update, I just developed the same patch (except I
check arriving device after I check rcvq depth).  It looks pretty sane to me,
and passes all my tests.  I'll post this upstream, and to rhkl once its accepted.

Comment 9 Jere Leppanen 2005-05-18 08:21:30 UTC

After I posted the patch here, I asked about this on the lksctp-developers list 
and Sridhar Samudrala replied.

> If a socket is bound to a specific interface and we receive a matching packet 
> from another interface, should it be discarded or treated as an OOTB packet? 
> (I'm guessing OOTB. That would be the equivalent of what TCP does, right?)

> Treating as OOTB seems to be the right option.

Treating as OOTB should be straightforward. If you can still look into it 
before posting upstream, that would be great.

Comment 10 Neil Horman 2005-05-18 12:29:20 UTC
I'm having the same conversation with him now, and I interpreted the TCP
implementation to treat the frame as though it never arrived, although I'm not
100% sure.  Now that I'm looking at it again this morning though, it appears TCP
only checks the sk_bound_dev_if field in the receive path in
__tcp_v4_lookup_listener, so I could see how properly formatted TCP frames which
arrived on the wrong interface might go down the same path at OOTB frames.  What
do you think the correct action should be for frames arriving on the wrong
interface should be in sctp?  Should we strictly follow section 8.4 of the RFC,
or just send an ABORT chunk in response to each chunk of the frame?

Comment 11 Jere Leppanen 2005-05-18 13:07:15 UTC
My logic is that a packet coming in via the wrong interface is basically the 
same as a packet with a wrong destination address. We don't have a socket bound 
to that interface so it's exactly as if we didn't have a socket bound to the 
packet's destination address. Therefore we cannot identify the assoc for the 
packet, which is the precondition in Sec. 8.4. So there's room for 
interpretation here but I think Sec 8.4 is the closest thing there is in the 
RFC. I also considered just replying to each packet with an ABORT, but what 
part of the RFC would that behaviour relate to?

The same case with TCP always produces RST as reply to a SYN (I tested this), 
so respectively with SCTP, at least an INIT should always produce an ABORT.

Comment 12 Neil Horman 2005-05-18 15:06:47 UTC
Created attachment 114512 [details]
new version of patch to treat packets from unbound interfaces as OOTB

Jere, would you mind giving this patch a go please?  I agree that 8.4 of the
RFC is the closest thing that mimics TCP behavior for packets received on
unbound interfaces, and this packet should do just that, in that it will treat
all packets received on unbound interfaces as OOTB.  We may need some tweaking
for some chunk types (specifically I'm a little worried about new association
chunks), but I think this will more or less bring us into line with how TCP
does this.

Comment 13 Neil Horman 2005-05-18 17:08:05 UTC
hmm, scratch that last comment.  I still think doing some ootb work may be in
order, but just using sctp_rcv_ootb seems be oopsing for me.  I've got some more
work to do.

Comment 14 Neil Horman 2005-05-19 01:41:34 UTC
Created attachment 114545 [details]
updated version of BINDTODEVICE/OOTB patch

Well, I'm a dummy.  I found the error that I made in the previous patch (My
last patch wasn't guaranteed to set the ep pointer, which was needed to find
the sock strucutre in some cases.  The result was the oops that the previous
patch produced.  This new patch corrects that, and passes the brief tests that
I've done here.  Jere, if you wouldn't mind, Could you please test out this
most recent patch and give me your feedback?  Thanks!

Comment 15 Neil Horman 2005-05-19 20:06:53 UTC
Created attachment 114589 [details]
new version of bindtodevice/ootb patch

new version of the last patch, which correctly sets the sk structure when
receiving the ootb frame.

Comment 16 Neil Horman 2005-05-20 16:06:54 UTC
Created attachment 114633 [details]
again a new version of the ootb/bindtodevice patch

Urhg....Its a gutterball week.	New version of the same patch that fixes up the
reference count leaks that I missed last time around.

Comment 17 Neil Horman 2005-05-24 20:23:20 UTC
Created attachment 114800 [details]
new approach to handling bindtodevice as ootb

After a little discussion on the list, the consensus is that this may be a
better approach for handling frames that arrive on an interface other than the
one specified in SO_BINDTODEVICE.

Jere, can you test this patch out please.  I'm setting up to try some more
involved testing of this here, but I'd appreciate a second opinion.  Thanks

Comment 18 Neil Horman 2005-05-26 19:40:55 UTC
Created attachment 114882 [details]
update to previous patch

update to previous version of patch that corrects a reference count imbalance.

Comment 19 Neil Horman 2005-06-02 18:54:25 UTC
Created attachment 115101 [details]
final version of the patch which has upstream acceptance.

Comment 20 Jere Leppanen 2005-06-09 08:28:49 UTC
I tried the latest patch version but there's no change with my test case. Is

    /* If the packet is an OOTB packet which is temporarily on the
     * control endpoint, respond with an ABORT.
    if (ep == sctp_sk((sctp_get_ctl_sock()))->ep)

in sctp_sf_do_5_1B_init() the condition we're trying to hit in the "INIT from 
wrong interface" case?

Comment 21 Neil Horman 2005-06-09 11:03:43 UTC
I believe so, yes.  It appeared to work for my test case, can you provide me
with yours?

Comment 22 Jere Leppanen 2005-06-09 11:17:43 UTC
I can make one that I can provide to you. It's a small task but it may be a 
while before I can get to it. Meanwhile, let's assume your patch works until I 
can prove otherwise.

Comment 40 Red Hat Bugzilla 2005-10-05 12:41:51 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on the solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.


Note You need to log in before you can comment on or make changes to this bug.