Bug 156785 - Missing SHUTDOWN notification with SCTP stream socket
Missing SHUTDOWN notification with SCTP stream socket
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 4
Classification: Red Hat
Component: kernel (Show other bugs)
4.0
i386 Linux
medium Severity medium
: ---
: ---
Assigned To: Neil Horman
Brian Brock
:
Depends On:
Blocks: 168429
  Show dependency treegraph
 
Reported: 2005-05-04 05:03 EDT by Jere Leppanen
Modified: 2007-11-30 17:07 EST (History)
3 users (show)

See Also:
Fixed In Version: RHSA-2006-0132
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2006-03-07 13:59:46 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
backport of my upstream proposal to allow tcp style sctp sockets get shutdown notices (1.28 KB, patch)
2005-09-22 13:17 EDT, Neil Horman
no flags Details | Diff
corrected patch w/ upstream revisions (1.37 KB, patch)
2005-09-22 21:10 EDT, Neil Horman
no flags Details | Diff

  None (edit)
Description Jere Leppanen 2005-05-04 05:03:58 EDT
Description of problem:
SCTP SOCK_STREAM sockets do not deliver SCTP_SHUTDOWN_EVENT notification when 
the peer closes connection. Instead recvmsg() returns 0, which it should do 
only after delivering the notification. With SCTP SOCK_SEQPACKET sockets 
SCTP_SHUTDOWN_EVENT is correctly delivered.

Version-Release number of selected component (if applicable):
kernel-2.6.9-5.EL

How reproducible:
Every time.


Steps to Reproduce:

1. Server: socket(PF_INET, SOCK_STREAM, 0x84 /* IPPROTO_SCTP */) = 3
2. Server: bind(3, {sa_family=AF_INET, sin_port=htons(61000), sin_addr=inet_addr
("127.0.0.1")}, 16) = 0
3. Server: setsockopt(3, 0x84 /* IPPROTO_SCTP */, 11 /* SCTP_EVENTS */, "\1\1\1
\1\1\1\1\1", 8) = 0
4. Server: listen(3, 16)                           = 0

5. Client: socket(PF_INET, SOCK_STREAM, 0x84 /* IPPROTO_SCTP */) = 3
6. Client: bind(3, {sa_family=AF_INET, sin_port=htons(61001), sin_addr=inet_addr
("127.0.0.1")}, 16) = 0
7. Client: connect(3, {sa_family=AF_INET, sin_port=htons(61000), 
sin_addr=inet_addr("127.0.0.1")}, 16) = 0

8. Server: accept(3, 0, NULL)                      = 4
9. Server: recvmsg(4, {msg_name(16)={sa_family=AF_INET, sin_port=htons
(61001),sin_addr=inet_addr("127.0.0.1")}, msg_iov(1)=[{"\0"..., 1048575}], 
msg_controllen=0, msg_flags=MSG_EOR|MSG_MORE}, MSG_NOSIGNAL) = 20

10. Client: <killed>

11. Server: recvmsg(4, {msg_name(0)={sa_family=AF_UNSPEC, sa_data="\0"}, msg_iov
(1)=[{"\0"..., 1048575}], msg_controllen=0, msg_flags=0}, MSG_NOSIGNAL) = 0

  
Actual results:
All SCTP events are subscribed in step 3. In step 9, the server gets an 
SCTP_ASSOC_CHANGE with COMM_UP notification and the association is up. The 
client is then killed in step 10, which causes the association to shut down. In 
step 11 recvmsg() returns 0 for the server.

Expected results:
In step 11, recvmsg() should deliver the SCTP_SHUTDOWN_EVENT notification. The 
next recvmsg() should return 0.

Additional info:
This is RFC draft non-compliance. The required behaviour is defined in "Sockets 
API Extensions for Stream Control Transmission Protocol" (draft-ietf-tsvwg-
sctpsocket-10.txt). The obvious workaround is to treat recvmsg()=0 as an 
SCTP_SHUTDOWN_EVENT.
Comment 5 Neil Horman 2005-09-09 16:34:11 EDT
As a personal test here, I modified the sctp_darn utility to subscribe to
shutdown notifications, so that when operating in receive mode, it would bail
out in test_verify_assoc_change, printing the notification type and exiting on
shutdown, when a shutdown event was recieved from it peer (assuming that the
notification was in fact delivered to userspace, which I did not expect, given
this bug report).  When I ran two interactive sctp_darn sessions (one in send
mode, the other in listen), with this modification, I sent a shutdown from the
send side, and was suprised to see the listening side bail out, printing an
indication that it had received an "unknown" notification (one other than the
SCTP_ASSOC_CHANGE that it expected).  The utility printed the notification type,
which was sn_type=32773.  This appears to be the defined value for
SCTP_SHUTDOWN_EVENT.  So it would appear that as long as the the peer socket is
shutdown cleanly, shutdown notifications are properly received.  If I repeat the
test and exit the sending peer with a ctrl-c or a kill -9, I get the same
results.  So I'm unable to reproduce this misssing shutdown notification using
my test setup.  Can you please confirm this?  I'm currently using the 2.6.9-17
RHEL4 kernel, and the sctp_darn utility from lksctp-tools-1.0.2-6.4E.1, modified
to subscribe to shutdown notifications in with this patch:
--- lksctp-tools-1.0.2/src/apps/sctp_darn.c.orig        2005-09-09
16:34:08.000000000 -0400
+++ lksctp-tools-1.0.2/src/apps/sctp_darn.c     2005-09-09 16:34:24.000000000 -0400
@@ -476,6 +476,7 @@ build_endpoint(char *argv0, int portnum)
                memset(&subscribe, 0, sizeof(subscribe));
                subscribe.sctp_data_io_event = 1;
                subscribe.sctp_association_event = 1;
+               subscribe.sctp_shutdown_event = 1;
                error = setsockopt(retval, SOL_SCTP, SCTP_EVENTS,
                                   (char *)&subscribe, sizeof(subscribe));
                if (error) {
Comment 6 Jere Leppanen 2005-09-14 05:37:35 EDT
Re-tested with kernel 2.6.9-17.TEST.IT79426smp (should be no different from 
plain 2.6.9-17 in this case, right?) and sctp_darn from lksctp-tools-1.0.2-
6.4E.1 with Neil's patch from comment #5. Results were identical with my first 
description.

Neil, if you were able to successfully send the shutdown, I guess you were 
using SOCK_SEQPACKET at least on the client side. Sending shutdown doesn't 
appear to be working with SOCK_STREAM sockets. Please double-check that you're 
using SOCK_STREAM on the server side. SHUTDOWN notification works correctly 
with SOCK_SEQPACKET, this problem exists only with SOCK_STREAM sockets.
Comment 7 Patrick C. F. Ernzer 2005-09-14 05:57:43 EDT
Jere,

2.6.9-17.TEST.IT79426smp is not a kernel I built, so I cannot comment. Please do
test with plain 2.6.9-17.

Kind regards,

Patrick
Comment 8 Jere Leppanen 2005-09-15 08:53:08 EDT
Re-tested with kernel 2.6.9-17.ELsmp and sctp_darn from lksctp-tools-1.0.2-
6.4E.1 with Neil's patch from comment #5. Again, results were identical with my 
first description.
Comment 9 Neil Horman 2005-09-21 15:30:22 EDT
Ok, I see the behavior now.  I think this may be because in
sctp_sf_do_9_2_shutdown, we add the command to transition state to
SCTP_STATE_SHUTDOWN_RECEIVED before we add the command to queue the SHUTDOWN
ulpevent to the recieve queue.  This causes sctp_cmd_new_state to run in the
state machine first, setting the RCV_SHUTDOWN state on the socket.  Later, for
tcp sytle sockets in sctp_ulpq_tail_event (where the ulpevent is queued), we
check for the RCV_SHUTDOWN flag, and if its set, we drop the frame.  I'm going
to work up a patch that adds the state transition command to the state machine
after the queue command for testing.
Comment 10 Neil Horman 2005-09-22 13:17:44 EDT
Created attachment 119153 [details]
backport of my upstream proposal to allow tcp style sctp sockets get shutdown notices

Heres a backport of the patch I'm proposing upstream to queue shutdown
notifications to the socket receive queue prior to shutting down the socket.
Comment 11 Neil Horman 2005-09-22 21:10:04 EDT
Created attachment 119171 [details]
corrected patch w/ upstream revisions

This is the final upstream version of the patch, taking into account sri's
suggestion changes for the things I broke in the process of fixing the
origional problem :)
Comment 12 Jere Leppanen 2005-09-26 11:13:37 EDT
Re-tested with the latest patch and it's working now, I'm getting SHUTDOWN 
correctly. Good job.
Comment 13 Neil Horman 2005-09-26 12:56:32 EDT
Thanks, This has been accepted upstream, so I'll post internally today.
Comment 20 Red Hat Bugzilla 2006-03-07 13:59:48 EST
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on the solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2006-0132.html

Note You need to log in before you can comment on or make changes to this bug.