Bug 205451 (sctp_sendmsg_bug)

Summary: Cannot use sctp_sendmsg() from 32bit app on 64bit kernel
Product: Red Hat Enterprise Linux 4 Reporter: Eric Z. Ayers <eric.ayers>
Component: lksctp-toolsAssignee: Karsten Hopp <karsten>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: medium Docs Contact:
Priority: medium    
Version: 4.4CC: jesse.marlin
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: 1.0.2-6.4E.1 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2007-06-08 17:50:49 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Sample sctp client/server code. none

Description Eric Z. Ayers 2006-09-06 13:35:40 UTC
Description of problem:
We distribute a 32 bit version of our application.  We cannot run the 32 bit
version when we use SCTP on 64 bit RHEL 4 installation.  

Version-Release number of selected component (if applicable):

]$ uname -a
Linux blade01-1 2.6.9-34.0.1.ELsmp #1 SMP Wed May 17 16:59:36 EDT 2006 x86_64
x86_64 x86_64 GNU/Linux

$ cat /etc/issue
Red Hat Enterprise Linux ES release 4 (Nahant Update 4)

$ rpm -q lksctp-tools
lksctp-tools-1.0.2-6.4E.1


How reproducible:

#1) Try to compile application with 32 bit gcc linked against libsctp (no 32 bit
libsctp is provided)

#2) If you copy the 32 bit library from the 32 bit RHEL4 distribution, you get 
EINVAL on the call to sctp_sendmsg():

The same binary works if you run in on a 32 bit kernel, or if you compile the
code in 64 bit and run it on a 64 bit kernel.

#3) We took the sctp_sendmsg() code from libsctp and included it inline and
recompiled to eliminate the libsctp dependency.  It is the sendmsg() system call
that is failing.

Actual results:

-bash-3.00$ /home/cgi/tmp/test_sctp_sendmsg_server_32

SCTP TEST SERVER Starting
Server Listening on Port 30492
Client says --Client Says Hello--, 17 bytes
sctp_sendmsg()=-1, errno=22(Invalid argument)

SCTP TEST SERVER Ending


Additional info:

See attached sample server and client.
Run the client with the argument -host localhost.

Comment 1 Eric Z. Ayers 2006-09-06 13:35:40 UTC
Created attachment 135651 [details]
Sample sctp client/server code.

Comment 2 Eric Z. Ayers 2006-09-06 13:37:51 UTC
Related to issues 203059, 203054 which have been closed because they are using
the wrong version of lksctp, but that is not the case here.  Can you give us a
way to use our 32 bit application on a RHEL4 installed with a 64 bit kernel?

Comment 3 Eric Z. Ayers 2006-09-14 14:07:38 UTC
Can someone give me an update on this issue?

Comment 4 Eric Z. Ayers 2006-10-09 20:55:52 UTC
By the time anyone looks at this issue I might have to raise a new one about
running on 128bit processors...  No matter, it works OK on Sun so that is what
we will be proposing.

Comment 5 Neil Horman 2007-06-08 14:58:23 UTC
#1) Try to compile application with 32 bit gcc linked against libsctp (no 32 bit
libsctp is provided)

Not sure what you mean by this, 32 bit libsctp is included in
lksctp-tools-1.0.2-6.4E.1.i386.rpm

You should be able to install that on your 64 bit system and link against it no
problem.  Not sure what the result will be if you link against the 64 bit
library and run against the 32 bit.  Please try installing/linking against the
32 bit libraries first.  Thanks!

Comment 6 Jesse Marlin 2007-06-08 16:39:32 UTC
(In reply to comment #5)
> #1) Try to compile application with 32 bit gcc linked against libsctp (no 32 bit
> libsctp is provided)
> 
> Not sure what you mean by this, 32 bit libsctp is included in
> lksctp-tools-1.0.2-6.4E.1.i386.rpm
> 
> You should be able to install that on your 64 bit system and link against it no
> problem.  Not sure what the result will be if you link against the 64 bit
> library and run against the 32 bit.  Please try installing/linking against the
> 32 bit libraries first.  Thanks!

You cannot link 32/64 bit binaries.  They have to be one or the other.

Here are the configurations we tested:

- 64 bit kernel, 64 bit app   Worked
- 64 bit kernel, 32 bit app   Did not work (EINVAL)
- 32 bit kernel, 32 bit app   Worked

We thought there might be something either wrong with the library or kernel
module.  I can't remember if we built libsctp for 32 bit or used a version from
RHEL4, but only the 64 bit version was included with RHEL4.5.

This was for RHEL 4.5, and we have since upgraded to RHEL 5 and have not
revisited this again.


Comment 7 Eric Z. Ayers 2007-06-08 17:07:18 UTC
Hell Neil,

I could not find such a package in the distribution I was given to install on m
the RHEL4u3 cluster:

/srv/kits/X86-64/redhat/rhel4u3/RedHat/RPMS/lksctp-tools-devel-1.0.2-6.4E.1.x86_64.rpm
/srv/kits/X86-64/redhat/rhel4u3/RedHat/RPMS/lksctp-tools-1.0.2-6.4E.1.x86_64.rpm
/srv/kits/X86-64/redhat/rhel4u3/RedHat/RPMS/lksctp-tools-doc-1.0.2-6.4E.1.x86_64.rpm

We have since upgrade to RHEL 5.1 and I found the .i386 RPM in that
distribution.  When rebuilding the test programs after installing the RPM I
noticed there was no symlink

/usr/lib/libsctp.so  --> libsctp.so.1.0.6 

which kept GCC from being able to link when invoked with -m32 to compile 32 bit
executables:

eric@blade01:/src/dcs_5.1/dcs/src$ gcc -m32  -o
/obj/dcs_5.1/dcs/bin/test_sctp_server /obj/dcs_5.1/dcs/obj/test_sctp_server.o
-L/obj/dcs_5.1/dcs/lib -ldcs -ldcs_dp -ldisk -ldict_swap -lcgdc -lpra -ldcs_cgdc
-lfeat -leps -L/opt/dcs_5.1/lib -lcgr -lcgip -leal -luio -lcom -levent -ldl
-lmenu -lrudp -lrun -lrpt -ldict_swap -lpthread  -lfile  -L/usr/lib -lsctp
/usr/bin/ld: skipping incompatible /usr/lib64/libsctp.so when searching for -lsctp
/usr/bin/ld: skipping incompatible /usr/lib64/libsctp.a when searching for -lsctp
/usr/bin/ld: cannot find -lsctp
collect2: ld returned 1 exit status

After creating the symlink I can link a 32 bit binary.

After re-running the tests on RHEL 5.1, I don't see the problem anymore, so
consider the issue resolved.

Thanks for looking into it,
-Eric.