Bug 106005 - LTC4637-insmod -m SEGVs on RHEL 3.0 beta2 on x86_64
LTC4637-insmod -m SEGVs on RHEL 3.0 beta2 on x86_64
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 3
Classification: Red Hat
Component: kernel (Show other bugs)
3.0
x86_64 Linux
medium Severity low
: ---
: ---
Assigned To: Arjan van de Ven
Brian Brock
:
: 110622 (view as bug list)
Depends On:
Blocks: 107563
  Show dependency treegraph
 
Reported: 2003-10-01 17:55 EDT by IBM Bug Proxy
Modified: 2007-11-30 17:06 EST (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2004-05-12 00:48:10 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description IBM Bug Proxy 2003-10-01 17:55:34 EDT
The following has be reported by IBM LTC:  
insmod -m SEGVs on RHEL 3.0 beta2 on x86_64
Hardware Environment:
x325 (Opteron)
Software Environment:
RHEL 3.0 beta2 for x86_64

Steps to Reproduce:
1.Load any kernel module, prebuilt or build from source, with -m option,
e.g. insmod -m /lib/modules/2.4.21-1.1931.2.393.ent/kernel/fs/smbfs/smbfs.o
2.
3.

Actual Results:
(gdb) run -m /lib/modules/2.4.21-1.1931.2.393.ent/kernel/fs/smbfs/smbfs.o
Starting program: /sbin/insmod -m
/lib/modules/2.4.21-1.1931.2.393.ent/kernel/fs/smbfs/smbfs.o
(no debugging symbols found)...(no debugging symbols found)...Sections:      
Size      Address           Align
.this           000000b8  ffffffffa009f000  2**3
.text           00007287  ffffffffa009f0c0  2**6
.fixup          00000014  ffffffffa00a6347  2**0
.rodata         00000698  ffffffffa00a6360  2**3
.rodata.str1.1  0000040f  ffffffffa00a69f8  2**0
.rodata.str1.32 00000fe2  ffffffffa00a6e20  2**5
__ex_table      00000020  ffffffffa00a7e08  2**3
.kstrtab        00000426  ffffffffa00a7e28  2**0
__ksymtab       00000390  ffffffffa00a8250  2**3
__archdata      00000000  ffffffffa00a85e0  2**4
__kallsyms      00001b17  ffffffffa00a85e0  2**2
.data           00000640  ffffffffa00aa100  2**5
.bss            00000000  ffffffffa00aa740  2**2

Program received signal SIGSEGV, Segmentation fault.
0x0000007fbfffb840 in ?? ()
(gdb) bt
#0  0x0000007fbfffb840 in ?? ()
#1  0x0000002a9569c871 in msort_with_tmp () from /lib64/tls/libc.so.6
#2  0x0000002a9569c786 in msort_with_tmp () from /lib64/tls/libc.so.6
#3  0x0000002a9569c786 in msort_with_tmp () from /lib64/tls/libc.so.6
#4  0x0000002a9569c786 in msort_with_tmp () from /lib64/tls/libc.so.6
#5  0x0000002a9569c786 in msort_with_tmp () from /lib64/tls/libc.so.6
#6  0x0000002a9569c786 in msort_with_tmp () from /lib64/tls/libc.so.6
#7  0x0000002a9569c786 in msort_with_tmp () from /lib64/tls/libc.so.6
#8  0x0000002a9569c959 in qsort () from /lib64/tls/libc.so.6
#9  0x0000000000402f4c in ?? ()
#10 0x000000000040455c in ?? ()
#11 0x0000000000405422 in ?? ()
#12 0x0000000000405a88 in ?? ()
#13 0x0000002a956884c1 in __libc_start_main () from /lib64/tls/libc.so.6
#14 0x00000000004025ea in ?? ()


Expected Results:
module load map should be dumped to stdout.

Additional Information:
insmod -m worked fine on SLES8 for x86_64.  It doesn't matter is the module 
being loaded is built from scratch or comes with the distro.Glen/Greg - please
report this to Red Hat.  Yet another thing that works
on SLES8 but not on RHEL3 beta's on Opteron.  Thanks.
Comment 1 Arjan van de Ven 2003-10-01 17:57:34 EDT
insmod is part of modutils; reassigning to that component
Comment 2 Bill Nottingham 2003-10-01 18:07:13 EDT
Works for me with modutils-2.4.25-9.EL and kernel-2.4.21-2.EL.

Please try more current code.
Comment 3 IBM Bug Proxy 2003-10-02 14:36:29 EDT
------ Additional Comments From volobuev@us.ibm.com  2003-02-10 14:31 -------
This still appears broken on my machine after running up2date this morning.  

(gdb) run -m /lib/modules/2.4.21-3.EL/kernel/fs/smbfs/smbfs.o
Starting program: /sbin/insmod -m /lib/modules/2.4.21-3.EL/kernel/fs/smbfs/smbfs.o
(no debugging symbols found)...(no debugging symbols found)...Sections:      
Size      Address           Align
.this           000000b8  ffffffffa009c000  2**3
.text           00007287  ffffffffa009c0c0  2**6
.fixup          00000014  ffffffffa00a3347  2**0
.rodata         00000698  ffffffffa00a3360  2**3
.rodata.str1.1  0000040f  ffffffffa00a39f8  2**0
.rodata.str1.32 00000fe2  ffffffffa00a3e20  2**5
__ksymtab       00000040  ffffffffa00a4e08  2**3
__ex_table      00000020  ffffffffa00a4e48  2**3
.kstrtab        000000b8  ffffffffa00a4e68  2**0
__archdata      00000000  ffffffffa00a4f20  
__kallsyms      00001b39  ffffffffa00a4f20  
.data           00000640  ffffffffa00a6a60  
.bss            00000000  ffffffffa00a70a0  

Program received signal SIGSEGV, Segmentatio
0x0000007fbfffb840 in ?? ()
(gdb) bt
#0  0x0000007fbfffb840 in ?? ()
#1  0x0000002a9569bd31 in msort_with_tmp () 
#2  0x0000002a9569bc46 in msort_with_tmp () 
#3  0x0000002a9569bc46 in msort_with_tmp () 
#4  0x0000002a9569bc46 in msort_with_tmp () 
#5  0x0000002a9569bc46 in msort_with_tmp () 
#6  0x0000002a9569bc46 in msort_with_tmp () 
#7  0x0000002a9569bc46 in msort_with_tmp () 
#8  0x0000002a9569be19 in qsort () from /lib
#9  0x0000000000402f4c in ?? ()
#10 0x000000000040455c in ?? ()
#11 0x0000000000405422 in ?? ()
#12 0x0000000000405a88 in ?? ()
#13 0x0000002a95688101 in __libc_start_main 
#14 0x00000000004025ea in ?? ()

I have 
modutils-2.4.25-9.EL
kernel-2.4.21-3.EL
glibc-2.3.2-90 
Comment 4 Bill Nottingham 2003-10-02 14:53:05 EDT
Are you running insmod -m on 32-bit modules?
Comment 5 IBM Bug Proxy 2003-10-02 16:11:34 EDT
------ Additional Comments From volobuev@us.ibm.com  2003-02-10 16:05 -------
I think it would be non-trivial to build a 32-bit kernel modules on an Opteron
box, and all the prebuilt modules are of course 64-bit.  I'm definitely running
insmod -m on a 64b module:

# file /lib/modules/2.4.21-3.EL/kernel/fs/smbfs/smbfs.o 
/lib/modules/2.4.21-3.EL/kernel/fs/smbfs/smbfs.o: ELF 64-bit LSB relocatable,
AMD x86-64, version 1 (SYSV), not stripped 
Comment 6 Bill Nottingham 2003-11-21 15:17:02 EST
*** Bug 110622 has been marked as a duplicate of this bug. ***
Comment 7 IBM Bug Proxy 2004-01-23 23:51:17 EST
----- Additional Comments From khoa@us.ibm.com  2004-01-23 10:20 -------
This is on RHEL3 U2 list as a Sev 4. 
Comment 8 Bill Nottingham 2004-02-11 12:17:40 EST
Received in e-mail:

Just an FYI -
When I looked at this about a month ago, this is what I found;
The insmod program calls qsort() to sort the symbols from the module.
Then qsort() tries to decide if it should malloc memory to sort the
symbols, as
+an optimization,
depending on how many symbols there are, and how much memory is available.
The qsort() routine calls the sysctl() function to see how much memory is
+available on the system.
THE SYSCTL() ROUTINE IS BROKEN ON AMD X86_64.
Qsort() ends up trying to free an invalid pointer.
The sysctl routine is system dependent and needs to be fixed.
The problem is not with insmod, it is in glibc.
You only see this when qsort decides to malloc memory.
                                                                     
          
Comment 9 Bill Nottingham 2004-02-11 12:59:21 EST
followup:

In the previous comment I meant __sysconf() not sysctl(), sorry.
Here are the relevant files -
                                                                     
          
__sysconf() comes from one of these files -
    glibc-2.3.2/sysdeps/generic/sysconf.c
    glibc-2.3.2/sysdeps/posix/sysconf.c
    glibc-2.3.2/sysdeps/unix/bsd/ultrix4/sysconf.c
    glibc-2.3.2/sysdeps/unix/sysv/irix4/sysconf.c
    glibc-2.3.2/sysdeps/unix/sysv/sysv4/sysconf.c
                                                                     
          
The qsort() routine is in this file -
    glibc-2.3.2/stdlib/msort.c
                                                                     
          
and makes these calls -
    __sysconf(_SC_PHYS_PAGES)
    __sysconf(_SC_PAGESIZE)
                                                                     
          
By default, the second one returns EINVAL and
needs to be ported to your platform.
                                                                     
          
                                                                     
          
Comment 10 Jakub Jelinek 2004-02-11 14:23:09 EST
#define _GNU_SOURCE
#include <stdio.h>
#include <unistd.h>

int main (void)
{
  printf ("%d\n", sysconf (_SC_PAGESIZE));
  printf ("%d\n", sysconf (_SC_PHYS_PAGES));
}

works just fine for me in glibc-2.3.2-95.6 (and 2.3.3-8), both as -m64 and -m32, dynamicly linked and -static.
So it must be something else.
Comment 11 Jakub Jelinek 2004-02-11 14:49:00 EST
But what this perhaps could be is executing code on the stack when
kernel has non-executable stack.  At least the first address in the
backtrace looks like stack address.
Does booting with noexec=off noexec32=off kernel command line options
help?
Comment 12 IBM Bug Proxy 2004-02-19 16:29:31 EST
----- Additional Comments From khoa@us.ibm.com  2004-02-19 16:27 -------
Yuri - can you respond to RH's question above ? 
Comment 13 IBM Bug Proxy 2004-02-23 15:09:29 EST
----- Additional Comments From volobuev@us.ibm.com  2004-02-23 15:08 -------
Yes, booting with noexec=off noexec32=off does help, insmod -m works as it 
should, no SEGV anymore, so glibc apparently does try to execute something off 
the stack.  Turning on executable stack is not an acceptable solution though, 
glibc still needs to be fixed. 
Comment 14 Jakub Jelinek 2004-02-23 16:31:13 EST
It is insmod, not glibc which executes it on the stack, it is trampoline
for a nested function passed to qsort.
And it is not glibc nor insmod which needs to be fixed, but kernel,
which should handle PT_GNU_STACK program header and make stack
executable or non-executable based on that.
Comment 15 IBM Bug Proxy 2004-03-28 18:01:45 EST
----- Additional Comments From khoa@us.ibm.com  2004-03-28 18:02 -------
This is now a Sev3 on RHEL3 U3 list. 
Comment 16 Arjan van de Ven 2004-03-29 01:04:11 EST
it's also fixed in U2 beta.
Comment 18 John Flanagan 2004-05-12 00:48:10 EDT
An errata has been issued which should help the problem described in this bug report. 
This report is therefore being closed with a resolution of ERRATA. For more information
on the solution and/or where to find the updated files, please follow the link below. You may reopen 
this bug report if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2004-089.html
Comment 19 IBM Bug Proxy 2005-03-27 13:01:49 EST
---- Additional Comments From khoa@us.ibm.com  2005-03-27 12:56 EST -------
Yuri - please close this bug report if this is fixed in RHEL3 U3/U4.  Thanks. 

Note You need to log in before you can comment on or make changes to this bug.