The following has be reported by IBM LTC: insmod -m SEGVs on RHEL 3.0 beta2 on x86_64 Hardware Environment: x325 (Opteron) Software Environment: RHEL 3.0 beta2 for x86_64 Steps to Reproduce: 1.Load any kernel module, prebuilt or build from source, with -m option, e.g. insmod -m /lib/modules/2.4.21-1.1931.2.393.ent/kernel/fs/smbfs/smbfs.o 2. 3. Actual Results: (gdb) run -m /lib/modules/2.4.21-1.1931.2.393.ent/kernel/fs/smbfs/smbfs.o Starting program: /sbin/insmod -m /lib/modules/2.4.21-1.1931.2.393.ent/kernel/fs/smbfs/smbfs.o (no debugging symbols found)...(no debugging symbols found)...Sections: Size Address Align .this 000000b8 ffffffffa009f000 2**3 .text 00007287 ffffffffa009f0c0 2**6 .fixup 00000014 ffffffffa00a6347 2**0 .rodata 00000698 ffffffffa00a6360 2**3 .rodata.str1.1 0000040f ffffffffa00a69f8 2**0 .rodata.str1.32 00000fe2 ffffffffa00a6e20 2**5 __ex_table 00000020 ffffffffa00a7e08 2**3 .kstrtab 00000426 ffffffffa00a7e28 2**0 __ksymtab 00000390 ffffffffa00a8250 2**3 __archdata 00000000 ffffffffa00a85e0 2**4 __kallsyms 00001b17 ffffffffa00a85e0 2**2 .data 00000640 ffffffffa00aa100 2**5 .bss 00000000 ffffffffa00aa740 2**2 Program received signal SIGSEGV, Segmentation fault. 0x0000007fbfffb840 in ?? () (gdb) bt #0 0x0000007fbfffb840 in ?? () #1 0x0000002a9569c871 in msort_with_tmp () from /lib64/tls/libc.so.6 #2 0x0000002a9569c786 in msort_with_tmp () from /lib64/tls/libc.so.6 #3 0x0000002a9569c786 in msort_with_tmp () from /lib64/tls/libc.so.6 #4 0x0000002a9569c786 in msort_with_tmp () from /lib64/tls/libc.so.6 #5 0x0000002a9569c786 in msort_with_tmp () from /lib64/tls/libc.so.6 #6 0x0000002a9569c786 in msort_with_tmp () from /lib64/tls/libc.so.6 #7 0x0000002a9569c786 in msort_with_tmp () from /lib64/tls/libc.so.6 #8 0x0000002a9569c959 in qsort () from /lib64/tls/libc.so.6 #9 0x0000000000402f4c in ?? () #10 0x000000000040455c in ?? () #11 0x0000000000405422 in ?? () #12 0x0000000000405a88 in ?? () #13 0x0000002a956884c1 in __libc_start_main () from /lib64/tls/libc.so.6 #14 0x00000000004025ea in ?? () Expected Results: module load map should be dumped to stdout. Additional Information: insmod -m worked fine on SLES8 for x86_64. It doesn't matter is the module being loaded is built from scratch or comes with the distro.Glen/Greg - please report this to Red Hat. Yet another thing that works on SLES8 but not on RHEL3 beta's on Opteron. Thanks.
insmod is part of modutils; reassigning to that component
Works for me with modutils-2.4.25-9.EL and kernel-2.4.21-2.EL. Please try more current code.
------ Additional Comments From volobuev.com 2003-02-10 14:31 ------- This still appears broken on my machine after running up2date this morning. (gdb) run -m /lib/modules/2.4.21-3.EL/kernel/fs/smbfs/smbfs.o Starting program: /sbin/insmod -m /lib/modules/2.4.21-3.EL/kernel/fs/smbfs/smbfs.o (no debugging symbols found)...(no debugging symbols found)...Sections: Size Address Align .this 000000b8 ffffffffa009c000 2**3 .text 00007287 ffffffffa009c0c0 2**6 .fixup 00000014 ffffffffa00a3347 2**0 .rodata 00000698 ffffffffa00a3360 2**3 .rodata.str1.1 0000040f ffffffffa00a39f8 2**0 .rodata.str1.32 00000fe2 ffffffffa00a3e20 2**5 __ksymtab 00000040 ffffffffa00a4e08 2**3 __ex_table 00000020 ffffffffa00a4e48 2**3 .kstrtab 000000b8 ffffffffa00a4e68 2**0 __archdata 00000000 ffffffffa00a4f20 __kallsyms 00001b39 ffffffffa00a4f20 .data 00000640 ffffffffa00a6a60 .bss 00000000 ffffffffa00a70a0 Program received signal SIGSEGV, Segmentatio 0x0000007fbfffb840 in ?? () (gdb) bt #0 0x0000007fbfffb840 in ?? () #1 0x0000002a9569bd31 in msort_with_tmp () #2 0x0000002a9569bc46 in msort_with_tmp () #3 0x0000002a9569bc46 in msort_with_tmp () #4 0x0000002a9569bc46 in msort_with_tmp () #5 0x0000002a9569bc46 in msort_with_tmp () #6 0x0000002a9569bc46 in msort_with_tmp () #7 0x0000002a9569bc46 in msort_with_tmp () #8 0x0000002a9569be19 in qsort () from /lib #9 0x0000000000402f4c in ?? () #10 0x000000000040455c in ?? () #11 0x0000000000405422 in ?? () #12 0x0000000000405a88 in ?? () #13 0x0000002a95688101 in __libc_start_main #14 0x00000000004025ea in ?? () I have modutils-2.4.25-9.EL kernel-2.4.21-3.EL glibc-2.3.2-90
Are you running insmod -m on 32-bit modules?
------ Additional Comments From volobuev.com 2003-02-10 16:05 ------- I think it would be non-trivial to build a 32-bit kernel modules on an Opteron box, and all the prebuilt modules are of course 64-bit. I'm definitely running insmod -m on a 64b module: # file /lib/modules/2.4.21-3.EL/kernel/fs/smbfs/smbfs.o /lib/modules/2.4.21-3.EL/kernel/fs/smbfs/smbfs.o: ELF 64-bit LSB relocatable, AMD x86-64, version 1 (SYSV), not stripped
*** Bug 110622 has been marked as a duplicate of this bug. ***
----- Additional Comments From khoa.com 2004-01-23 10:20 ------- This is on RHEL3 U2 list as a Sev 4.
Received in e-mail: Just an FYI - When I looked at this about a month ago, this is what I found; The insmod program calls qsort() to sort the symbols from the module. Then qsort() tries to decide if it should malloc memory to sort the symbols, as +an optimization, depending on how many symbols there are, and how much memory is available. The qsort() routine calls the sysctl() function to see how much memory is +available on the system. THE SYSCTL() ROUTINE IS BROKEN ON AMD X86_64. Qsort() ends up trying to free an invalid pointer. The sysctl routine is system dependent and needs to be fixed. The problem is not with insmod, it is in glibc. You only see this when qsort decides to malloc memory.
followup: In the previous comment I meant __sysconf() not sysctl(), sorry. Here are the relevant files - __sysconf() comes from one of these files - glibc-2.3.2/sysdeps/generic/sysconf.c glibc-2.3.2/sysdeps/posix/sysconf.c glibc-2.3.2/sysdeps/unix/bsd/ultrix4/sysconf.c glibc-2.3.2/sysdeps/unix/sysv/irix4/sysconf.c glibc-2.3.2/sysdeps/unix/sysv/sysv4/sysconf.c The qsort() routine is in this file - glibc-2.3.2/stdlib/msort.c and makes these calls - __sysconf(_SC_PHYS_PAGES) __sysconf(_SC_PAGESIZE) By default, the second one returns EINVAL and needs to be ported to your platform.
#define _GNU_SOURCE #include <stdio.h> #include <unistd.h> int main (void) { printf ("%d\n", sysconf (_SC_PAGESIZE)); printf ("%d\n", sysconf (_SC_PHYS_PAGES)); } works just fine for me in glibc-2.3.2-95.6 (and 2.3.3-8), both as -m64 and -m32, dynamicly linked and -static. So it must be something else.
But what this perhaps could be is executing code on the stack when kernel has non-executable stack. At least the first address in the backtrace looks like stack address. Does booting with noexec=off noexec32=off kernel command line options help?
----- Additional Comments From khoa.com 2004-02-19 16:27 ------- Yuri - can you respond to RH's question above ?
----- Additional Comments From volobuev.com 2004-02-23 15:08 ------- Yes, booting with noexec=off noexec32=off does help, insmod -m works as it should, no SEGV anymore, so glibc apparently does try to execute something off the stack. Turning on executable stack is not an acceptable solution though, glibc still needs to be fixed.
It is insmod, not glibc which executes it on the stack, it is trampoline for a nested function passed to qsort. And it is not glibc nor insmod which needs to be fixed, but kernel, which should handle PT_GNU_STACK program header and make stack executable or non-executable based on that.
----- Additional Comments From khoa.com 2004-03-28 18:02 ------- This is now a Sev3 on RHEL3 U3 list.
it's also fixed in U2 beta.
An errata has been issued which should help the problem described in this bug report. This report is therefore being closed with a resolution of ERRATA. For more information on the solution and/or where to find the updated files, please follow the link below. You may reopen this bug report if the solution does not work for you. http://rhn.redhat.com/errata/RHBA-2004-089.html
---- Additional Comments From khoa.com 2005-03-27 12:56 EST ------- Yuri - please close this bug report if this is fixed in RHEL3 U3/U4. Thanks.