Bug 88399
| Summary: | segfault in /lib/libpthread.so.0 | ||
|---|---|---|---|
| Product: | [Retired] Red Hat Linux | Reporter: | Joseph Shraibman <jks> |
| Component: | glibc | Assignee: | Jakub Jelinek <jakub> |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Brian Brock <bbrock> |
| Severity: | high | Docs Contact: | |
| Priority: | medium | ||
| Version: | 8.0 | CC: | fweimer |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | athlon | ||
| OS: | Linux | ||
| URL: | http://p1.selectacast.net/~jks/hs_err_pid25570.log | ||
| Whiteboard: | |||
| Fixed In Version: | 2.3.2-4.80.6 | Doc Type: | Bug Fix |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2003-10-08 22:57:17 UTC | Type: | --- |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Please try ftp://people.redhat.com/jakub/glibc/errata/8.0/ 2.3.2-4.80 because of a kernel bug doesn't use /lib/i686/libpthread.so.0 but /lib/libpthread.so.0 (that is a compatibility with old kernels version, with fixed size 2MB thread stacks, so with 3000 * 2M you might very well reach address space limits (libraries are mapped at around 1GB, kernel starts at 3GB and thread stack array is in between). What is the kernel bug? Will I run into it if I install these rpms? Do I need to reboot my machine after this? And is this fixed in redhat 9? The kernel bug is that AT_PLATFORM aux vector is passed wrong value to the applications by the kernel. Try LD_SHOW_AUXV=1 /bin/echo, AT_PLATFORM should be i686 on i686+ box, but 8.0 kernel passes an empty string. This is fixed in RHL9 kernel and hopefully will be fixed in next 8.0 errata kernel too when/if that happens. As for the glibc errata, 2.3.2-4.80-6 will be pushed officially as errata today. Upgrading to glibc-2.3.2-4.80.6 solved this problem. I just upgraded a system to redhat 9 and this bug seems to have regressed.
An unexpected exception has been detected in native code outside the VM.
Unexpected Signal : 11 occurred at PC=0x4002CB28
Function=(null)+0x4002CB28
Library=/lib/libpthread.so.0
NOTE: We are unable to locate the function name symbol for the error
just occurred. Please refer to release documentation for possible
reason and solutions.
Current Java thread:
at java.lang.Thread.start(Native Method)
- locked <0x44cd3df8> (a java.lang.Thread)
at com.xtenit.mail.DeliverySystem2Imp.deliver0(DeliverySystem2Imp.java:923)
at
com.xtenit.mail.DeliverySystem2Imp.deliver0wrapper(DeliverySystem2Imp.java:496)
at com.xtenit.mail.DeliverySystem2Imp.deliver(DeliverySystem2Imp.java:481)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:324)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:261)
at sun.rmi.transport.Transport$1.run(Transport.java:148)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:144)
at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:460)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:701)
at java.lang.Thread.run(Thread.java:534)
Dynamic libraries:
08048000-0804e000 r-xp 00000000 03:03 2736898 /usr/local/j2sdk1.4.2/bin/java
0804e000-0804f000 rw-p 00005000 03:03 2736898 /usr/local/j2sdk1.4.2/bin/java
40000000-40012000 r-xp 00000000 03:03 376922 /lib/ld-2.3.2.so
40012000-40013000 rw-p 00011000 03:03 376922 /lib/ld-2.3.2.so
40014000-4001c000 r-xp 00000000 03:03 1294916
/usr/local/j2sdk1.4.2/jre/lib/i386/native_threads/libhpi.so
4001c000-4001d000 rw-p 00007000 03:03 1294916
/usr/local/j2sdk1.4.2/jre/lib/i386/native_threads/libhpi.so
4001d000-40021000 rw-s 00000000 03:03 868361 /tmp/hsperfdata_runner/22448
40021000-40022000 r--p 00a08000 03:03 425990 /usr/lib/locale/locale-archive
40022000-40030000 r-xp 00000000 03:03 376917 /lib/libpthread-0.10.so
40030000-40033000 rw-p 0000e000 03:03 376917 /lib/libpthread-0.10.so
40073000-40075000 r-xp 00000000 03:03 376860 /lib/libdl-2.3.2.so
40075000-40076000 rw-p 00002000 03:03 376860 /lib/libdl-2.3.2.so
40076000-40196000 r-xp 00000000 03:03 376837 /lib/libc-2.3.2.so
40196000-40199000 rw-p 00120000 03:03 376837 /lib/libc-2.3.2.so
4019c000-40748000 r-xp 00000000 03:03 1294918
/usr/local/j2sdk1.4.2/jre/lib/i386/server/libjvm.so
40748000-407a2000 rw-p 005ab000 03:03 1294918
/usr/local/j2sdk1.4.2/jre/lib/i386/server/libjvm.so
407b5000-407c5000 r-xp 00000000 03:03 376868 /lib/libnsl-2.3.2.so
407c5000-407c6000 rw-p 00010000 03:03 376868 /lib/libnsl-2.3.2.so
407c8000-407e8000 r-xp 00000000 03:03 376866 /lib/libm-2.3.2.so
407e8000-407e9000 rw-p 00020000 03:03 376866 /lib/libm-2.3.2.so
407e9000-407ec000 r--s 00000000 03:03 3801317
/usr/local/j2sdk1.4.2/jre/lib/ext/dnsns.jar
407ec000-407f4000 r-xp 00000000 03:03 376907 /lib/libnss_nis-2.3.2.so
407f4000-407f5000 rw-p 00007000 03:03 376907 /lib/libnss_nis-2.3.2.so
407f5000-407f6000 r-xp 00000000 03:03 1098823
/usr/local/j2sdk1.4.2/jre/lib/i386/librmi.so
407f6000-407f7000 rw-p 00000000 03:03 1098823
/usr/local/j2sdk1.4.2/jre/lib/i386/librmi.so
407f7000-40801000 r-xp 00000000 03:03 376893 /lib/libnss_files-2.3.2.so
40801000-40802000 rw-p 00009000 03:03 376893 /lib/libnss_files-2.3.2.so
40802000-40812000 r-xp 00000000 03:03 1098805
/usr/local/j2sdk1.4.2/jre/lib/i386/libverify.so
40812000-40814000 rw-p 0000f000 03:03 1098805
/usr/local/j2sdk1.4.2/jre/lib/i386/libverify.so
40814000-40834000 r-xp 00000000 03:03 1098806
/usr/local/j2sdk1.4.2/jre/lib/i386/libjava.so
40834000-40836000 rw-p 0001f000 03:03 1098806
/usr/local/j2sdk1.4.2/jre/lib/i386/libjava.so
40836000-4084a000 r-xp 00000000 03:03 1098808
/usr/local/j2sdk1.4.2/jre/lib/i386/libzip.so
4084a000-4084d000 rw-p 00013000 03:03 1098808
/usr/local/j2sdk1.4.2/jre/lib/i386/libzip.so
4084d000-421d8000 r--s 00000000 03:03 1115216
/usr/local/j2sdk1.4.2/jre/lib/rt.jar
42222000-42238000 r--s 00000000 03:03 1114890
/usr/local/j2sdk1.4.2/jre/lib/sunrsasign.jar
42238000-42313000 r--s 00000000 03:03 1115213
/usr/local/j2sdk1.4.2/jre/lib/jsse.jar
42313000-42324000 r--s 00000000 03:03 1114921
/usr/local/j2sdk1.4.2/jre/lib/jce.jar
42324000-4287d000 r--s 00000000 03:03 1115214
/usr/local/j2sdk1.4.2/jre/lib/charsets.jar
44925000-4492e000 r--s 00000000 03:03 1081522 /usr/local/java/classes/cos.jar
78c48000-78e48000 r--p 00000000 03:03 425990 /usr/lib/locale/locale-archive
78e48000-78e7a000 r--p 009ba000 03:03 425990 /usr/lib/locale/locale-archive
78e7a000-78e96000 r--s 00000000 03:03 3801316
/usr/local/j2sdk1.4.2/jre/lib/ext/sunjce_provider.jar
78e96000-78ea3000 r--s 00000000 03:03 3801318
/usr/local/j2sdk1.4.2/jre/lib/ext/ldapsec.jar
78ea3000-78f5f000 r--s 00000000 03:03 3801319
/usr/local/j2sdk1.4.2/jre/lib/ext/localedata.jar
78f5f000-78faa000 r--s 00000000 03:03 1081481 /usr/local/java/classes/smtp.jar
78faa000-78fba000 r-xp 00000000 03:03 1098811
/usr/local/j2sdk1.4.2/jre/lib/i386/libnet.so
78fba000-78fbb000 rw-p 0000f000 03:03 1098811
/usr/local/j2sdk1.4.2/jre/lib/i386/libnet.so
78fbb000-78fc9000 r--s 00000000 03:03 1081451
/usr/local/java/classes/activation.102.jar
78fc9000-78fe5000 r--s 00000000 03:03 1081774
/usr/local/java/classes/xml-apis.xalan241.jar
78fe5000-78ffb000 r--s 00000000 03:03 1081514
/usr/local/java/classes/NetComponents.jar
78ffb000-78ffe000 r-xp 00000000 03:03 376878 /lib/libnss_dns-2.3.2.so
78ffe000-78fff000 rw-p 00002000 03:03 376878 /lib/libnss_dns-2.3.2.so
79100000-79150000 r--s 00000000 03:03 1081779
/usr/local/java/classes/mail-1.3.1ea.jar
79150000-7921c000 r--s 00000000 03:03 1081444
/usr/local/java/classes/xerces.221.jar
7921c000-79318000 r--s 00000000 03:03 1081773
/usr/local/java/classes/xalan.241.jar
79318000-79334000 r--s 00000000 03:03 1081774
/usr/local/java/classes/xml-apis.xalan241.jar
79334000-79370000 r--s 00000000 03:03 1081780
/usr/local/java/classes/bsh-1.3b2.jar
79370000-7937d000 r--s 00000000 03:03 1081546 /usr/local/java/classes/mbox.jar
7937d000-79385000 r--s 00000000 03:03 1081543
/usr/local/java/classes/gnu-regexp-1.1.2.jar
79385000-793b3000 r--s 00000000 03:03 1081776
/usr/local/java/classes/dnsjava-1.3.2.jar
793c1000-793ce000 r-xp 00000000 03:03 376926 /lib/libresolv-2.3.2.so
793ce000-793cf000 rw-p 0000d000 03:03 376926 /lib/libresolv-2.3.2.so
Heap at VM Abort:
Heap
def new generation total 8384K, used 5549K [0x44930000, 0x45240000, 0x49e80000)
eden space 7488K, 62% used [0x44930000, 0x44dbb600, 0x45080000)
from space 896K, 100% used [0x45080000, 0x45160000, 0x45160000)
to space 896K, 0% used [0x45160000, 0x45160000, 0x45240000)
tenured generation total 50556K, used 37999K [0x49e80000, 0x4cfdf000, 0x74930000)
the space 50556K, 75% used [0x49e80000, 0x4c39bde0, 0x4c39be00, 0x4cfdf000)
compacting perm gen total 16384K, used 7293K [0x74930000, 0x75930000, 0x78930000)
the space 16384K, 44% used [0x74930000, 0x7504f428, 0x7504f600, 0x75930000)
Local Time = Tue Oct 7 22:52:54 2003
Elapsed Time = 36848
#
# The exception above was detected in native code outside the VM
#
# Java VM: Java HotSpot(TM) Server VM (1.4.2-b28 mixed mode)
#
# An error report file has been saved as hs_err_pid22448.log.
# Please refer to the file for further information.
#
****************
Another exception has been detected while we were handling last error.
Dumping information about last error:
ERROR REPORT FILE = hs_err_pid22448.log
PC = 0x4002cb28
SIGNAL = 11
FUNCTION NAME = (N/A)
OFFSET = 0x4002CB28
LIBRARY NAME = /lib/libpthread.so.0
Please check ERROR REPORT FILE for further information, if there is any.
Good bye.
Also interesting: On my single athlon laptop:
$ strace date 2>&1 | grep lib
open("/lib/tls/libpthread.so.0", O_RDONLY) = 3
open("/lib/tls/librt.so.1", O_RDONLY) = 3
open("/lib/tls/libc.so.6", O_RDONLY) = 3
open("/usr/lib/locale/locale-archive", O_RDONLY|O_LARGEFILE) = 3
$ uname -a
Linux jks-laptop 2.4.20-20.9 #1 Mon Aug 18 11:27:43 EDT 2003 i686 athlon i386
GNU/Linux
On my dual athlon server:
]# strace date 2>&1 | grep lib
open("/lib/libpthread.so.0", O_RDONLY) = 3
open("/lib/librt.so.1", O_RDONLY) = 3
open("/lib/libc.so.6", O_RDONLY) = 3
open("/usr/lib/locale/locale-archive", O_RDONLY|O_LARGEFILE) = 3
]# uname -a
Linux p2.selectacast.net 2.4.20-20.9smp #1 SMP Mon Aug 18 11:18:01 EDT 2003 i686
athlon i386 GNU/Linux
Never mind. It seems that during the upgrade somehow the i386 version of glibc was installed instead of the i686 version. So this isn't a glibc problem but a different problem. Perhaps a check can be added to the rpms to make sure that at least a warning is printed if the i386 rpm gets installed on a better system? Redhat won't run on a 386 anymore anyway. |
From Bugzilla Helper: User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.4a) Gecko/20030401 Description of problem: When running a java program that uses a lot of threads (up to 3000ish) I get this: An unexpected exception has been detected in native code outside the VM. Unexpected Signal : 11 occurred at PC=0x4002CFEC Function=(null)+0x4002CFEC Library=/lib/libpthread.so.0 NOTE: We are unable to locate the function name symbol for the error just occurred. Please refer to release documentation for possible reason and solutions. Current Java thread: at java.lang.Thread.start(Native Method) - locked <0x4578a288> (a java.lang.Thread) at org.xbill.DNS.SimpleResolver.sendAsync(SimpleResolver.java:315) at org.xbill.DNS.ExtendedResolver.sendTo(ExtendedResolver.java:148) - locked <0x45789ed0> (a java.util.HashMap) at org.xbill.DNS.ExtendedResolver.send(ExtendedResolver.java:248) - locked <0x45789ea0> (a java.util.LinkedList) at org.xbill.DNS.Lookup.lookup(Lookup.java:384) at org.xbill.DNS.Lookup.resolve(Lookup.java:424) at org.xbill.DNS.Lookup.run(Lookup.java:441) at org.xbill.DNS.dns.getRecords(dns.java:146) at org.xbill.DNS.dns.getAnyRecords(dns.java:195) <snip> Dynamic libraries: 08048000-0804e000 r-xp 00000000 03:03 2245061 /usr/local/j2sdk1.4.1_02/bin/java 0804e000-0804f000 rw-p 00005000 03:03 2245061 /usr/local/j2sdk1.4.1_02/bin/java 40000000-40012000 r-xp 00000000 03:03 376907 /lib/ld-2.3.2.so 40012000-40013000 rw-p 00011000 03:03 376907 /lib/ld-2.3.2.so 40013000-4001c000 r-xp 00000000 03:03 1655143 /usr/local/j2sdk1.4.1_02/jre/lib/i386/native_threads/libhpi.so 4001c000-4001d000 rw-p 00008000 03:03 1655143 /usr/local/j2sdk1.4.1_02/jre/lib/i386/native_threads/libhpi.so 4001e000-4001f000 r--p 00a08000 03:03 426253 /usr/lib/locale/locale-archive 4001f000-40022000 r--s 00000000 03:03 2457778 /usr/local/j2sdk1.4.1_02/jre/lib/ext/dnsns.jar 40022000-40030000 r-xp 00000000 03:03 376926 /lib/libpthread-0.10.so 40030000-40033000 rw-p 0000e000 03:03 376926 /lib/libpthread-0.10.so 40074000-40076000 r-xp 00000000 03:03 376857 /lib/libdl-2.3.2.so 40076000-40077000 rw-p 00001000 03:03 376857 /lib/libdl-2.3.2.so 40077000-40199000 r-xp 00000000 03:03 376854 /lib/libc-2.3.2.so 40199000-4019e000 rw-p 00121000 03:03 376854 /lib/libc-2.3.2.so 401a1000-4063a000 r-xp 00000000 03:03 2097204 /usr/local/j2sdk1.4.1_02/jre/lib/i386/server/libjvm.so 4063a000-40872000 rw-p 00498000 03:03 2097204 /usr/local/j2sdk1.4.1_02/jre/lib/i386/server/libjvm.so <rest snipped becuse if I don't apache complains that the request uri is too long> After that error appears the jvm doesn't core dump or go away, it just sits there. Existing rmi calls just never return and new ones result in: java.rmi.ConnectIOException: error during JRMP connection establishment; nested exception is: java.net.SocketTimeoutException: Read timed out Version-Release number of selected component (if applicable): glibc-2.3.2-4.80 How reproducible: Sometimes Steps to Reproduce: Seems hard to reproduce. It seems to require a lot of threads and high memory usage Additional info: redhat 8.0 dual athlon system java version 1.4.1_02 from java.sun.com We never got this crash before the recent glibc upgrade, but that doesn't mean anything. This crash also occured with jvm 1.4.1_01. The crash occurs when spinning off a thread. We have had this crash with in different parts of the code that spin off threads. It occurs with client and server jvms.