Bug 197343
Summary: | BEA 1.5 JVM crash on JBoss 4.0.4GA test case | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Product: | Red Hat Enterprise Linux 4 | Reporter: | Thomas Fitzsimmons <fitzsim> | ||||||||||
Component: | java-1.5.0-bea | Assignee: | Georges Saab <gsaab> | ||||||||||
Status: | CLOSED DUPLICATE | QA Contact: | |||||||||||
Severity: | medium | Docs Contact: | |||||||||||
Priority: | medium | ||||||||||||
Version: | 4.0 | CC: | cilla, fnasser, johan.walles, ldimaggi, ryan.campbell | ||||||||||
Target Milestone: | --- | ||||||||||||
Target Release: | --- | ||||||||||||
Hardware: | x86_64 | ||||||||||||
OS: | Linux | ||||||||||||
Whiteboard: | |||||||||||||
Fixed In Version: | Doc Type: | Bug Fix | |||||||||||
Doc Text: | Story Points: | --- | |||||||||||
Clone Of: | Environment: | ||||||||||||
Last Closed: | 2006-11-30 17:33:29 UTC | Type: | --- | ||||||||||
Regression: | --- | Mount Type: | --- | ||||||||||
Documentation: | --- | CRM: | |||||||||||
Verified Versions: | Category: | --- | |||||||||||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||||||
Cloudforms Team: | --- | Target Upstream Version: | |||||||||||
Embargoed: | |||||||||||||
Bug Depends On: | |||||||||||||
Bug Blocks: | 203681 | ||||||||||||
Attachments: |
|
Description
Thomas Fitzsimmons
2006-06-30 12:31:52 UTC
Modified Summary only - added version #'s Created attachment 132227 [details]
JRockit internal bug filing guidelines, for reference
BEA's internal JRockit bug filing guidelines. Some of this talks about
internal systems that you obviously cannot access, but the questions asked
should be easy enough to answer.
Here's the reply I wrote in our internal bug tracking system. This case is CR284516 to us. This answer is really directed at support (which is from where I usually get bug reports), but I prefer to post an exact copy here rather than re-writing it. We'll see for how long I manage to keep the two bug trackers consistent :-). " Hi! 1. Having core dumps, as well as .dump files from the Dragon crashes as well would be excellent! 2. How reproducable is this? That's one of the questions asked by http://wiki.jrpg.bea.com/display/SUPPORT/Create+a+JRockit+CR. Please follow those instructions when filing bugs for CCE. 3. I'd like to eliminate code-gc as a reason for these problems. Please run R26.4 with -XXnocodegc and see if you can still reproduce. 4. I'd like to eliminate code optimizations as a reason for these problems. Please run with -Xnoopt and see if you can still reproduce. If you can't, please run with -Xverbose:opt and try to find which method gets mis-optimized by creating an opt file from the last optimized methods before the crash. Run with -Xnoopt and put +themethodsyouwantoptimized in the opt file: " Directives are used for saying that certain methods should not be optimized, or should be forced optimized. By specifying a file with -Djrockit.optfile=<filename> this file is used for reading directives to the optimizer. The file contains lines with methods and directives. * + means: preoptimize this method and allow the hotspot detector to optimize it. * - means: never optimize this method. An optfile could look something like this: " - bm/time/StaticFactorialTest.fact(I)I + java/lang/FloatingDecimal.dtoa + java/lang/Object - sun/awt/windows/WComponentPeer.set* " " " I added responses between lines of ---- characters below - I haven't been able to locate any output from -Xverbose:opt - where should it be visible? ------ Additional Comments From johan.walles 2006-07-11 07:17 EST ------- Here's the reply I wrote in our internal bug tracking system. This case is CR284516 to us. This answer is really directed at support (which is from where I usually get bug reports), but I prefer to post an exact copy here rather than re-writing it. We'll see for how long I manage to keep the two bug trackers consistent :-) . " Hi! 1. Having core dumps, as well as .dump files from the Dragon crashes as well would be excellent! The core files are not being created - any ideas why? ------------------------------------------------------------------ .qa.[root@x86_64-4as bin]# more jrockit.26635.dump ===== BEGIN DUMP ============================================================= JRockit dump produced after 0 days, 00:00:47 on Tue Jul 11 13:45:13 2006 Additional information is available in: /opt/jboss-4.0.4.GA-src/build/output/jboss-4.0.4.GA/bin/jrockit.26635.dump /opt/jboss-4.0.4.GA-src/build/output/jboss-4.0.4.GA/bin/core or core.26635 If you see this dump, please open a support case with BEA and supply as much information as you can on your system setup and the program you were running. You can also search for solutions to your problem at http://forums.bea.com in the forum jrockit.developer.interest.general. Error Message: Illegal memory access. [54] Signal info : si_signo=11, si_code=1 si_addr=0x229c6813e0 Version : BEA JRockit(R) R26.4.0-63-63688-1.5.0_06-20060626-2259-linux-x86_64 GC : System optimized over throughput (initial strategy singleparpar) .qa.[root@x86_64-4as bin]# ulimit unlimited .qa.[root@x86_64-4as bin]# ls -lt total 360 -rw-r--r-- 1 root root 16562 Jul 11 13:45 jrockit.26635.dump No idea why core files are not being written. ------------------------------------------------------------------ 2. How reproducible is this? That's one of the questions asked by http://wiki.jrpg.bea.com/display/SUPPORT/Create+a+JRockit+CR. Please follow those instructions when filing bugs for CCE. ------------------------------------------------------------------ 100% reproducible ------------------------------------------------------------------ 3. I'd like to eliminate code-gc as a reason for these problems. Please run R26.4 with -XXnocodegc and see if you can still reproduce. ------------------------------------------------------------------ The above dump is with that switch set. ------------------------------------------------------------------ 4. I'd like to eliminate code optimizations as a reason for these problems. Please run with -Xnoopt and see if you can still reproduce. ------------------------------------------------------------------ It's still reproducible with -Xnoopt. ------------------------------------------------------------------ If you can't, please run with -Xverbose:opt and try to find which method gets mis-optimized by creating an opt file from the last optimized methods before the crash. Run with -Xnoopt and put +themethodsyouwantoptimized in the opt file: ------------------------------------------------------------------ I tried running with -Xverbose:opt, but I can't see any verbose output being generated or written to a file - where should it be? ------------------------------------------------------------------ " Directives are used for saying that certain methods should not be optimized, or should be forced optimized. By specifying a file with -Djrockit.optfile=<filename> this file is used for reading directives to the optimizer. The file contains lines with methods and directives. * + means: preoptimize this method and allow the hotspot detector to optimize it. * - means: never optimize this method. An optfile could look something like this: " - bm/time/StaticFactorialTest.fact(I)I + java/lang/FloatingDecimal.dtoa + java/lang/Object - sun/awt/windows/WComponentPeer.set* " " The test that is causing the failure is: org.jboss.test.cmp2.audit.test.AuditUnitTestCase This is the test following the test originally listed in this bug - org.jboss.test.cmp2.batchcascadedelete.test.BatchCascadeDeleteUnitTestCase I'm attaching a jrockit dump file with this failure. I'm not sure why the test that causes the failure changed - the server in question has not been reinstalled. Created attachment 132265 [details]
jrockit dump file
I don't understand either why core files aren't being generated. It usually depends on not having write access (but you do, otherwise you wouldn't get the .dump file), having limited the core file size using ulimit (but then the .dump file should say so), not having enough disk space (but that should give you a truncated core file). 1. Can you try "ulimit -a" before running? Doing just "ulimit" reports only one limit, and it's not the core file size one. 2. I have a vague feeling that I've heard that core files don't get dumped onto NFS file systems. Could that be the case here? 3. Try doing "export JROCKIT_DUMP_PATH=/tmp" before running and see if you get a core file in /tmp. 4. Please attach the full .dump file from running with -XXnocodegc to the bugzilla. 5. If / when you get core file(s), please post *both* the core file and the corresponding .dump file. Core files compress well, bzip2 is your friend :-). The output from -Xverbose:opt goes to stderr. But if no methods are optimized (which is definitely possible), you won't get any output. I've numbered a couple of requests above to make them easier to answer. Setting the tmp dir env variable fixed the no-core file problem. I have a 40+MB bzip2'ed core file. It's too large to attach to this bugzilla - I'm looking for an externally visible place to store the file. Also - I've been trying to find a subset of the test suite that can run successfully without a core dump - not much luck yet - I've eliminated these tests from the test suite but am still seeing core dumps: <exclude name="**/AuditUnitTestCase.class"/> <exclude name="**/cmp2/**/*UnitTestCase.class"/> <exclude name="**/cts/**/*UnitTestCase.class"/> <exclude name="**/deployers/**/*UnitTestCase.class"/> <exclude name="**/dbtest/**/*UnitTestCase.class"/> Added Ryan Campbell to cc: list. Created attachment 132329 [details]
Dump file
Ryan Campbell putting the corresponding core file on a JBoss server.
Here is the file for download: http://testsuite.jboss.com/core.1526.bz2 Thanks, I got the core file and it looks good. Will get back to you with an analysis when I have one. This is being tracked at BEA as CR284516 George and Johan - have you found anything yet? The core dumping is still blocking our testing. Thanks! I'm sorry for not getting back to you. This issue has just gotten its BEA-internal priority raised, but it may still be a couple of days before you get any update here. The original bug report and dump file were against JRockit R26.4. Unfortunately the core file you supplied is produced with an older version (R26.3). We want to make sure that we are analysing the right problem - could you produce and provide us with a core file obtained running on R26.4, please? Sorry about the mixup - the server that was used to generate the core file has since been reinstalled. Can you send me a pointer to a URL from which I can download the correct version of the JDK/JVM? Thanks! Follow the JRockit 5.0 link from http://commerce.bea.com/products/weblogicjrockit/jrockit_prod_fam.jsp Just to be certain - this is the JRockit version, right? BEA JRockit(R) 1.4.2_08 JVM R24.5.0-61 http://download2.bea.com/pub/jrockit/142_08/jrockit-24.5.0-j2sdk1.4.2_08-linux-ia32.bin No, that is a fairly old 1.4.2 VM! You want the 1.5 (marketing name 5.0) VM, the correct version is (as shown in the dump above) BEA JRockit(R) R26.4.0-63-63688-1.5.0_06-20060626-2259-linux-x86_64 If you downloaded from the link I added above, then check your path, because what you are running is not what I directed you to... Thanks - I'm retrying the test with this download: http://download2.bea.com/pub/jrockit/50/jrockit-R26.4.0-jdk1.5.0_06-linux-ia32.bin It's this version: [root@dl385-02 opt]# ./jrockit-R26.4.0-jdk1.5.0_06/bin/java -version java version "1.5.0_06" Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_06-b05) BEA JRockit(R) (build R26.4.0-63-63688-1.5.0_06-20060626-2259-linux-ia32, ) Created attachment 134461 [details]
jrockit.NNN.dump file
Test results - on an x86_64 system: ---> 32-bit JDK java version "1.5.0_06" Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_06-b05) BEA JRockit(R) (build R26.4.0-63-63688-1.5.0_06-20060626-2259-linux-ia32, ) http://download2.bea.com/pub/jrockit/50/jrockit-R26.4.0-jdk1.5.0_06-linux-ia32.bin Results = JBoss 4.0.4GA test suite runs to completion - no core dumps, pass rate = 94% ---> 64-bit JDK java version "1.5.0_06" Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_06-b05)BEA JRockit(R) (build R26.4.0-63-63688-1.5.0_06-20060626-2259-linux-x86_64, ) http://download2.bea.com/pub/jrockit/50/jrockit-R26.4.0-jdk1.5.0_06-linux-x64.bin Core dump with test: org.jboss.test.cmp2.audit.test.AuditUnitTestCase This is the same problem as is listed above. I'll attach the jrockit.NNN.dump file to this bugzilla and will add a pointer to the core file as soon as I can get it onto an externally visible http server. The core file can be accessed here: server: ftp.jboss.com user: public password: st@pl3r Just login and do a "get bea.tar.gz" Got a very short and simple repro from Vivek Lakshmanan in bug 203681. Patched builds available at: ftp://jrpgftp.jrpg.bea.com/cce/patches/CR284516/R26.4.0 java -version should yield: java version "1.5.0_06" Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_06-b05) BEA JRockit(R) (build R26.4.0-63_CR284516-68626-1.5.0_06-20060926-0859-linux-x86_64, ) Please let us know if they fix this problem. *** This bug has been marked as a duplicate of 203681 *** |