JVM crashes of the same type occurred multiple times during stress-tests. The crash log below can be found on https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/jdg-perf-cs-leveldb/ws/hs_err_pid24541.log Core dump is located on https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/jdg-perf-cs-leveldb/ws/core.24541 More crash logs along with core dumps can be found in https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/jdg-perf-cs-leveldb/ws/ # # A fatal error has been detected by the Java Runtime Environment: # # SIGBUS (0x7) at pc=0x00007fe66e041d37, pid=24541, tid=140622771934976 # # JRE version: 7.0_17-b02 # Java VM: Java HotSpot(TM) 64-Bit Server VM (23.7-b01 mixed mode linux-amd64 compressed oops) # Problematic frame: # v ~StubRoutines::jbyte_disjoint_arraycopy # # Core dump written. Default location: /qa/hudson_workspace/workspace/jdg-perf-cs-leveldb/core or core.24541 # # If you would like to submit a bug report, please visit: # http://bugreport.sun.com/bugreport/crash.jsp # --------------- T H R E A D --------------- Current thread (0x00007fe5c804c800): JavaThread "leveldb-compaction-0" [_thread_in_Java, id=25036, stack(0x00007fe54a4e5000,0x00007fe54a5e6000)] siginfo:si_signo=SIGBUS: si_errno=0, si_code=2 (BUS_ADRERR), si_addr=0x00007fe5ce919000 Registers: RAX=0x00000000adfa3d46, RBX=0x0000000000000017, RCX=0x0000000000000007, RDX=0x0000000000000000 RSP=0x00007fe54a5e4750, RBP=0x00007fe54a5e4750, RSI=0x00007fe5ce918ff8, RDI=0x00000006f37863f8 R8 =0x000000071a0e8498, R9 =0x0000000000018000, R10=0x00007fe66e042760, R11=0x00007fe5ce919000 R12=0x0000000000000000, R13=0x00000006f37863f0, R14=0x00007fe5ce919007, R15=0x00007fe5c804c800 RIP=0x00007fe66e041d37, EFLAGS=0x0000000000010202, CSGSFS=0x0000000000000033, ERR=0x0000000000000006 TRAPNO=0x000000000000000e Top of Stack: (sp=0x00007fe54a5e4750) 0x00007fe54a5e4750: 0000000000000000 00007fe66ec49be7 0x00007fe54a5e4760: 0000000720661610 0000000000000007 0x00007fe54a5e4770: 000000071a0e8498 0000000700018000 0x00007fe54a5e4780: 0000000720661610 0000018c00000007 0x00007fe54a5e4790: 0000000720d65810 00000006f37863d0 0x00007fe54a5e47a0: de6f0c7e00000001 00000006f3786318 0x00007fe54a5e47b0: 00000006f37863a0 0000005700000007 0x00007fe54a5e47c0: ffffff9dffffff93 0000018300000183 0x00007fe54a5e47d0: fffffff8ffffffdb bb684693ffffffba 0x00007fe54a5e47e0: 00000037ffffffaf 00000006f3785d68 0x00007fe54a5e47f0: 0000000000000031 0000000000000246 0x00007fe54a5e4800: 00000006f3784c50 00007fe66f38329c 0x00007fe54a5e4810: 00000006f36991d8 0000018cde6d31e7 0x00007fe54a5e4820: 00000006f3698f38 00000006f3698f18 0x00007fe54a5e4830: de6d323be40e3eec 000000072071f760 0x00007fe54a5e4840: 0000000000000000 0000000720134460 0x00007fe54a5e4850: 00000006f3699578 de6f098ae402688c 0x00007fe54a5e4860: 00000006f3786318 00000006f3785c80 0x00007fe54a5e4870: e40cc2c800000081 0000000719f3acb8 0x00007fe54a5e4880: 0000000720cc0c80 00000006f3699170 0x00007fe54a5e4890: 0000000720a85968 00007fe66e1345b4 0x00007fe54a5e48a0: 00000000e40cc2b6 00007fe66ed54a60 0x00007fe54a5e48b0: 0000000000000007 00000007e2da26b8 0x00007fe54a5e48c0: 00000005cffbe801 0000000000000001 0x00007fe54a5e48d0: 00007fe54a5e4ae0 00007fe66e7294d4 0x00007fe54a5e48e0: 00000000000000c4 00000007e0aa1488 0x00007fe54a5e48f0: 0000000000000000 0000000000000000 0x00007fe54a5e4900: 0000000000000000 0000000000000000 0x00007fe54a5e4910: 0000000000000000 0000000000000000 0x00007fe54a5e4920: 0000000000000000 0000000000000000 0x00007fe54a5e4930: 0000000000000000 0000000000000000 0x00007fe54a5e4940: 0000000000000000 0000000000000000 Instructions: (pc=0x00007fe66e041d37) 0x00007fe66e041d17: da e9 79 00 00 00 48 8b 44 d7 08 48 89 44 d6 08 0x00007fe66e041d27: 48 ff c2 75 f1 f7 c1 04 00 00 00 74 0e 8b 47 08 0x00007fe66e041d37: 89 46 08 48 83 c7 04 48 83 c6 04 f7 c1 02 00 00 0x00007fe66e041d47: 00 74 10 66 8b 47 08 66 89 46 08 48 83 c7 02 48 Register to memory mapping: RAX=0x00000000adfa3d46 is an unknown value RBX=0x0000000000000017 is an unknown value RCX=0x0000000000000007 is an unknown value RDX=0x0000000000000000 is an unknown value RSP=0x00007fe54a5e4750 is pointing into the stack for thread: 0x00007fe5c804c800 RBP=0x00007fe54a5e4750 is pointing into the stack for thread: 0x00007fe5c804c800 RSI=0x00007fe5ce918ff8 is an unknown value RDI=
I'll check if this occurs on other JVMs (1.6)
The same error occurred on all machines with JVM versions 6.0_43-b01 and 7.0_09-b05
LevelDB uses reflection in order to unmap memory-mapped files in an unsafe way, according to http://bugs.sun.com/view_bug.do?bug_id=4724038 This may be reason why the JVM is crashing. Suggested workaround is command-line flag -Dleveldb.mmap=false which forces using FileChannelTable implementation for accessing the disk.
Since this is a JVM bug, and there is a workaround I'm closing this issue as CANTFIX.