Bug 1206656

Example of a failed task: http://koji.fedoraproject.org/koji/taskinfo?taskID=9324657

This problem only shows for a release type build. A slowdebug build on arm 32 works fine.

I have this reproduced in an arm virtual machine, but cannot look at this with gdb because of bug 1196181.

Product build:

$ ./jdk8.build/images/j2sdk-image/bin/java -version
Error occurred during initialization of VM
java.lang.StackOverflowError
	at java.lang.Object.<clinit>(Object.java:41)

Slowdebug build:

$ ./linux-arm-normal-zero-slowdebug/images/j2sdk-image/bin/java -version
openjdk version "1.8.0_40-debug"
OpenJDK Runtime Environment (build 1.8.0_40-debug-b25)
OpenJDK Zero VM (build 25.40-b25-debug, interpreted mode)

$ uname -a
Linux f22-arm 4.0.0-0.rc1.git0.1.fc22.armv7hl+lpae #1 SMP Mon Feb 23 21:13:18 UTC 2015 armv7l armv7l armv7l GNU/Linux

I was able to get a core file by building a fastdebug hotspot. Then, I've used gdb to get a back trace of when this StackOverflowError happens. It looks something like this:

#2  0xb69f0204 in os::abort (dump_core=<optimized out>) at /home/sgehwolf/java-1.8.0-openjdk/java-1.8.0-openjdk-1.8.0.40-21.b25.fc22.arm/jdk8/hotspot/src/os/linux/vm/os_linux.cpp:1542
#3  0xb6bb3064 in VMError::report_and_die (this=this@entry=0xb60d674c) at /home/sgehwolf/java-1.8.0-openjdk/java-1.8.0-openjdk-1.8.0.40-21.b25.fc22.arm/jdk8/hotspot/src/share/vm/utilities/vmError.cpp:1094
#4  0xb6545294 in report_vm_error (file=<optimized out>, line=line@entry=227, error_msg=0xb6c44410 "assert(InstanceKlass::cast(k)->is_initialized()) failed", 
    detail_msg=0xb6c44004 "need to increase min_stack_allowed calculation") at /home/sgehwolf/java-1.8.0-openjdk/java-1.8.0-openjdk-1.8.0.40-21.b25.fc22.arm/jdk8/hotspot/src/share/vm/utilities/debug.cpp:226
#5  0xb65a1a98 in Exceptions::throw_stack_overflow_exception (__the_thread__=__the_thread__@entry=0xb5e07ba8, 
    file=0xb6c80cfc "/home/sgehwolf/java-1.8.0-openjdk/java-1.8.0-openjdk-1.8.0.40-21.b25.fc22.arm/jdk8/hotspot/src/share/vm/runtime/javaCalls.cpp", line=line@entry=386, method=...)
    at /home/sgehwolf/java-1.8.0-openjdk/java-1.8.0-openjdk-1.8.0.40-21.b25.fc22.arm/jdk8/hotspot/src/share/vm/utilities/exceptions.cpp:227
#6  0xb6733434 in JavaCalls::call_helper (result=<optimized out>, m=<optimized out>, args=<optimized out>, __the_thread__=0xb5e07ba8)
    at /home/sgehwolf/java-1.8.0-openjdk/java-1.8.0-openjdk-1.8.0.40-21.b25.fc22.arm/jdk8/hotspot/src/share/vm/runtime/javaCalls.cpp:386
#7  0xb673219c in JavaCalls::call (result=result@entry=0xb60d68f8, method=..., args=args@entry=0xb60d690c, __the_thread__=__the_thread__@entry=0xb5e07ba8)
    at /home/sgehwolf/java-1.8.0-openjdk/java-1.8.0-openjdk-1.8.0.40-21.b25.fc22.arm/jdk8/hotspot/src/share/vm/runtime/javaCalls.cpp:307
#8  0xb66c33c8 in InstanceKlass::call_class_initializer_impl (this_oop=..., __the_thread__=__the_thread__@entry=0xb5e07ba8)
    at /home/sgehwolf/java-1.8.0-openjdk/java-1.8.0-openjdk-1.8.0.40-21.b25.fc22.arm/jdk8/hotspot/src/share/vm/oops/instanceKlass.cpp:1213
#9  0xb66c3568 in InstanceKlass::call_class_initializer (this=<optimized out>, __the_thread__=__the_thread__@entry=0xb5e07ba8)
    at /home/sgehwolf/java-1.8.0-openjdk/java-1.8.0-openjdk-1.8.0.40-21.b25.fc22.arm/jdk8/hotspot/src/share/vm/oops/instanceKlass.cpp:1181
#10 0xb66c3868 in InstanceKlass::initialize_impl (this_oop=..., __the_thread__=0xb5e07ba8)
    at /home/sgehwolf/java-1.8.0-openjdk/java-1.8.0-openjdk-1.8.0.40-21.b25.fc22.arm/jdk8/hotspot/src/share/vm/oops/instanceKlass.cpp:917
#11 0xb66c6700 in InstanceKlass::initialize (this=0xa37b52a8, __the_thread__=0xb5e07ba8)
    at /home/sgehwolf/java-1.8.0-openjdk/java-1.8.0-openjdk-1.8.0.40-21.b25.fc22.arm/jdk8/hotspot/src/share/vm/oops/instanceKlass.cpp:564
#12 0xb66c3a88 in InstanceKlass::initialize_impl (this_oop=..., __the_thread__=0xb5e07ba8)
    at /home/sgehwolf/java-1.8.0-openjdk/java-1.8.0-openjdk-1.8.0.40-21.b25.fc22.arm/jdk8/hotspot/src/share/vm/oops/instanceKlass.cpp:882
#13 0xb66c6700 in InstanceKlass::initialize (this=0xa37bbf10, __the_thread__=0xb5e07ba8)
    at /home/sgehwolf/java-1.8.0-openjdk/java-1.8.0-openjdk-1.8.0.40-21.b25.fc22.arm/jdk8/hotspot/src/share/vm/oops/instanceKlass.cpp:564
#14 0xb6b34bbc in initialize_class (class_name=<optimized out>, __the_thread__=0xb5e07ba8)
    at /home/sgehwolf/java-1.8.0-openjdk/java-1.8.0-openjdk-1.8.0.40-21.b25.fc22.arm/jdk8/hotspot/src/share/vm/runtime/thread.cpp:993
#15 0xb6b4405c in Threads::create_vm (args=args@entry=0xb60d6e2c, canTryAgain=canTryAgain@entry=0xb60d6d80)
    at /home/sgehwolf/java-1.8.0-openjdk/java-1.8.0-openjdk-1.8.0.40-21.b25.fc22.arm/jdk8/hotspot/src/share/vm/runtime/thread.cpp:3487
#16 0xb6755fdc in JNI_CreateJavaVM (vm=0xb60d6e24, penv=0xb60d6e28, args=0xb60d6e2c) at /home/sgehwolf/java-1.8.0-openjdk/java-1.8.0-openjdk-1.8.0.40-21.b25.fc22.arm/jdk8/hotspot/src/share/vm/prims/jni.cpp:5201
#17 0xb6f3122c in InitializeJVM (ifn=<synthetic pointer>, penv=0xb60d6e20, pvm=0xb60d6e1c) at /home/sgehwolf/java-1.8.0-openjdk/java-1.8.0-openjdk-1.8.0.40-21.b25.fc22.arm/jdk8/jdk/src/share/bin/java.c:1214
#18 JavaMain (_args=<optimized out>) at /home/sgehwolf/java-1.8.0-openjdk/java-1.8.0-openjdk-1.8.0.40-21.b25.fc22.arm/jdk8/jdk/src/share/bin/java.c:376
#19 0xb6f6ef44 in start_thread (arg=0xb60d7470) at pthread_create.c:335
#20 0xb6ea6540 in ?? () at ../sysdeps/unix/sysv/linux/arm/clone.S:89 from /lib/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

The interesting clue is this: src/share/vm/runtime/javaCalls.cpp:386 so the branch where no stack shadow pages are available is taken. Looking at the implementation of that (src/share/vm/runtime/os.cpp:1301) we see this:

address sp = current_stack_pointer();

Then the Zero implementation of current_stack_pointer() looks like this (src/os_cpu/linux_zero/vm/os_linux_zero.cpp):

address os::current_stack_pointer() {
  address dummy = (address) &dummy;
  return dummy;
}

Knowing that we needed -Wno-return-local-addr in F22/rawhide and a slowdebug Zero JVM does not show the problem, this was the final clue to try this code instead. Note that a release type build compiles os_linux_zero.cpp with -O3.

address __attribute__((optimize("O0"))) os::current_stack_pointer() {
  address dummy = (address) &dummy;
  return dummy;
}

And, indeed, without GCC optimizing this function the JVM starts to work (this is with recompiled os_linux_zero.o and relinked libjvm.so in /home/sgehwolf/java-1.8.0-openjdk/java-1.8.0-openjdk-1.8.0.40-21.b25.fc22.arm/jdk8-debug/build/linux-arm-normal-zero-fastdebug/hotspot/linux_arm_zero/fastdebug:

$ JAVA_HOME=/home/sgehwolf/java-1.8.0-openjdk/java-1.8.0-openjdk-1.8.0.40-21.b25.fc22.arm/jdk8-debug/build/jdk8.build/images/j2sdk-image /home/sgehwolf/java-1.8.0-openjdk/java-1.8.0-openjdk-1.8.0.40-21.b25.fc22.arm/jdk8-debug/build/jdk8.build/images/j2sdk-image/bin/java -XXaltjvm=/home/sgehwolf/java-1.8.0-openjdk/java-1.8.0-openjdk-1.8.0.40-21.b25.fc22.arm/jdk8-debug/../jdk8/build/linux-arm-normal-zero-fastdebug/hotspot/linux_arm_zero/fastdebug -Dsun.java.launcher=gamma -version
openjdk version "1.8.0_40"
OpenJDK Runtime Environment (build 1.8.0_40-b25)
OpenJDK Zero VM (build 25.40-b25-fastdebug, interpreted mode)

Created attachment 1009087 [details]
Proposed work-around spec-file patch for F22.

I've kicked off a scratch build with the patch in comment 5:
http://koji.fedoraproject.org/koji/taskinfo?taskID=9380164

Created attachment 1009534 [details]
Proposed final fix for f22/rawhide ARM32 build problem

F22 scratch build with patch in comment 7:
http://koji.fedoraproject.org/koji/taskinfo?taskID=9382154

Assigning over to Jiri so that he can push the patch and kick of real builds. Jiri, the patch in comment 7 can be applied with "git am" on f22 branch and then cherry-picked over to master (rawhide). Feel free to close this bug once done. Thanks!

After Jiri notified me about this error - I'm seeing the same error with java-1.8.0-openjdk-1.8.0.40-25.b25.fc21.src.rpm when built on f22 on s390 (http://s390.koji.fedoraproject.org/koji/taskinfo?taskID=1771499)

...
+ /builddir/build/BUILD/java-1.8.0-openjdk-1.8.0.40-25.b25.fc22.s390/jdk8/build/jdk8.build/images/j2sdk-image/bin/javac -d . /builddir/build/SOURCES/TestCryptoLevel.java
Error occurred during initialization of VM
java.lang.StackOverflowError
	at java.lang.Object.<clinit>(Object.java:41)
error: Bad exit status from /var/tmp/rpm-tmp.ebS1Kl (%build)


will retry with the latest java-1.8.0-openjdk-1.8.0.40-26.b25.fc22

(In reply to Dan Horák from comment #10)
> After Jiri notified me about this error - I'm seeing the same error with
> java-1.8.0-openjdk-1.8.0.40-25.b25.fc21.src.rpm when built on f22 on s390
> (http://s390.koji.fedoraproject.org/koji/taskinfo?taskID=1771499)
> 
> ...
> +
> /builddir/build/BUILD/java-1.8.0-openjdk-1.8.0.40-25.b25.fc22.s390/jdk8/
> build/jdk8.build/images/j2sdk-image/bin/javac -d .
> /builddir/build/SOURCES/TestCryptoLevel.java
> Error occurred during initialization of VM
> java.lang.StackOverflowError
> 	at java.lang.Object.<clinit>(Object.java:41)
> error: Bad exit status from /var/tmp/rpm-tmp.ebS1Kl (%build)
> 
> 
> will retry with the latest java-1.8.0-openjdk-1.8.0.40-26.b25.fc22

That's not surprising since s390 also builds a Zero JVM. It seems GCC 5 + Zero related. GCC 5 seems to perform some additional optimizations that GCC 4.9.x didn't do.

and for the record - java-1.8.0-openjdk-1.8.0.40-26.b25.fc22 builds fine also on s390/s390x - http://s390.koji.fedoraproject.org/koji/taskinfo?taskID=1771508

(In reply to Dan Horák from comment #12)
> and for the record - java-1.8.0-openjdk-1.8.0.40-26.b25.fc22 builds fine
> also on s390/s390x -
> http://s390.koji.fedoraproject.org/koji/taskinfo?taskID=1771508

Thanks, Dan!

java-1.8.0-openjdk-1.8.0.45-32.b13.fc23[1] was built with this patch and it properly self-built. java-1.8.0-openjdk-1.8.0.45-31.b13.fc22 built fine as well. I'm closing this bug.

[1] http://koji.fedoraproject.org/koji/buildinfo?buildID=628648

Is this going upstream? I don't see it in current OpenJDK 9 Hotspot.

(In reply to Andrew John Hughes from comment #15)
> Is this going upstream? I don't see it in current OpenJDK 9 Hotspot.

Yes, the intent is to propose this upstream. It's still on my radar to upstream it.

Pushed upstream with:
http://hg.openjdk.java.net/jdk9/dev/hotspot/rev/6217fd2c767b