Bug 1306558 - java-1.8.0-openjdk: FTBFS in rawhide
java-1.8.0-openjdk: FTBFS in rawhide
Status: CLOSED CURRENTRELEASE
Product: Fedora
Classification: Fedora
Component: java-1.8.0-openjdk (Show other bugs)
24
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Severin Gehwolf
Fedora Extras Quality Assurance
: Reopened
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2016-02-11 04:59 EST by Severin Gehwolf
Modified: 2016-06-22 03:31 EDT (History)
9 users (show)

See Also:
Fixed In Version: java-1.8.0-openjdk-1.8.0.72-5.b15.fc24
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-06-22 03:29:52 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Use older C/C++ standard with GCC6. (3.74 KB, patch)
2016-02-11 05:10 EST, Severin Gehwolf
no flags Details | Diff
configure work-around (869 bytes, patch)
2016-02-11 05:11 EST, Severin Gehwolf
no flags Details | Diff
spec file patch changing flag to -fno-lifetime-dse (2.01 KB, patch)
2016-02-16 11:44 EST, Severin Gehwolf
omajid: review+
Details | Diff

  None (edit)
Description Severin Gehwolf 2016-02-11 04:59:29 EST
java-1.8.0-openjdk fails to build from source in rawhide. Failed task is here:
http://koji.fedoraproject.org/koji/taskinfo?taskID=12902039

build.log tail looks like this:
b15.fc24.x86_64/openjdk/hotspot/src/share/vm/ci/ciConstant.hpp:29,
                 from /builddir/build/BUILD/java-1.8.0-openjdk-1.8.0.72-4.b15.fc24.x86_64/openjdk/hotspot/src/share/vm/ci/ciArray.hpp:29,
                 from /builddir/build/BUILD/java-1.8.0-openjdk-1.8.0.72-4.b15.fc24.x86_64/openjdk/hotspot/src/share/vm/precompiled/precompiled.hpp:33:
/builddir/build/BUILD/java-1.8.0-openjdk-1.8.0.72-4.b15.fc24.x86_64/openjdk/hotspot/src/share/vm/code/dependencies.hpp:169:59: error: left operand of shift expression '(-1 << 1)' is negative [-fpermissive]
     all_types           = ((1 << TYPE_LIMIT) - 1) & ((-1) << FIRST_TYPE),
                                                     ~~~~~~^~~~~~~~~~~~~~
/builddir/build/BUILD/java-1.8.0-openjdk-1.8.0.72-4.b15.fc24.x86_64/openjdk/hotspot/src/share/vm/code/dependencies.hpp:169:72: error: enumerator value for 'all_types' is not an integer constant
     all_types           = ((1 << TYPE_LIMIT) - 1) & ((-1) << FIRST_TYPE),
                                                                        ^
cc1plus: all warnings being treated as errors
/builddir/build/BUILD/java-1.8.0-openjdk-1.8.0.72-4.b15.fc24.x86_64/openjdk/hotspot/make/linux/makefiles/vm.make:361: recipe for target 'precompiled.hpp.gch' failed
gmake[6]: *** [precompiled.hpp.gch] Error 1
gmake[6]: Leaving directory '/builddir/build/BUILD/java-1.8.0-openjdk-1.8.0.72-4.b15.fc24.x86_64/openjdk/build/jdk8.build/hotspot/linux_amd64_compiler2/product'
/builddir/build/BUILD/java-1.8.0-openjdk-1.8.0.72-4.b15.fc24.x86_64/openjdk/hotspot/make/linux/makefiles/top.make:119: recipe for target 'the_vm' failed
gmake[5]: Leaving directory '/builddir/build/BUILD/java-1.8.0-openjdk-1.8.0.72-4.b15.fc24.x86_64/openjdk/build/jdk8.build/hotspot/linux_amd64_compiler2/product'
gmake[5]: *** [the_vm] Error 2
/builddir/build/BUILD/java-1.8.0-openjdk-1.8.0.72-4.b15.fc24.x86_64/openjdk/hotspot/make/linux/Makefile:293: recipe for target 'product' failed
gmake[4]: Leaving directory '/builddir/build/BUILD/java-1.8.0-openjdk-1.8.0.72-4.b15.fc24.x86_64/openjdk/build/jdk8.build/hotspot'
Makefile:230: recipe for target 'generic_build2' failed
gmake[3]: Leaving directory '/builddir/build/BUILD/java-1.8.0-openjdk-1.8.0.72-4.b15.fc24.x86_64/openjdk/hotspot/make'
gmake[4]: *** [product] Error 2
gmake[3]: *** [generic_build2] Error 2
Makefile:177: recipe for target 'product' failed
gmake[2]: Leaving directory '/builddir/build/BUILD/java-1.8.0-openjdk-1.8.0.72-4.b15.fc24.x86_64/openjdk/hotspot/make'
gmake[2]: *** [product] Error 2
HotspotWrapper.gmk:44: recipe for target '/builddir/build/BUILD/java-1.8.0-openjdk-1.8.0.72-4.b15.fc24.x86_64/openjdk/build/jdk8.build/hotspot/_hotspot.timestamp' failed
gmake[1]: Leaving directory '/builddir/build/BUILD/java-1.8.0-openjdk-1.8.0.72-4.b15.fc24.x86_64/openjdk/make'
gmake[1]: *** [/builddir/build/BUILD/java-1.8.0-openjdk-1.8.0.72-4.b15.fc24.x86_64/openjdk/build/jdk8.build/hotspot/_hotspot.timestamp] Error 2
/builddir/build/BUILD/java-1.8.0-openjdk-1.8.0.72-4.b15.fc24.x86_64/openjdk//make/Main.gmk:108: recipe for target 'hotspot-only' failed
make: *** [hotspot-only] Error 2
Comment 1 Severin Gehwolf 2016-02-11 05:05:39 EST
We have two intermediate patches (attached) which gets us further. However with those it fails in java TestCryptoLevel with:

+ /builddir/build/BUILD/java-1.8.0-openjdk-1.8.0.72-5.b15.fc24.x86_64/openjdk/build/jdk8.build/images/j2sdk-image/bin/javac -d
 . /builddir/build/SOURCES/TestCryptoLevel.java
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007f6b25ed736b, pid=5973, tid=140097488013056
#
# JRE version: OpenJDK Runtime Environment (8.0_72-b15) (build 1.8.0_72-b15)
# Java VM: OpenJDK 64-Bit Server VM (25.72-b15 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# V  [libjvm.so+0x88636b]  Node::Node(unsigned int)+0x2b
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting \
Java again
#
# An error report file with more information is saved as:
# /builddir/build/BUILD/java-1.8.0-openjdk-1.8.0.72-5.b15.fc24.x86_64/hs_err_pid5973.log
[thread 140097475966720 also had an error]
[thread 140097474914048 also had an error]
[thread 140097473861376 also had an error]
[thread 140097477019392 also had an error]
[error occurred during error reporting , id 0xb]

Building a fastdebug hotspot that asserts with:

#  Internal Error (/builddir/build/BUILD/java-1.8.0-openjdk-1.8.0.72-5.b15.fc24.i386/openjdk/hotspot/src/share/vm/opto/node.cpp:298), pid=13094, tid=2730490688
#  assert(Compile::current() == C) failed: must use operator new(Compile*)

See:
http://koji.fedoraproject.org/koji/getfile?taskID=12930370&name=build.log&offset=-4000
Comment 2 Severin Gehwolf 2016-02-11 05:10 EST
Created attachment 1123106 [details]
Use older C/C++ standard with GCC6.
Comment 3 Severin Gehwolf 2016-02-11 05:11 EST
Created attachment 1123107 [details]
configure work-around
Comment 4 Severin Gehwolf 2016-02-11 05:15:45 EST
This is starting to look like an issue with hotspot + GCC6. We are still investigating.
Comment 5 Severin Gehwolf 2016-02-11 05:16:49 EST
A better trace of the fastdebug assertion is:

+ /builddir/build/BUILD/java-1.8.0-openjdk-1.8.0.72-5.b15.fc24.i386/openjdk/build/jdk8.build-debug/images/j2sdk-image/bin/javac -d . /builddir/build/SOURCES/TestCryptoLevel.java
# To suppress the following error report, specify this argument
# after -XX: or in .hotspotrc:  SuppressErrorAt=/node.cpp:298
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  Internal Error (/builddir/build/BUILD/java-1.8.0-openjdk-1.8.0.72-5.b15.fc24.i386/openjdk/hotspot/src/share/vm/opto/node.cpp:298), pid=13094, tid=2730490688
#  assert(Compile::current() == C) failed: must use operator new(Compile*)
#
# JRE version: OpenJDK Runtime Environment (8.0_72-b15) (build 1.8.0_72-fastdebug-b15)
# Java VM: OpenJDK Server VM (25.72-b15-fastdebug mixed mode linux-x86 )
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /builddir/build/BUILD/java-1.8.0-openjdk-1.8.0.72-5.b15.fc24.i386/hs_err_pid13094.log
[thread -1565533376 also had an error]
[error occurred during error reporting , id 0xb]
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
#
Comment 6 Andrew Haley 2016-02-11 05:26:02 EST
Adding a compiler barrier, as shown here, fixes the segfault, which is caused by Node._out not being initialized.  I strongly suspect that GCC does not initialize _out because Compile is not compatible with its type.  While this is, strictly speaking, correct, we are compiling with -fno-strict-aliasing and GCC should respect that.

I very strongly suspect that -fno-strict-aliasing is broken in this version of GCC.

inline void* operator new( size_t x, Compile* C) throw() {
   Node* n = (Node*)C->node_arena()->Amalloc_D(x);
#ifdef ASSERT
   n->_in = (Node**)n; // magic cookie for assertion check
#endif
   n->_out = (Node**)C;
   __asm__ volatile("nop" : : : "memory");
   return (void*)n;
}
Comment 7 Jakub Jelinek 2016-02-11 05:36:01 EST
Have you tried the code with -fsanitize=undefined?  Does -fno-delete-null-pointer-checks help?  Does building with -O0 help?  If yes, can you bisect it to a problematic *.o file, attach it preprocessed here, including all gcc options?
Comment 8 Severin Gehwolf 2016-02-11 05:42:27 EST
(In reply to Jakub Jelinek from comment #7)
> Does building with -O0 help?

Yes. We've done a slowdebug build which seems to work. That's -O0 I believe.

> If yes, can you bisect it to a problematic *.o file, attach it preprocessed
> here, including all gcc options?

It's going to take a while, but it's possible.
Comment 9 Jakub Jelinek 2016-02-11 05:47:53 EST
Well, you shouldn't need to rebuild everything each time, just keep around copies of an optimized and non-optimized builds and then in a third copy always copy in + touch always some set of optimized *.o files and rest of unoptimized *.o files, relink, retest, and using binary search it should take just log2 of number of *.o files.  If multiple shared libraries or binaries are involved, first step can be to bisect among the shared libraries/binaries to find the problematic one.
Comment 10 Andrew Haley 2016-02-11 06:12:16 EST
(In reply to Jakub Jelinek from comment #7)
> Have you tried the code with -fsanitize=undefined? 

No.  -fsanitize=undefined is fairly problematic with HotSpot, given that it warns about several things which are well-defined in GCC and everybody uses all the time, e.g. left shifts of negative integers.  It'd be nice to have a sensible set of options for building with fsanitize=undefined.

> Does
> -fno-delete-null-pointer-checks help?  Does building with -O0 help?  If yes,
> can you bisect it to a problematic *.o file, attach it preprocessed here,
> including all gcc options?

OK.  It's node.o.  I'll get you the preprocessed file.

Andrew.
Comment 11 Marek Polacek 2016-02-11 06:15:51 EST
(In reply to Andrew Haley from comment #10)
> (In reply to Jakub Jelinek from comment #7)
> > Have you tried the code with -fsanitize=undefined? 
> 
> No.  -fsanitize=undefined is fairly problematic with HotSpot, given that it
> warns about several things which are well-defined in GCC and everybody uses
> all the time, e.g. left shifts of negative integers.  It'd be nice to have a
> sensible set of options for building with fsanitize=undefined.

You can try using -fsanitize=null to see if the code isn't calling a member function on a null pointer or similar.
Comment 17 Severin Gehwolf 2016-02-11 06:56:09 EST
(In reply to Jakub Jelinek from comment #9)
> Well, you shouldn't need to rebuild everything each time, just keep around
> copies of an optimized and non-optimized builds and then in a third copy
> always copy in + touch always some set of optimized *.o files and rest of
> unoptimized *.o files, relink, retest, and using binary search it should
> take just log2 of number of *.o files.  If multiple shared libraries or
> binaries are involved, first step can be to bisect among the shared
> libraries/binaries to find the problematic one.

Understood. Still getting a local setup with a non-optimized build, an optimized build and then the script run would have taken a while. Anyway, a moot point now.
Comment 47 Andrew Haley 2016-02-16 09:21:35 EST
-fno-guess-branch-probability is the wrong flag to use because it hits too much other stuff.

The real bug is in due to an object field being accessed before the object is constructed, in breach of C++98 [class.cdtor]:

  For an object of non-POD class type ... before the constructor begins execution ... referring to any non-static member or base class of the object results in undefined behavior

Instead of -fno-guess-branch-probability we should use -fno-lifetime-dse:

  -fno-lifetime-dse

  In C++ the value of an object is only affected by changes within its
  lifetime: when the constructor begins, the object has an indeterminate
  value, and any changes during the lifetime of the object are dead when
  the object is destroyed.  Normally dead store elimination will take
  advantage of this; if your code relies on the value of the object
  storage persisting beyond the lifetime of the object, you can use this
  flag to disable this optimization.

Please try this flag instead.
Comment 48 Severin Gehwolf 2016-02-16 09:44:26 EST
Scratch build with -fno-lifetime-dse instead of -fno-guess-branch-probability:

http://koji.fedoraproject.org/koji/taskinfo?taskID=13009074

FWIW, reproducer works fine for me for a fastdebug hotspot-only build using -fno-lifetime-dse.
Comment 50 Severin Gehwolf 2016-02-16 11:44 EST
Created attachment 1127635 [details]
spec file patch changing flag to -fno-lifetime-dse
Comment 51 Severin Gehwolf 2016-02-16 11:55:19 EST
Task which properly boot-cycled on i686 with -fno-lifetime-dse:
http://koji.fedoraproject.org/koji/taskinfo?taskID=13009566
Comment 54 Jan Kurik 2016-02-24 09:29:30 EST
This bug appears to have been reported against 'rawhide' during the Fedora 24 development cycle.
Changing version to '24'.

More information and reason for this action is here:
https://fedoraproject.org/wiki/Fedora_Program_Management/HouseKeeping/Fedora24#Rawhide_Rebase
Comment 55 Severin Gehwolf 2016-06-22 03:29:52 EDT
There is a work-around for this issue. Use -fno-lifetime-dse and -fno-delete-null-pointer-checks. It's unlikely to get fixed in upstream JDK 8. I'm closing this bug.
Comment 56 Severin Gehwolf 2016-06-22 03:31:29 EDT
(In reply to Severin Gehwolf from comment #55)
> It's unlikely to get fixed in upstream JDK 8.

That is, the underlying issue of undefined behaviour being present in JDK 8 hotspot.

Note You need to log in before you can comment on or make changes to this bug.