Bug 1799087 - java-11-openjdk: [s390x] FTBFS in Fedora rawhide/f32 due to GCC 10
Summary: java-11-openjdk: [s390x] FTBFS in Fedora rawhide/f32 due to GCC 10
Keywords:
Status: CLOSED DUPLICATE of bug 1799408
Alias: None
Product: Fedora
Classification: Fedora
Component: java-11-openjdk
Version: 32
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Severin Gehwolf
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: ZedoraTracker F32FTBFS 1799531
TreeView+ depends on / blocked
 
Reported: 2020-02-06 15:36 UTC by Fedora Release Engineering
Modified: 2020-03-25 19:25 UTC (History)
9 users (show)

Fixed In Version: java-11-openjdk-11.0.7.2-0.3.ea.fc33
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-03-25 19:25:17 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
build.log (32.00 KB, text/plain)
2020-02-06 15:36 UTC, Fedora Release Engineering
no flags Details
root.log (32.00 KB, text/plain)
2020-02-06 15:36 UTC, Fedora Release Engineering
no flags Details
state.log (1.02 KB, text/plain)
2020-02-06 15:36 UTC, Fedora Release Engineering
no flags Details


Links
System ID Private Priority Status Summary Last Updated
IBM Linux Technology Center 184365 0 None None None 2020-03-12 15:07:54 UTC

Description Fedora Release Engineering 2020-02-06 15:36:02 UTC
java-11-openjdk failed to build from source in Fedora rawhide/f32

https://koji.fedoraproject.org/koji/taskinfo?taskID=41213309


For details on the mass rebuild see:

https://fedoraproject.org/wiki/Fedora_32_Mass_Rebuild
Please fix java-11-openjdk at your earliest convenience and set the bug's status to
ASSIGNED when you start fixing it. If the bug remains in NEW state for 8 weeks,
java-11-openjdk will be orphaned. Before branching of Fedora 33,
java-11-openjdk will be retired, if it still fails to build.

For more details on the FTBFS policy, please visit:
https://fedoraproject.org/wiki/Fails_to_build_from_source

Comment 1 Fedora Release Engineering 2020-02-06 15:36:15 UTC
Created attachment 1658259 [details]
build.log

file build.log too big, will only attach last 32768 bytes

Comment 2 Fedora Release Engineering 2020-02-06 15:36:17 UTC
Created attachment 1658260 [details]
root.log

file root.log too big, will only attach last 32768 bytes

Comment 3 Fedora Release Engineering 2020-02-06 15:36:19 UTC
Created attachment 1658261 [details]
state.log

Comment 4 Severin Gehwolf 2020-02-06 20:07:19 UTC
We weren't seeing this issue prior GCC 10 in build roots.

Symptom looks like this:

Exception in thread "main" java.lang.ExceptionInInitializerError
Exception in thread "main" java.lang.ExceptionInInitializerError
	at jdk.jlink/jdk.tools.jmod.JmodTask.getMessage(JmodTask.java:1547)
	at jdk.jlink/jdk.tools.jmod.JmodTask.handleOptions(JmodTask.java:1284)
	at jdk.jlink/jdk.tools.jmod.JmodTask.run(JmodTask.java:191)
	at jdk.jlink/jdk.tools.jmod.Main.main(Main.java:34)
Caused by: java.security.PrivilegedActionException: java.lang.ClassNotFoundException: jdk.tools.jmod.resources.spi.jmodProvider
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/java.util.ResourceBundle.getResourceBundleProviderType(ResourceBundle.java:1913)
	at java.base/java.util.ResourceBundle.getServiceLoader(ResourceBundle.java:1880)
	at java.base/java.util.ResourceBundle$CacheKey.getProviders(ResourceBundle.java:699)
	at java.base/java.util.ResourceBundle$CacheKey.hasProviders(ResourceBundle.java:706)
	at java.base/java.util.ResourceBundle.loadBundle(ResourceBundle.java:1809)
	at java.base/java.util.ResourceBundle.findBundle(ResourceBundle.java:1774)
	at java.base/java.util.ResourceBundle.findBundle(ResourceBundle.java:1728)
	at java.base/java.util.ResourceBundle.findBundle(ResourceBundle.java:1728)
	at java.base/java.util.ResourceBundle.getBundleImpl(ResourceBundle.java:1662)
	at java.base/java.util.ResourceBundle.getBundleImpl(ResourceBundle.java:1582)
	at java.base/java.util.ResourceBundle.getBundleImpl(ResourceBundle.java:1556)
	at java.base/java.util.ResourceBundle.getBundle(ResourceBundle.java:932)
	at jdk.jlink/jdk.tools.jmod.JmodTask$ResourceBundleHelper.<clinit>(JmodTask.java:1559)
	... 4 more
Caused by: java.lang.ClassNotFoundException: jdk.tools.jmod.resources.spi.jmodProvider
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
	at java.base/java.lang.Class.forName0(Native Method)
	at java.base/java.lang.Class.forName(Class.java:398)
	at java.base/java.util.ResourceBundle$3.run(ResourceBundle.java:1918)
	at java.base/java.util.ResourceBundle$3.run(ResourceBundle.java:1914)
	... 18 more
	at jdk.jlink/jdk.tools.jmod.JmodTask.getMessage(JmodTask.java:1547)
	at jdk.jlink/jdk.tools.jmod.JmodTask.handleOptions(JmodTask.java:1284)
	at jdk.jlink/jdk.tools.jmod.JmodTask.run(JmodTask.java:191)
	at jdk.jlink/jdk.tools.jmod.Main.main(Main.java:34)
Caused by: java.security.PrivilegedActionException: java.lang.ClassNotFoundException: jdk.tools.jmod.resources.spi.jmodProvider
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/java.util.ResourceBundle.getResourceBundleProviderType(ResourceBundle.java:1913)
	at java.base/java.util.ResourceBundle.getServiceLoader(ResourceBundle.java:1880)
	at java.base/java.util.ResourceBundle$CacheKey.getProviders(ResourceBundle.java:699)
	at java.base/java.util.ResourceBundle$CacheKey.hasProviders(ResourceBundle.java:706)
	at java.base/java.util.ResourceBundle.loadBundle(ResourceBundle.java:1809)
	at java.base/java.util.ResourceBundle.findBundle(ResourceBundle.java:1774)
	at java.base/java.util.ResourceBundle.findBundle(ResourceBundle.java:1728)
	at java.base/java.util.ResourceBundle.findBundle(ResourceBundle.java:1728)
	at java.base/java.util.ResourceBundle.getBundleImpl(ResourceBundle.java:1662)
	at java.base/java.util.ResourceBundle.getBundleImpl(ResourceBundle.java:1582)
	at java.base/java.util.ResourceBundle.getBundleImpl(ResourceBundle.java:1556)
	at java.base/java.util.ResourceBundle.getBundle(ResourceBundle.java:932)
	at jdk.jlink/jdk.tools.jmod.JmodTask$ResourceBundleHelper.<clinit>(JmodTask.java:1559)
	... 4 more
Caused by: java.lang.ClassNotFoundException: jdk.tools.jmod.resources.spi.jmodProvider
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
	at java.base/java.lang.Class.forName0(Native Method)
	at java.base/java.lang.Class.forName(Class.java:398)
	at java.base/java.util.ResourceBundle$3.run(ResourceBundle.java:1918)
	at java.base/java.util.ResourceBundle$3.run(ResourceBundle.java:1914)
	... 18 more
+ exitcode=1

Comment 5 Ben Cotton 2020-02-11 16:38:08 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 32 development cycle.
Changing version to 32.

Comment 6 Severin Gehwolf 2020-02-12 19:21:53 UTC
slowdebug build (no opt) works.

$ uname -m
s390x
$  ./images/jdk/bin/java -version
openjdk version "11.0.6" 2020-01-14
OpenJDK Runtime Environment 18.9 (slowdebug build 11.0.6+10)
OpenJDK 64-Bit Server VM 18.9 (slowdebug build 11.0.6+10, mixed mode)

Comment 7 Severin Gehwolf 2020-02-13 11:42:22 UTC
Interestingly enough, fastdebug build works too:

$ uname -m
s390x
$ ./build/images/jdk/bin/java -version
openjdk version "11.0.6" 2020-01-14
OpenJDK Runtime Environment 18.9 (fastdebug build 11.0.6+10)
OpenJDK 64-Bit Server VM 18.9 (fastdebug build 11.0.6+10, mixed mode)

Comment 8 Severin Gehwolf 2020-02-13 14:54:09 UTC
What do we know so far?

- slowdebug/fastdebug builds work fine on s390x
- release build fails on first invocation of jmod when
  trying to create interim jmods.
  "jmod --help" is actually sufficient to reproduce.

Using a libjvm.so from a fastdebug build in a partial 'release' build fixes the problem.

Thus, the problem seems to be in libjvm.so of a release build.

Comment 9 Severin Gehwolf 2020-02-17 19:59:03 UTC
Like JDK 8, compiling with these options "fixes" the build:

-fno-tree-coalesce-vars
-fno-tree-copy-prop
-fno-tree-dce
-fno-tree-dominator-opts
-fno-tree-dse
-fno-tree-forwprop
-fno-tree-fre
-fno-tree-loop-distribute-patterns
-fno-tree-loop-distribution
-fno-tree-loop-vectorize
-fno-tree-partial-pre
-fno-tree-phiprop
-fno-tree-pre
-fno-tree-pta
-fno-tree-scev-cprop
-fno-tree-sink
-fno-tree-slp-vectorize
-fno-tree-slsr
-fno-tree-sra
-fno-tree-switch-conversion
-fno-tree-tail-merge
-fno-tree-ter
-fno-tree-vrp
-fno-unit-at-a-time
-fno-unswitch-loops
-fno-vect-cost-model
-fno-version-loops-for-strides

It also seems that the issue is in the C1 compiler. -Xint works, -XX:-TieredCompilation too.

One of the objects getting compiled badly is c1_Canonicalizer.o (it's not the only one, though).

Determined with:
https://github.com/jerboaa/hotspot-tools-find-bad-object/commit/08e9c374ecf6629fb944c1840d37d4879d50c994

Comment 10 Severin Gehwolf 2020-02-26 18:17:19 UTC
PR to get java-11-openjdk building again. This is really ugly and I don't like it :-/ Anyway:

https://src.fedoraproject.org/rpms/java-11-openjdk/pull-request/68

Comment 11 Severin Gehwolf 2020-02-27 13:54:42 UTC
Built in rawhide, but I'll keep this bug open for root-cause analysis.

https://koji.fedoraproject.org/koji/buildinfo?buildID=1471503

Comment 12 Dan Horák 2020-03-12 14:39:25 UTC
Do you need any help from the IBM s390x experts?

Comment 13 Severin Gehwolf 2020-03-12 14:57:17 UTC
(In reply to Dan Horák from comment #12)
> Do you need any help from the IBM s390x experts?

Sure! Would love to figure out where things go wrong. Whether it's GCC 10 or some hotspot UB. All I know is in the comments of this bug. We currently work around by disabling a bunch of optimizations, which isn't great.

Comment 14 IBM Bug Proxy 2020-03-12 15:32:41 UTC
------- Comment From Andreas.Krebbel.com 2020-03-12 11:20 EDT-------
Jakub recently fixed a nasty bug in combine which triggered miscompiles on S/390 (e.g. git). Was that patch already part of your GCC when you tried to build the package?

------- Comment From Andreas.Krebbel.com 2020-03-12 11:28 EDT-------
This was the patch from Jakub:

commit 73dc4ae47418aef2eb470b8f71cef57dce37349e
Author: Jakub Jelinek <jakub>
Date:   Tue Feb 25 13:56:47 2020 +0100

combine: Fix find_split_point handling of constant store into ZERO_EXTRACT [PR93908]

Comment 15 Severin Gehwolf 2020-03-12 17:02:55 UTC
(In reply to IBM Bug Proxy from comment #14)
> ------- Comment From Andreas.Krebbel.com 2020-03-12 11:20 EDT-------
> Jakub recently fixed a nasty bug in combine which triggered miscompiles on
> S/390 (e.g. git). Was that patch already part of your GCC when you tried to
> build the package?

Thanks! When I looked at the gcc builds today in koji this bugfix caught my eye. Unfortunately, our builds were done before that fix went in. I'll try again today without the disabled OPT flags on s390x and see how far we get. Will report back here once I have that going.

Comment 16 Severin Gehwolf 2020-03-13 17:34:44 UTC
Here is a scratch build with the GCC flags for s390x reverted. Took longer than expected since make 4.3 landed since in rawhide which needed another patch.

https://koji.fedoraproject.org/koji/taskinfo?taskID=42450266

I'm seeing this in root.log for s390x which is the GCC version which should have the fix above:

DEBUG util.py:598:   libgcc                     s390x  10.0.1-0.9.fc33                  build  72 k

We'll see how that goes.

Built from this branch:
https://src.fedoraproject.org/fork/jerboaa/rpms/java-11-openjdk/commits/gcc_10_fixes_new

Comment 17 Severin Gehwolf 2020-03-24 13:06:56 UTC
(In reply to Severin Gehwolf from comment #16)
> Here is a scratch build with the GCC flags for s390x reverted. Took longer
> than expected since make 4.3 landed since in rawhide which needed another
> patch.
> 
> https://koji.fedoraproject.org/koji/taskinfo?taskID=42450266

OK that one passed. It looks like this is a duplicate of bug 1799408. I'll prepare a pull request removing the workaround.

Comment 19 Severin Gehwolf 2020-03-25 19:25:17 UTC
Fixed in rawhide. Marking as duplicate of 1799408 as the actual fix was a GCC 10 bugfix.

*** This bug has been marked as a duplicate of bug 1799408 ***


Note You need to log in before you can comment on or make changes to this bug.