Bug 1600426

Summary: Reconsider auto-requires: javapackages-tools
Product: [Fedora] Fedora Reporter: Severin Gehwolf <sgehwolf>
Component: javapackages-toolsAssignee: Mikolaj Izdebski <mizdebsk>
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: rawhideCC: ctubbsii, java-sig-commits, mizdebsk, msrb, sgehwolf, sochotni
Target Milestone: ---Keywords: FutureFeature, Reopened
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: javapackages-tools-5.2.0-6.fc29 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-04-09 18:47:12 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Severin Gehwolf 2018-07-12 08:29:38 UTC
Description of problem:
In rawhide when I build a Java package I get a run-time requirement of javapackakges-tools whether I want that or not. This wouldn't concern me as much if javapackages-tools wouldn't drag in java-1.8.0-openjdk-headless. So for a java package that runs fine with, say JDK 10, I'd get JDK 8 too, due to javapackages-tools auto-requires.

Example from the byteman build:
-------------------------------
[INFO osgi.prov] input: ['/builddir/build/BUILDROOT/byteman-4.0.4-1.module_1916+32d410e4.noarch/usr/share/byteman/contrib/jboss-modules-system/byteman-jboss-modules-plugin.jar']
[INFO osgi.req] input: ['/builddir/build/BUILDROOT/byteman-4.0.4-1.module_1916+32d410e4.noarch/usr/share/byteman/contrib/jboss-modules-system/byteman-jboss-modules-plugin.jar']
[INFO osgi.prov] input: ['/builddir/build/BUILDROOT/byteman-4.0.4-1.module_1916+32d410e4.noarch/usr/share/byteman/lib/byteman-install.jar']
[INFO osgi.req] input: ['/builddir/build/BUILDROOT/byteman-4.0.4-1.module_1916+32d410e4.noarch/usr/share/byteman/lib/byteman-install.jar']
[INFO osgi.prov] input: ['/builddir/build/BUILDROOT/byteman-4.0.4-1.module_1916+32d410e4.noarch/usr/share/byteman/lib/byteman-sample.jar']
[INFO osgi.req] input: ['/builddir/build/BUILDROOT/byteman-4.0.4-1.module_1916+32d410e4.noarch/usr/share/byteman/lib/byteman-sample.jar']
[INFO osgi.prov] input: ['/builddir/build/BUILDROOT/byteman-4.0.4-1.module_1916+32d410e4.noarch/usr/share/byteman/lib/byteman-submit.jar']
[INFO osgi.req] input: ['/builddir/build/BUILDROOT/byteman-4.0.4-1.module_1916+32d410e4.noarch/usr/share/byteman/lib/byteman-submit.jar']
[INFO osgi.prov] input: ['/builddir/build/BUILDROOT/byteman-4.0.4-1.module_1916+32d410e4.noarch/usr/share/byteman/lib/byteman.jar']
[INFO osgi.req] input: ['/builddir/build/BUILDROOT/byteman-4.0.4-1.module_1916+32d410e4.noarch/usr/share/byteman/lib/byteman.jar']
[INFO osgi.prov] input: ['/builddir/build/BUILDROOT/byteman-4.0.4-1.module_1916+32d410e4.noarch/usr/share/java/byteman/byteman-agent.jar']
[INFO osgi.req] input: ['/builddir/build/BUILDROOT/byteman-4.0.4-1.module_1916+32d410e4.noarch/usr/share/java/byteman/byteman-agent.jar']
[INFO osgi.prov] input: ['/builddir/build/BUILDROOT/byteman-4.0.4-1.module_1916+32d410e4.noarch/usr/share/java/byteman/byteman-install.jar']
[INFO osgi.prov] osgi(org.jboss.byteman.agent.install) = 4.0.4
[INFO osgi.req] input: ['/builddir/build/BUILDROOT/byteman-4.0.4-1.module_1916+32d410e4.noarch/usr/share/java/byteman/byteman-install.jar']
[INFO osgi.prov] input: ['/builddir/build/BUILDROOT/byteman-4.0.4-1.module_1916+32d410e4.noarch/usr/share/java/byteman/byteman-jboss-modules-plugin.jar']
[INFO osgi.req] input: ['/builddir/build/BUILDROOT/byteman-4.0.4-1.module_1916+32d410e4.noarch/usr/share/java/byteman/byteman-jboss-modules-plugin.jar']
[INFO osgi.prov] input: ['/builddir/build/BUILDROOT/byteman-4.0.4-1.module_1916+32d410e4.noarch/usr/share/java/byteman/byteman-jigsaw.jar']
[INFO osgi.req] input: ['/builddir/build/BUILDROOT/byteman-4.0.4-1.module_1916+32d410e4.noarch/usr/share/java/byteman/byteman-jigsaw.jar']
[INFO osgi.prov] input: ['/builddir/build/BUILDROOT/byteman-4.0.4-1.module_1916+32d410e4.noarch/usr/share/java/byteman/byteman-layer.jar']
[INFO osgi.req] input: ['/builddir/build/BUILDROOT/byteman-4.0.4-1.module_1916+32d410e4.noarch/usr/share/java/byteman/byteman-layer.jar']
[INFO osgi.prov] input: ['/builddir/build/BUILDROOT/byteman-4.0.4-1.module_1916+32d410e4.noarch/usr/share/java/byteman/byteman-sample.jar']
[INFO osgi.req] input: ['/builddir/build/BUILDROOT/byteman-4.0.4-1.module_1916+32d410e4.noarch/usr/share/java/byteman/byteman-sample.jar']
[INFO osgi.prov] input: ['/builddir/build/BUILDROOT/byteman-4.0.4-1.module_1916+32d410e4.noarch/usr/share/java/byteman/byteman-submit.jar']
[INFO osgi.prov] osgi(org.jboss.byteman.agent.submit) = 4.0.4
[INFO osgi.req] input: ['/builddir/build/BUILDROOT/byteman-4.0.4-1.module_1916+32d410e4.noarch/usr/share/java/byteman/byteman-submit.jar']
[INFO osgi.prov] input: ['/builddir/build/BUILDROOT/byteman-4.0.4-1.module_1916+32d410e4.noarch/usr/share/java/byteman/byteman.jar']
[INFO osgi.req] input: ['/builddir/build/BUILDROOT/byteman-4.0.4-1.module_1916+32d410e4.noarch/usr/share/java/byteman/byteman.jar']
[INFO maven.prov] input: ['/builddir/build/BUILDROOT/byteman-4.0.4-1.module_1916+32d410e4.noarch/usr/share/maven-metadata/byteman.xml']
[INFO maven.prov] mvn(org.jboss.byteman:byteman-jigsaw:pom:) = 4.0.4, mvn(org.jboss.byteman:byteman-submit:pom:) = 4.0.4, mvn(org.jboss.byteman:byteman-jboss-modules-plugin) = 4.0.4, mvn(org.jboss.byteman:byteman-submit) = 4.0.4, mvn(org.jboss.byteman:byteman) = 4.0.4, mvn(org.jboss.byteman:byteman-install:pom:) = 4.0.4, mvn(org.jboss.byteman:byteman-jboss-modules:pom:) = 4.0.4, mvn(org.jboss.byteman:tests:pom:) = 4.0.4, mvn(org.jboss.byteman:byteman:pom:) = 4.0.4, mvn(org.jboss.byteman:byteman-jboss-modules-plugin:pom:) = 4.0.4, mvn(org.jboss.byteman:byteman-jigsaw) = 4.0.4, mvn(org.jboss.byteman:byteman-install) = 4.0.4, mvn(org.jboss.byteman:byteman-layer:pom:) = 4.0.4, mvn(org.jboss.byteman:byteman-sample) = 4.0.4, mvn(org.jboss.byteman:byteman-sample:pom:) = 4.0.4, mvn(org.jboss.byteman:byteman-root:pom:) = 4.0.4, mvn(org.jboss.byteman:byteman-agent) = 4.0.4, mvn(org.jboss.byteman:byteman-layer) = 4.0.4, mvn(org.jboss.byteman:byteman-agent:pom:) = 4.0.4
[INFO maven.req] input: ['/builddir/build/BUILDROOT/byteman-4.0.4-1.module_1916+32d410e4.noarch/usr/share/maven-metadata/byteman.xml']
[INFO maven.req] javapackages-tools, mvn(org.ow2.asm:asm-commons), mvn(org.ow2.asm:asm-tree), mvn(org.ow2.asm:asm-analysis), java-headless >= 1:1.9, mvn(org.ow2.asm:asm), mvn(java_cup:java_cup)
Provides: bundled(java_cup) = 1:0.11b-8 bundled(objectweb-asm) = 6.2 byteman = 4.0.4-1.module_1916+32d410e4 mvn(org.jboss.byteman:byteman) = 4.0.4 mvn(org.jboss.byteman:byteman-agent) = 4.0.4 mvn(org.jboss.byteman:byteman-agent:pom:) = 4.0.4 mvn(org.jboss.byteman:byteman-install) = 4.0.4 mvn(org.jboss.byteman:byteman-install:pom:) = 4.0.4 mvn(org.jboss.byteman:byteman-jboss-modules-plugin) = 4.0.4 mvn(org.jboss.byteman:byteman-jboss-modules-plugin:pom:) = 4.0.4 mvn(org.jboss.byteman:byteman-jboss-modules:pom:) = 4.0.4 mvn(org.jboss.byteman:byteman-jigsaw) = 4.0.4 mvn(org.jboss.byteman:byteman-jigsaw:pom:) = 4.0.4 mvn(org.jboss.byteman:byteman-layer) = 4.0.4 mvn(org.jboss.byteman:byteman-layer:pom:) = 4.0.4 mvn(org.jboss.byteman:byteman-root:pom:) = 4.0.4 mvn(org.jboss.byteman:byteman-sample) = 4.0.4 mvn(org.jboss.byteman:byteman-sample:pom:) = 4.0.4 mvn(org.jboss.byteman:byteman-submit) = 4.0.4 mvn(org.jboss.byteman:byteman-submit:pom:) = 4.0.4 mvn(org.jboss.byteman:byteman:pom:) = 4.0.4 mvn(org.jboss.byteman:tests:pom:) = 4.0.4 osgi(org.jboss.byteman.agent.install) = 4.0.4 osgi(org.jboss.byteman.agent.submit) = 4.0.4
Requires(rpmlib): rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(FileDigests) <= 4.6.0-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1
Requires: /bin/bash /bin/sh javapackages-tools
-------------------------------

The byteman package doesn't have any explicit 'Requires: javapackages-tools' in its spec[1].


What is the reason for auto-requires of javapackages-tools? If it's the directory structure this should get switched to javapackages-filesystem instead which won't have a dep on the JDK.

Please re-assign component as necessary.

[1] https://src.fedoraproject.org/rpms/byteman/blob/byteman/f/byteman.spec

Comment 1 Mikolaj Izdebski 2018-07-12 16:15:03 UTC
(In reply to Severin Gehwolf from comment #0)
> What is the reason for auto-requires of javapackages-tools? If it's the
> directory structure this should get switched to javapackages-filesystem
> instead which won't have a dep on the JDK.

The reason is that this requirement is mandated by Java packaging guidelines [1].
I'm not going to change generator unless guidelines are updated first. The process to change guidelines is described at [2].

[1] https://fedoraproject.org/wiki/Packaging:Java#BuildRequires_and_Requires
[2] https://fedoraproject.org/wiki/Packaging_Committee#Guideline_Change_Procedure

Comment 2 Severin Gehwolf 2018-07-12 16:35:39 UTC
(In reply to Mikolaj Izdebski from comment #1)
> (In reply to Severin Gehwolf from comment #0)
> > What is the reason for auto-requires of javapackages-tools? If it's the
> > directory structure this should get switched to javapackages-filesystem
> > instead which won't have a dep on the JDK.
> 
> The reason is that this requirement is mandated by Java packaging guidelines
> [1].

[...]

> [1] https://fedoraproject.org/wiki/Packaging:Java#BuildRequires_and_Requires

That doesn't really answer the question why it's there in the first place. There must have been a reason as to why a runtime requirement on javapackages-tools is needed. I'd have thought a requirement on some java-headless >= X to be sufficient plus dir-ownership, which javapackages-filesystem would do.

Would you happen to remember why it was added? If we know that it gives us better grounds to change the guidelines (if needed). Thanks!

Comment 3 Severin Gehwolf 2018-07-12 16:44:07 UTC
https://pagure.io/packaging-committee/issue/260#comment-145632 seems to suggest it's for dir ownership.

Comment 4 Mikolaj Izdebski 2018-07-12 17:18:15 UTC
Why it's in the guidelines would be question for packaging list or FPC, not javapackages-tools maintainers. But I can try to answer it best I can.

From what I know JPackage project had jpackage-utils, which was a single package containing various Java-related utilities. It also owned Java-related directories. There was practice (not documented in "JPackage Policy") for all packages to require jpackage-utils. For directory ownership, and more - packages could import shell functions from JPackage library, run utility scripts to locate JVM, find extension JARs and so on.

Fedora Java packaging was inherited from JPackage. Java Packaging guidelines was based on JPackage policy and practices. Over time it was simplified to better match rest of the distribution, diverging from JPackage. At some point auto-requires generators were implemented and we started generating requires on jpackage-utils so that explicit Requires wouldn't have to be listed in spec files by packagers manually.

Later a single FPC member decided to change guidelines and replace jpackage-utils with javapackages-tools - without FPC vote, without contacting Java SIG or javapakcages-tools maintainers. I didn't see this change announced either. A few months later, when I noticed it, I updated javapackages to follow guidelines.

When javapackages-filesystem was introduced I didn't switch generators to it because I would either need to violate packaging guidelines, or engage with FPC to change it, for which I don't have time and energy.

I consider this bug closed, you can reopen it once guidelines is changed and I will obey FPC decision.

Comment 5 Severin Gehwolf 2018-07-13 12:38:12 UTC
https://pagure.io/packaging-committee/issue/781

Comment 6 Mat Booth 2018-07-13 14:27:04 UTC
(In reply to Mikolaj Izdebski from comment #4)
> Later a single FPC member decided to change guidelines and replace
> jpackage-utils with javapackages-tools - without FPC vote, without
> contacting Java SIG or javapakcages-tools maintainers. I didn't see this
> change announced either. A few months later, when I noticed it, I updated
> javapackages to follow guidelines.
> 

Decision wasn't made in a vacuum, someone asked the committee to change the guidelines: https://pagure.io/packaging-committee/issue/521

Looks like Tibbs thought he was updating guidelines to reflect what was actually happening in real life (factual corrections and typos are not normally put to the vote.)

Comment 7 Severin Gehwolf 2018-07-25 08:05:44 UTC
(In reply to Mikolaj Izdebski from comment #1)
> (In reply to Severin Gehwolf from comment #0)
> > What is the reason for auto-requires of javapackages-tools? If it's the
> > directory structure this should get switched to javapackages-filesystem
> > instead which won't have a dep on the JDK.
> 
> The reason is that this requirement is mandated by Java packaging guidelines
> [1]. I'm not going to change generator unless guidelines are updated first.

Java packaging guidelines have been updated:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/HP4LRC2RJNMOMXITZZYJ6FLF2RITY6H6/

Comment 8 Mikolaj Izdebski 2018-07-27 15:37:39 UTC
Severin, IIRC you had a list of packages that were importing shell functions. Could you add explicit requires on javapackages-tools to them before generators are updated?

Comment 9 Severin Gehwolf 2018-07-30 07:56:27 UTC
(In reply to Mikolaj Izdebski from comment #8)
> Severin, IIRC you had a list of packages that were importing shell
> functions.

Yes. See:
https://lists.fedoraproject.org/archives/list/java-devel@lists.fedoraproject.org/message/WVZOI7I27SAOUVF2SGCRLGVPEFFHXGWP/

> Could you add explicit requires on javapackages-tools to them
> before generators are updated?

Sure, I'll put that on the TODO. I'll add that no later than F30 branch point. Are we still going with a system wide change for this?

Comment 10 Severin Gehwolf 2018-07-30 07:57:38 UTC
Adding need-info on myself so I won't forget.

Comment 11 Mikolaj Izdebski 2018-07-30 10:07:03 UTC
(In reply to Severin Gehwolf from comment #9)
> Are we still going with a system wide change for this?

The main purpose of the change process is to open discussion and gather feedback about the change. But since the decision to implement this change has already been made (by FPC which has FESCo mandate for these stuff), the change process would be pointless at this point.

Comment 12 Mikolaj Izdebski 2018-08-02 09:03:06 UTC
Generators were updated in rawhide.

Upstream change: https://github.com/fedora-java/javapackages/pull/63

Comment 13 Severin Gehwolf 2018-08-31 13:00:18 UTC
(In reply to Severin Gehwolf from comment #9)
> (In reply to Mikolaj Izdebski from comment #8)
> > Severin, IIRC you had a list of packages that were importing shell
> > functions.
> 
> Yes. See:
> https://lists.fedoraproject.org/archives/list/java-devel@lists.fedoraproject.
> org/message/WVZOI7I27SAOUVF2SGCRLGVPEFFHXGWP/
> 
> > Could you add explicit requires on javapackages-tools to them
> > before generators are updated?
> 
> Sure, I'll put that on the TODO. I'll add that no later than F30 branch
> point. Are we still going with a system wide change for this?

I've had a look at these 82 SRPMs as determined by earlier analysis. Some of them have explicit requirements on jpackage-utils, provided by javapackages-tools, so were a no-op. For the ones for which I've determined that they now need javapackages-tools I've added the requirement and have rebuilt them.

The list was:

389-console
accumulo
antlr
antlr3
antlr4
antlrworks
apache-rat
aqute-bnd
batik
bolzplatz2006
bsh
bundling-detection-java
CardManager
checkstyle
clapham
closure-compiler
console-image-viewer
derby
ditaa
eclipse
fop
freecol
freerouting
gherkin2-java
gogui
gradle
HdrHistogram
itext
jarjar
javacc
java-deptools
javadocofflinesearch
javahelp2
java-wakeonlan
jdiff
jenkins
jetty
jflex
jing-trang
jmol
josm
jpanoramamaker
jsonld-java-tools
jtidy
junit5
legendsbrowser
liquibase
log4j
logisim
Mars
maven
metadata-extractor2
modello
msv
mvel
nekohtml
neurord
objectweb-asm3
objectweb-asm
ohc
OpenStego
plantuml
portecle
proguard
rachota
rhino
scala
shiro
sunflow
svnkit
tika
tomcat
tonto
weka
will-crash
woden
xerces-j2
xjparse
xml-commons-resolver
xmvn
yuicompressor
zanata-platform

I believe we can close this bug with resolution RAWHIDE.