Bug 1312019

Summary: java-1.8.0-openjdk-headless must provide /usr/bin/jjs
Product: Red Hat Enterprise Linux 7 Reporter: Andrey Loskutov <loskutov>
Component: java-1.8.0-openjdkAssignee: jiri vanek <jvanek>
Status: CLOSED ERRATA QA Contact: Lukáš Zachar <lzachar>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.0CC: dbhole, jvanek, lzachar, srandhaw
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-11-03 22:56:19 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1312357, 1317597, 1371146    
Bug Blocks:    

Description Andrey Loskutov 2016-02-25 14:27:13 UTC
Description of problem:

rpm containing an executable file with shebang #!/usr/bin/jjs cannot be installed even if /usr/bin/jjs exists (installed by java-1.8.0-openjdk-headless).

The problem is that java-1.8.0-openjdk-headless does not specify that it provides /usr/bin/jjs executable.

Steps to reproduce: create spec file as proposed below, build the rpm and try to install it.

Example hello.spec file to reproduce the problem:

#############################################################
Summary: Creates shell javascript file printing hello world
Name: hello_world
Version: 1.0.0
Release: 1
License: public domain
BuildRoot: /var/tmp/%{name}-buildroot

%define openjdk_min_version 1:1.8.0.71
requires: java-1.8.0-openjdk >= %{openjdk_min_version}
requires: java-1.8.0-openjdk-headless >= %{openjdk_min_version}

%description
Shell javascript file printing hello world

%build
mkdir -pv %{buildroot}/tmp/hello_world/
echo -e "#!/usr/bin/jjs\nprint(\"Hello world\")" > %{buildroot}/tmp/hello_world/script.js
chmod u+x %{buildroot}/tmp/hello_world/script.js

%post
 echo "this is the post-install sh commans"

%files

/tmp/hello_world/script.js

%preun
 echo "the pre-uninstall sh commands"

%postun
 echo "the post-uninstall sh commands"

##############################################################

The spec file allows to build an rpm:

rpm -ba hello_world-1.0.spec

rpmbuild -ba hello.spec
Executing(%build): /bin/sh -e /var/tmp/rpm-tmp.wFvWIH
+ umask 022
+ cd /home/aloskuto/rpmbuild/BUILD
+ mkdir -pv /home/aloskuto/rpmbuild/BUILDROOT/hello_world-1.0.0-1.x86_64/tmp/hello_world/
mkdir: created directory '/home/aloskuto/rpmbuild/BUILDROOT/hello_world-1.0.0-1.x86_64'
mkdir: created directory '/home/aloskuto/rpmbuild/BUILDROOT/hello_world-1.0.0-1.x86_64/tmp'
mkdir: created directory '/home/aloskuto/rpmbuild/BUILDROOT/hello_world-1.0.0-1.x86_64/tmp/hello_world/'
+ echo -e '#!/usr/bin/jjs\nprint("Hello world")'
+ chmod u+x /home/aloskuto/rpmbuild/BUILDROOT/hello_world-1.0.0-1.x86_64/tmp/hello_world/script.js
+ exit 0
Processing files: hello_world-1.0.0-1.x86_64
Provides: hello_world = 1.0.0-1 hello_world(x86-64) = 1.0.0-1
Requires(interp): /bin/sh /bin/sh /bin/sh
Requires(rpmlib): rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(FileDigests) <= 4.6.0-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1
Requires(post): /bin/sh
Requires(preun): /bin/sh
Requires(postun): /bin/sh
Requires: /usr/bin/jjs
Checking for unpackaged file(s): /usr/lib/rpm/check-files /home/aloskuto/rpmbuild/BUILDROOT/hello_world-1.0.0-1.x86_64
Wrote: /home/aloskuto/rpmbuild/SRPMS/hello_world-1.0.0-1.src.rpm
Wrote: /home/aloskuto/rpmbuild/RPMS/x86_64/hello_world-1.0.0-1.x86_64.rpm
Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.ac0ntq
+ umask 022
+ cd /home/aloskuto/rpmbuild/BUILD
+ /usr/bin/rm -rf /home/aloskuto/rpmbuild/BUILDROOT/hello_world-1.0.0-1.x86_64
+ exit 0

Please note the line "Requires: /usr/bin/jjs" above.

BUT installing this rpm fails:

sudo yum install /home/aloskuto/rpmbuild/RPMS/x86_64/hello_world-1.0.0-1.x86_64.rpm
[sudo] password for aloskuto: 
Loaded plugins: langpacks
Examining /home/aloskuto/rpmbuild/RPMS/x86_64/hello_world-1.0.0-1.x86_64.rpm: hello_world-1.0.0-1.x86_64
Marking /home/aloskuto/rpmbuild/RPMS/x86_64/hello_world-1.0.0-1.x86_64.rpm to be installed
Resolving Dependencies
--> Running transaction check
---> Package hello_world.x86_64 0:1.0.0-1 will be installed
--> Processing Dependency: /usr/bin/jjs for package: hello_world-1.0.0-1.x86_64
RHEL7-ATE_Workstation_tools                                                                                                                               | 1.5 kB  00:00:00     
RHEL7-ATE_Workstation_tools/primary                                                                                                                       |  25 kB  00:00:00     
RHEL7-ATE_Workstation_tools                                                                                                                                              118/118
--> Processing Dependency: /usr/bin/jjs for package: hello_world-1.0.0-1.x86_64
--> Finished Dependency Resolution
Error: Package: hello_world-1.0.0-1.x86_64 (/hello_world-1.0.0-1.x86_64)
           Requires: /usr/bin/jjs
 You could try using --skip-broken to work around the problem
 You could try running: rpm -Va --nofiles --nodigest

Comment 2 jiri vanek 2016-02-25 15:32:55 UTC
This is quite bad.

Java packages do not provide the /usr/bin/javastuff automatically,  because those are managed by alternatives.
If we have to provide each /usr/bin manually, then specfiel will be even more messy then it is.

IIUC then very simple workaround is  to add Requires: java-headless
and you are done.

Unless I missed something, this is real candidate for close-cantfix

Comment 3 jiri vanek 2016-02-25 15:35:26 UTC
ok. I see a bit lower that this workaround do not works for you. Isnt it just non-capital r in requires of yours?  Its quite weird. Isnt it some change in dnf/rpm which caused this?

Comment 4 Andrey Loskutov 2016-02-26 09:41:19 UTC
The fun is that the spec file does not require /usr/bin/jjs *anywhere*. This requirement is computed by rpm tooling "on the fly", while analyzing the *content* of the files placed into the rpm. Somewhere in rpm tooling someone also checks the shebang lines on exacutable files placed into the rpms and adds all executable paths from there to the "requirements".

Now since java-1.8.0-openjdk-headless provides the jjs executable *and* enables the possibility to write shell scripts in javascript by using the magic "#!/usr/bin/jjs" line in the script, and the example rpm properly adds the dependency to java-1.8.0-openjdk-headless, the only thing is missing is that java-1.8.0-openjdk-headless to declare that it is responsible for providing jjs exacutable.

Comment 5 jiri vanek 2016-02-26 12:32:04 UTC
Hm. this may be not be enough.

As I told, jjs is controled by alternatives.

So lets say you have installed jdk, which does NOT have jjs binary. And you have it set manually, that it is your preferred JDK. Now you install yours package, who requires java-1.8.0-openjdk-headless, and it requires (automagically) /usr/bin/jjs.  So your package will beinstalled (in same transaction) right after java-1.8.0-openjdk-headless. BUT when java-1.8.0-openjdk-headless was insatlled, it did NOT set alternatives to itself, because you set already installed jdk manualy. So the execution of your package will fail anyway. But at least transaction pass.

In jdk9, wia jshell and project kulla, you can do #!/usr/bin/java (or something similar) So I'm afraid this *bug* will bite in more serious shape soon.

Comment 6 jiri vanek 2016-02-26 12:52:06 UTC
Anyway probably best thing todo there is to ass provides on those alternatives-driven stuff - especually on those who might be used in shebangs
so java, jss maybe javac and maybe few more other...

Comment 7 jiri vanek 2016-02-26 13:23:12 UTC
I was recommended this reading: https://fedoraproject.org/wiki/Packaging:Alternatives

and it sugest %ghost. That may be much better solution.

Comment 8 jiri vanek 2016-02-26 13:45:51 UTC
hm. Ghost, as recommended approach may have flaw:

when isntalling two jdks which both have jjs as alternatives target, but one have ghost and second have not:

2 -  %pre of new package 
  - in this stage alternatives --install of new jdk happens
  - it may, but may not, set new jdk as currently set (priorities etc)
10 - (removal of old package) 
   - ghosted file is erased (unless some other package owns it, but that is not this cornercase)
11 - %postun of old package 
   - there the alternatives --remove is called.  If old package remained as target, then its ok, new package is selected, and remvoed target restored.
     - but when new package was set in 2(and that is more likely) then we are screwed,  as nothing (imho!) happens from alternatives themselves and so new package is kept selected, but removed ghosted file remians removed.

Comment 9 jiri vanek 2016-02-29 12:10:47 UTC
Currently from all my views, the only way is to provide
/usr/lib/jvm/java/jre-1.8.0/bin/jjs
and later
/usr/lib/jvm/java/jre-_1.9.0/bin/jjs
/usr/lib/jvm/java/jre-1.9.0/bin/jshell

This enforces usage of concrete version of jdk, which is correct, as the binaries are not guaranteed to be in al version of jre.

And so yours shenabg will end to require those.

Comment 10 Andrey Loskutov 2016-02-29 14:02:46 UTC
(In reply to jiri vanek from comment #9)
> And so yours shenabg will end to require those.

OK, thanks for the research... So basically I have to change the scripts to #!/usr/lib/jvm/java-1.8.0/bin/jjs and would have to touch each script once we migrate to jdk9 or have to use #!/usr/bin/env jjs, which requires jjs to be in PATH.

That's kind of unexpected, since the Oracle Java docs [1] says "You can specify the direct path to the jjs tool in the shebang, but it is a good practice to create a symbolic link in the /usr/bin directory".

Just wondering why executables provided by JDK are so exceptional and cannot work nicely with rpm/RHEL...

Question: is this a last word now, or are you are still investigating? 

In any case if the bug is going to be closed without a fix, can we *at least* update the documentation (at [1]) to include a warning that not all Linux flavours do properly support #!/usr/bin/jjs shebang line and that any software package distributed by rpm should *not* use that particular shebang style.

[1] http://docs.oracle.com/javase/8/docs/technotes/guides/scripting/nashorn/shell.html#CHDEGHJJ

Comment 11 jiri vanek 2016-02-29 14:16:11 UTC
Hello! Tahts definitely not last word. It deserves to be fixed but currently I don't know how to proceed in best way. c#9 looks like smallest evil, but still a bit evil (as you yourselves are suggesting in the links and citations)

the resolution will be placed there in clear way. Hoepfully with fix to packages.

Comment 12 Andrey Loskutov 2016-02-29 14:26:31 UTC
OK, thanks for clarification. Unfortunately I'm not rpm/Linux guru so I can't help you much with the right fix.

Comment 13 jiri vanek 2016-03-08 13:45:55 UTC
Hm. What do you think about much more general approach - replace the shebangs by /usr/bin/env jjs ? The same approach alter for jshell and jdk9.

Comment 14 Andrey Loskutov 2016-03-11 14:27:32 UTC
(In reply to jiri vanek from comment #13)
> Hm. What do you think about much more general approach - replace the
> shebangs by /usr/bin/env jjs ? The same approach alter for jshell and jdk9.

#!/usr/bin/env jjs requires jjs to be in PATH, which limits the general usage of this pattern, since most users do not have jjs in the PATH.

Comment 15 jiri vanek 2016-03-14 13:06:48 UTC
(In reply to Andrey Loskutov from comment #14)
> (In reply to jiri vanek from comment #13)
> > Hm. What do you think about much more general approach - replace the
> > shebangs by /usr/bin/env jjs ? The same approach alter for jshell and jdk9.
> 
> #!/usr/bin/env jjs requires jjs to be in PATH, which limits the general
> usage of this pattern, since most users do not have jjs in the PATH.

Wait. How so? /user/bin is usually on path. Isn't it?

Comment 16 Andrey Loskutov 2016-03-14 13:36:57 UTC
(In reply to jiri vanek from comment #15)
> > #!/usr/bin/env jjs requires jjs to be in PATH, which limits the general
> > usage of this pattern, since most users do not have jjs in the PATH.
> 
> Wait. How so? /user/bin is usually on path. Isn't it?

Sure, this should always work as long as jjs is installed (linked) to /user/bin.

Comment 17 jiri vanek 2016-06-09 13:34:46 UTC
Just heads up. This is still discussed how to approach this best.

Comment 18 jiri vanek 2016-06-23 11:39:29 UTC
Lukas, Deepak, I think the solution proposed in https://bugzilla.redhat.com/show_bug.cgi?id=1317597 are good, and once it is in, I will add provides /usr/bin/jjs to the ojdk8 package.

Comment 19 jiri vanek 2016-06-23 11:40:21 UTC
If you agree, may you ack?

Comment 20 Lukáš Zachar 2016-06-23 11:59:07 UTC
I agree, new behaviour in alternatives looks like the best solution.

Comment 22 Deepak Bhole 2016-06-23 13:03:41 UTC
I am actually sort of opposed to this and to 1317597 now.

If the master does not provide a corresponding link, it is because that JDK does not have the tool. What 1317597 is doing is mixing and matching tools. Imagine you cycle between 3 different jdks that have differing tools; you now have an odd combination of tools coming from various JDKs.

It is for this reason that critical items like the plugin were moved into their own master link, so that JDKs that didn't come with a plug-in won't break applets.

If a JDK does not provide a tool, it should not be our job to link to another one; the user should manually use the full path in that case IMO.

Comment 23 jiri vanek 2016-06-23 13:23:05 UTC
But the tool  will be working regardless.
Even if you set java and javac to java-1.7.0-* the /uss/bin/jjs would work properly, using its jdk8.  So from this point there is no  reason to not to do this.

Comment 24 jiri vanek 2016-06-23 14:19:28 UTC
In adition - is there any other way, how we can provide jjs or jshell in rhel7 for build root?

Imho not. Consider those cases:
If we do not provide /usr/bin/jjs all following builds fails


1) package BuildRequires: java
and  have somewhere inside # /usr/bin/jjs

Then jdk8 is taken to build root.  That may be ok


2) package BuildRequires: java-1.7.0-{openjdk,whatever}
and  have somewhere inside # /usr/bin/jjs

Then this will drag both jdk8 and jdk7 tobuildroot
 - However jdk8 have bigger priority, so after the install jdk8 is selected
 - very hidden failures may appear, as different jdk is used (8 instead of 7)
   - I dot have workaround for this, but generally it can be ok, in pre or during build alternatives maybe swithced to 7
   - Imidiately after this /usr/bin/jjs  is lost. (so we should not provide it)
   - workaround  may be to sed all jjs schebangs - to some /usr/lib/jvm/java-1.8.0-openjdk/jre/bin/jjs but you will agree that it is really inconvenient.  
   - or, with same inconvenience, to patch build and call  /usr/lib/jvm/java-1.8.0-openjdk/jre/bin/jjs  script_with_jjs_schebang
   - both those workarounds are highly against oracle documentation
   - and both those workarounds have prerequisite that our jdk8 provides /usr/bin/jjs. That I cannot reasonlby added, once we actually do not provide it in solid state. 
 


Now imagine this mess with BuildRequires java-1.7.0-openjdk devel, and on top of it with jshell.....

I belive that except unhappy users, we may soon face  similar problems in our ownbuildroot.


So what another solution you have in mind? Alternatives gys are ok with this patching from they perspective - It actually have sense to keep providing slave  when another master doesnt.

Comment 25 Deepak Bhole 2016-06-23 15:51:51 UTC
(In reply to jiri vanek from comment #23)
> But the tool  will be working regardless.
> Even if you set java and javac to java-1.7.0-* the /uss/bin/jjs would work
> properly, using its jdk8.  So from this point there is no  reason to not to
> do this.

I understand that, but we are still mixing things. Let's say jjs has some functionality that allows it to create a class file or something -- users will then use such tools to create an output file that cannot be used with the system set JDK any more (unless they change it).

Is the change to alternatives to preserve this already in?

Comment 26 jiri vanek 2016-06-24 05:44:02 UTC
(In reply to Deepak Bhole from comment #25)
> (In reply to jiri vanek from comment #23)
> > But the tool  will be working regardless.
> > Even if you set java and javac to java-1.7.0-* the /uss/bin/jjs would work
> > properly, using its jdk8.  So from this point there is no  reason to not to
> > do this.
> 
> I understand that, but we are still mixing things. Let's say jjs has some

Especially jjs is only javascript interpreter, so it have nothning to do with class files.

Same is jshell  - it is interpret of  java, so non of those two  I wont to "--persist"  will leave any "for default jdk unknown" artefacts.

> functionality that allows it to create a class file or something -- users
> will then use such tools to create an output file that cannot be used with
> the system set JDK any more (unless they change it).
> 
> Is the change to alternatives to preserve this already in?

Of course not. It will not go in without your approval. The experimental patch exists (which is nto perfect - see https://bugzilla.redhat.com/show_bug.cgi?id=1317597#c7 )  but even that is improvement from current state and allows to provide /usr/bin/{jjs,jshell}

We managed acks for it, but main decision point lies there.  If java-team come with better solution, alternative guys will be happy to drop the bug.

Comment 27 jiri vanek 2016-06-29 11:28:41 UTC
(In reply to Andrey Loskutov from comment #16)
> (In reply to jiri vanek from comment #15)
> > > #!/usr/bin/env jjs requires jjs to be in PATH, which limits the general
> > > usage of this pattern, since most users do not have jjs in the PATH.
> > 
> > Wait. How so? /user/bin is usually on path. Isn't it?
> 
> Sure, this should always work as long as jjs is installed (linked) to
> /user/bin.

Hello!

Current resolution is to provide /usr/bin/jjs (and /usr/bin/jshell in jdk9)  via provides in rpms.

This will be achieved by alternatives providing mechanism (still under discussion how to do so best) to keep those provided links alive.

Comment 28 jiri vanek 2016-07-12 14:23:14 UTC
Pushed and built.

Comment 30 Lukáš Zachar 2016-08-23 08:28:23 UTC
jjs is provided by -openjdk but alternatives is configured by -headless (which also owns the file).

IMHO that's wrong - headless can be installed without -openjdk - why it shouldn't be provided in such case?

# rpm -q --whatprovides /usr/bin/jjs
java-1.8.0-openjdk-1.8.0.102-0.b14.el7

# rpm -q --scripts java-1.8.0-openjdk-headless | grep jjs
  --slave /usr/bin/jjs jjs /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.102-0.b14.el7.x86_64/jre/bin/jjs \
  --slave /usr/share/man/man1/jjs.1$ext jjs.1$ext \
  /usr/share/man/man1/jjs-java-1.8.0-openjdk-1.8.0.102-0.b14.el7.x86_64.1$ext \

Comment 35 errata-xmlrpc 2016-11-03 22:56:19 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-2139.html