Bug 435235

Summary: Java fails to load 64 bit JNI bindings (*.so) located in lib64 folders (java.library.path problem)
Product: [Fedora] Fedora Reporter: Rajesh Krishnan <raj.dev.redhat>
Component: java-1.6.0-openjdkAssignee: Lillian Angel <langel>
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: low    
Version: rawhideKeywords: Reopened
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2008-04-28 14:29:49 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Rajesh Krishnan 2008-02-28 04:51:37 UTC
Description of problem:
64 bit JNI bindings (*.so files) located under /lib64/ or /usr/lib64/ 
or /usr/local/lib64 folders cannot be loaded from Java applications because 
the respective folders are missing from the java.library.path system property 
(only the 32 bit folders /lib/ and /usr/lib/ folders are available to 
the "java.library.path" system property.

Version-Release number of selected component (if applicable):
  # java -version
java version "1.7.0"
IcedTea Runtime Environment (build 1.7.0-b21)
IcedTea 64-Bit Server VM (build 1.7.0-b21, mixed mode)

  # file $(which java)
/usr/bin/java: symbolic link to `/etc/alternatives/java'

  # file /etc/alternatives/java
/etc/alternatives/java: symbolic link to 
`/usr/lib/jvm/jre-1.7.0-icedtea.x86_64/bin/java'

  # file /usr/lib/jvm/jre-1.7.0-icedtea.x86_64/bin/java
/usr/lib/jvm/jre-1.7.0-icedtea.x86_64/bin/java: ELF 64-bit LSB executable, 
x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 
2.6.9, stripped


How reproducible:
On an AMD64 machine (x86_64) create a simple java class to print out 
the "java.library.path" property using the System.getProperty() method as 
shown below.
Observe that at the end of the ":" separated list of directories, only /lib 
and /usr/lib are specified (besides others).  The /lib64 and /usr/lib64 are 
not available to the instance of running Java VM to load any 64 bit .so JNI 
native bindings that may be installed in the corresponding lib64 folders.

Steps to Reproduce:

1.  Create the following Java file (LibPath.java):
// ------------- Begin: file LibPath.java -------------
class LibPath
{
	public static void main(String[] saArgs)
	{
		System.out.println("java.library.path = " + 
System.getProperty("java.library.path"));
	}
}
// ------------- End: file LibPath.java -------------

2.  Compile the file:
  $ javac LibPath.java

3.  Run the LibPath classfile:
  $ java -cp . LibPath
java.library.path 
= /usr/lib/jvm/java-1.7.0-icedtea-1.7.0.0.x86_64/jre/lib/amd64/server:/usr/lib/jvm/java-1.7.0-icedtea-1.7.0.0.x86_64/jre/lib/amd64:/usr/lib/jvm/java-1.7.0-icedtea-1.7.0.0.x86_64/jre/../lib/amd64:/usr/java/packages/lib/amd64:/lib:/usr/lib

4. Observe that the end of the line only has "lib" and not "lib64" folders.

5.  Install (only) a 64-bit JNI binding library like kdebindings.x86_64:
  # yum install kdebindings.x86_64
  # yum list  kdebindings
Installed Packages
kdebindings.x86_64                       3.5.8-1.fc8            installed
Available Packages
kdebindings.i386                         3.5.8-1.fc8            fedora

6.  Update the above LibPath.java to do something interesting with 
kdebindings:
// ------------- Begin: file LibPath.java -------------
class LibPath
{
	public static void main(String[] saArgs)
	{
		org.kde.qt.qtjava.initialize();
		org.kde.koala.kdejava.initialize();
		// System.out.println("java.library.path = " + 
System.getProperty("java.library.path"));
		
	}
}
// ------------- End: file LibPath.java -------------

7.  Compile the file with kdebinding additions:
  $ javac -cp /usr/lib64/java/koala.jar:/usr/lib64/java/qtjava.jar:.   
LibPath.java

8.  Try to run the class now:
  $ java -cp /usr/lib64/java/koala.jar:/usr/lib64/java/qtjava.jar:.   LibPath
Exception in thread "main" java.lang.UnsatisfiedLinkError: no qtjava in 
java.library.path
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1700)
        at java.lang.Runtime.loadLibrary0(Runtime.java:841)
        at java.lang.System.loadLibrary(System.java:1067)
        at org.kde.qt.qtjava.initialize(qtjava.java:272)
        at org.kde.qt.qtjava.<clinit>(qtjava.java:283)
        at LibPath.main(LibPath.java:6)

Notice the Java exception. It is not able to load libkdejava.so and 
libqtjava.so because they are located in /usr/lib64 folders, and that is not 
abailable to the java runtime system.

9.  Just to test that the program would actually work if "java.library.path" 
sys-prop contains the lib*java.so files (create symlinks - WARNING:  DO NOT DO 
THIS ON PRODUCTION SYSTEM) execute the following as ROOT:
  #  cd /usr/lib
  # ln -s ../lib64/libkdejava.so.1 libkdejava.so
  # ln -s ../lib64/libqtjava.so.1 libqtjava.so

Now if you execute the Java class again as above, it executes fine without 
exceptions.  But DO NOT forget to remove the symlinks created above.

  
Actual results:
Java throws an Exception at runtime due to non-availability of corresponding 
*.so JNI bindings in java.library.path.


Expected results:
No exception should be thrown by the runtime if IcedTea has 
proper "java.library.path" configuration for x86_64 machines.


Additional info:
1. This issue is reproducible when logged in as both root and as regular user 
account.

2.  If the 64-bit JVM can also load the 32 bit *.so native bindings, then the 
32-bit *.so folder locations /lib/ /usr/lib and /usr/local/lib need to be 
appended to the "java.library.path" system property AFTER the respective 
64-bit folders.

3.  Open Question:  Does alternative lib64 folders /opt/lib64  folder (and 
maybe others) also need to be made available to the JVM?  What does SUN say 
about LSB folder locations for 64-bit binaries?

4.  Note that the current configuration may be "adequate" for 32-bit Fedora 
installations (the non-availibility of /usr/local/lib/ folder for placing JNI 
bindings remains an issue though).

5.  I know that the additional path could be specified 
using "java -Djava.library.path= ...."  option, but that is not the 
politically correct way of including the additional system path.  Moreover 
this would add additional burden on every Java application out there that uses 
JNI bindings.

6.  Maybe there should be a symlink called /usr/lib64/libkdejava.so 
to  /usr/lib64/libkdejava.so.1.0.0 as for libqtjava.so, but that is a 
different problem with the packaging of kdebindings package, not the IcedTea 
one in discussion:
  # rpm -ql kdebindings.x86_64 | grep -i libkdejava
/usr/lib64/libkdejava.la
/usr/lib64/libkdejava.so.1
/usr/lib64/libkdejava.so.1.0.0

  # rpm -ql kdebindings.x86_64 | grep -i libqtjava
/usr/lib64/libqtjava.la
/usr/lib64/libqtjava.so.1
/usr/lib64/libqtjava.so.1.0.0
/usr/lib64/libqtjavasupport.la
/usr/lib64/libqtjavasupport.so.1
/usr/lib64/libqtjavasupport.so.1.0.0

Comment 1 Lillian Angel 2008-02-28 15:45:30 UTC
This was fixed in the repository, and will be in rawhide soon.

java -version
java version "1.7.0"
IcedTea Runtime Environment (build 1.7.0-b24)
IcedTea 64-Bit Server VM (build 1.7.0-b24, mixed mode)

$ java LibPath
java.library.path=/usr/lib/jvm/java-1.7.0-icedtea-1.7.0.0.x86_64/jre/lib/amd64/server:/usr/lib/jvm/java-1.7.0-icedtea-1.7.0.0.x86_64/jre/lib/amd64:/usr/lib/jvm/java-1.7.0-icedtea-1.7.0.0.x86_64/jre/../lib/amd64:/usr/java/packages/lib/amd64:/lib:/usr/lib

Hence, it is not yet available in b24 (IcedTea in rawhide at the moment)

$ ./icedtea/openjdk/build/linux-amd64/bin/java LibPath
java.library.path=/notnfs/langel/icedtea/openjdk/build/linux-amd64/lib/amd64/server:/notnfs/langel/icedtea/openjdk/build/linux-amd64/lib/amd64:/notnfs/langel/icedtea/openjdk/build/linux-amd64/../lib/amd64:/usr/java/packages/lib/amd64:/lib:/usr/lib:/usr/lib64:/lib64

This version will be in soon
Thanks.

Comment 2 Rajesh Krishnan 2008-02-28 22:02:39 UTC
Thank you for the Rawhide related info.

IMHO your RawHide "java.library.path" is still not correct for the following 
reasons:

1.  The /lib and /usr/lib SHOULD NOT precede /lib64 and /usr/lib64 on x86_64 
systems. 
In orther words, instead of:
  java.library.path= .... :/lib:/usr/lib:/usr/lib64:/lib64
It should be:
  java.library.path= .... :/usr/lib64:/lib64:/lib:/usr/lib
on x86_64 systems, and it should be like:
  java.library.path= .... :/lib:/usr/lib
on 32-bit systems.

2.  The new Rawhide path does not include /usr/local/lib64 folder (again, this 
should precede before the corresponding /usr/local/lib 32-bit folder in the 
path).  So finally, the correct (my opinion) path should look like:
 
java.library.path= .... :/usr/lib64:/lib64:/usr/local/lib64:/lib:/usr/lib:/usr/local/lib
on 64-bit systems and it should look like:
java.library.path= .... :/lib:/usr/lib:/usr/local/lib
on 32-bit systems.

Notice inclusion of both /usr/local/lib64 and /usr/local/lib and their order 
above.


3.  Though strictly not required, for compatibility reasons, it would be a 
NICE-TO-HAVE feature if we also could include /opt/lib64 and /opt/lib also as 
one of the java.library.path folders for the sake of some custom built/3rd 
party packages could use it.

At least (1) and (2) above might need to be satisfied in the next build as 
part of this issue.  The (3) could be something the Rawhide devs might want to 
think about.  Hence I am reopening this bug.



Comment 3 Lillian Angel 2008-03-04 19:47:52 UTC
Thanks, I will look into this for the next release

Comment 4 Lillian Angel 2008-04-21 18:12:02 UTC
I fixed this in the repository. This will be in the next release.

Thx

Comment 5 Thomas Fitzsimmons 2008-04-21 21:44:32 UTC
(In reply to comment #0)

> 3.  Open Question:  Does alternative lib64 folders /opt/lib64  folder (and 
> maybe others) also need to be made available to the JVM?  What does SUN say 
> about LSB folder locations for 64-bit binaries?

Applications installed in /opt should set their own LD_LIBRARY_PATH.

> 4.  Note that the current configuration may be "adequate" for 32-bit Fedora 
> installations (the non-availibility of /usr/local/lib/ folder for placing JNI 
> bindings remains an issue though).

I don't see the rationale for /usr/local/lib being added to OpenJDK's default
LD_LIBRARY_PATH.

> 5.  I know that the additional path could be specified 
> using "java -Djava.library.path= ...."  option, but that is not the 
> politically correct way of including the additional system path.  Moreover 
> this would add additional burden on every Java application out there that uses 
> JNI bindings.

JNI-using applications packaged for Fedora can and should use System.load to
load the required JNI library directly, instead of System.loadLibrary.

> 6.  Maybe there should be a symlink called /usr/lib64/libkdejava.so 
> to  /usr/lib64/libkdejava.so.1.0.0 as for libqtjava.so, but that is a 
> different problem with the packaging of kdebindings package [...]

There is a section in the Fedora Java Packaging Guidelines covering this topic:

http://fedoraproject.org/wiki/Packaging/Java#head-61a3ee0d05ff616ef9be2021b489610e036fd932

Its recommendations are different than what kdebindings is currently doing (but
the guidelines were written after the kdebindings package).  kdebindings should
eventually be updated to the guidelines.

Comment 6 Thomas Fitzsimmons 2008-04-21 22:07:32 UTC
(In reply to comment #5)
> (In reply to comment #0)

> I don't see the rationale for /usr/local/lib being added to OpenJDK's default
> LD_LIBRARY_PATH.

"LD_LIBRARY_PATH" should read "dlopen path".