Bug 1306945 - Review Request: ohc - Java large off heap cache
Summary: Review Request: ohc - Java large off heap cache
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: Package Review
Version: rawhide
Hardware: All
OS: Linux
medium
medium
Target Milestone: ---
Assignee: jiri vanek
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-02-12 09:09 UTC by Tomas Repik
Modified: 2016-06-30 21:27 UTC (History)
2 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2016-06-30 21:27:34 UTC
Type: ---
Embargoed:
jvanek: fedora-review+


Attachments (Terms of Use)

Description Tomas Repik 2016-02-12 09:09:48 UTC
Spec URL: https://trepik.fedorapeople.org/ohc.spec
SRPM URL: https://trepik.fedorapeople.org/ohc-0.4.2-2.fc23.src.rpm

Description: To include cassandra in fedora ohc is required.
OHC - Off-Heap Concurrent hash map intended to store GBs of serialized data. When you have to deal with a huge amount of several/many GB of cache memory since that does not put any pressure on the Java garbage collector. Let the Java GC do its job for the application where this library does its job for the cached data.
Fedora Account System Username: trepik

Comment 1 jiri vanek 2016-03-07 11:19:43 UTC
Hello! Build fails on f24 - I did not investigate, but is your target really only rawhide? If not, then you shoudl probably investigate.

Comment 2 jiri vanek 2016-03-07 12:16:13 UTC
The benchmark subproject contains again few powerful launchers. I would say worthy to pack...

Comment 3 jiri vanek 2016-03-07 12:19:06 UTC
I maight be missing something - but it seems that it needs some aditional library to work fine with jdk8. I fiale dto verify this:


eg in logs: 
2016-03-07 12:10:58,036 WARN   [Uns] Failed to load Java8 implementation ohc-core-j8 : java.lang.ClassNotFoundException: org.caffinitas.ohc.linked.UnsExt8 

or in readme:
An extension jar that makes use of new sun.misc.Unsafe methods in Java 8 exists.


Thoughts?

Comment 4 jiri vanek 2016-03-07 12:24:57 UTC
Package Review
==============

Legend:
[x] = Pass, [!] = Fail, [-] = Not applicable, [?] = Not evaluated
[ ] = Manual review needed



===== MUST items =====

Generic:
[x]: Package is licensed with an open-source compatible license and meets
     other legal requirements as defined in the legal section of Packaging
     Guidelines.
[x]: License field in the package spec file matches the actual license.
     Note: Checking patched sources after %prep for licenses. Licenses
     found: "Apache (v2.0)", "Unknown or generated", "*No copyright* Apache
     (v2.0)". 5 files have unknown license. Detailed output of licensecheck
     in /home/jvanek/1306945-ohc/licensecheck.txt
[x]: License file installed when any subpackage combination is installed.
[x]: Package contains no bundled libraries without FPC exception.
[x]: Changelog in prescribed format.
[x]: Sources contain only permissible code or content.
[-]: Package contains desktop file if it is a GUI application.
[x]: Development files must be in a -devel package
[x]: Package uses nothing in %doc for runtime.
[x]: Package consistently uses macros (instead of hard-coded directory
     names).
[x]: Package is named according to the Package Naming Guidelines.
[x]: Package does not generate any conflict.
[x]: Package obeys FHS, except libexecdir and /usr/target.
[-]: If the package is a rename of another package, proper Obsoletes and
     Provides are present.
[x]: Requires correct, justified where necessary.
[x]: Spec file is legible and written in American English.
[x]: Package contains systemd file(s) if in need.
[x]: Package is not known to require an ExcludeArch tag.
[-]: Large documentation must go in a -doc subpackage. Large could be size
     (~1MB) or number of files.
     Note: Documentation size is 30720 bytes in 4 files.
[x]: Package complies to the Packaging Guidelines
[x]: Package successfully compiles and builds into binary rpms on at least
     one supported primary architecture.
[x]: Package installs properly.
[x]: Rpmlint is run on all rpms the build produces.
     Note: There are rpmlint messages (see attachment).
[x]: If (and only if) the source package includes the text of the
     license(s) in its own file, then that file, containing the text of the
     license(s) for the package is included in %license.
[x]: Package requires other packages for directories it uses.
[x]: Package must own all directories that it creates.
[x]: Package does not own files or directories owned by other packages.
[x]: All build dependencies are listed in BuildRequires, except for any
     that are listed in the exceptions section of Packaging Guidelines.
[x]: Package uses either %{buildroot} or $RPM_BUILD_ROOT
[x]: Package does not run rm -rf %{buildroot} (or $RPM_BUILD_ROOT) at the
     beginning of %install.
[x]: Macros in Summary, %description expandable at SRPM build time.
[x]: Dist tag is present.
[x]: Package does not contain duplicates in %files.
[x]: Permissions on files are set properly.
[x]: Package use %makeinstall only when make install DESTDIR=... doesn't
     work.
[x]: Package is named using only allowed ASCII characters.
[x]: Package does not use a name that already exists.
[x]: Package is not relocatable.
[x]: Sources used to build the package match the upstream source, as
     provided in the spec URL.
[x]: Spec file name must match the spec package %{name}, in the format
     %{name}.spec.
[x]: File names are valid UTF-8.
[x]: Packages must not store files under /srv, /opt or /usr/local

Java:
[x]: Bundled jar/class files should be removed before build
[x]: Packages have proper BuildRequires/Requires on jpackage-utils
     Note: Maven packages do not need to (Build)Require jpackage-utils. It
     is pulled in by maven-local
[x]: Javadoc documentation files are generated and included in -javadoc
     subpackage
[x]: Javadoc subpackages should not have Requires: jpackage-utils
[x]: Javadocs are placed in %{_javadocdir}/%{name} (no -%{version} symlink)

Maven:
[x]: If package contains pom.xml files install it (including metadata) even
     when building with ant
[x]: POM files have correct Maven mapping
[x]: Maven packages should use new style packaging
[x]: Old add_to_maven_depmap macro is not being used
[x]: Packages DO NOT have Requires(post) and Requires(postun) on jpackage-
     utils for %update_maven_depmap macro
[x]: Package DOES NOT use %update_maven_depmap in %post/%postun
[x]: Packages use .mfiles file list instead of %{_datadir}/maven2/poms

===== SHOULD items =====

Generic:
[-]: If the source package does not include license text(s) as a separate
     file from upstream, the packager SHOULD query upstream to include it.
[x]: Final provides and requires are sane (see attachments).
[-]: Fully versioned dependency in subpackages if applicable.
     Note: No Requires: %{name}%{?_isa} = %{version}-%{release} in ohc-
     benchmark , ohc-core-j8 , ohc-jmh , ohc-parent , ohc-javadoc
[?]: Package functions as described.
[x]: Latest version is packaged.
[x]: Package does not include license text files separate from upstream.
[!]: Patches link to upstream bugs/comments/lists or are otherwise
     justified.
[-]: Description and summary sections in the package spec file contains
     translations for supported Non-English languages, if available.
[x]: Package should compile and build into binary rpms on all supported
     architectures.
[x]: %check is present and all tests pass.
[x]: Packages should try to preserve timestamps of original installed
     files.
[x]: Packager, Vendor, PreReq, Copyright tags should not be in spec file
[x]: Sources can be downloaded from URI in Source: tag
[x]: Reviewer should test that the package builds in mock.
[x]: Buildroot is not present
[x]: Package has no %clean section with rm -rf %{buildroot} (or
     $RPM_BUILD_ROOT)
[x]: No file requires outside of /etc, /bin, /sbin, /usr/bin, /usr/sbin.
[x]: SourceX is a working URL.
[x]: Spec use %global instead of %define unless justified.

Java:
[x]: Package uses upstream build method (ant/maven/etc.)
[x]: Packages are noarch unless they use JNI

===== EXTRA items =====

Generic:
[x]: Rpmlint is run on all installed packages.
     Note: There are rpmlint messages (see attachment).
[x]: Spec file according to URL is the same as in SRPM.


Rpmlint
-------
Checking: ohc-0.4.2-2.fc24.noarch.rpm
          ohc-benchmark-0.4.2-2.fc24.noarch.rpm
          ohc-core-j8-0.4.2-2.fc24.noarch.rpm
          ohc-jmh-0.4.2-2.fc24.noarch.rpm
          ohc-parent-0.4.2-2.fc24.noarch.rpm
          ohc-javadoc-0.4.2-2.fc24.noarch.rpm
          ohc-0.4.2-2.fc24.src.rpm
ohc-core-j8.noarch: W: no-documentation
ohc-jmh.noarch: W: no-documentation
ohc-parent.noarch: W: no-documentation
7 packages and 0 specfiles checked; 0 errors, 3 warnings.




Rpmlint (installed packages)
----------------------------
sh: /usr/bin/python: No such file or directory
ohc-core-j8.noarch: W: no-documentation
ohc-jmh.noarch: W: no-documentation
ohc-parent.noarch: W: no-documentation
6 packages and 0 specfiles checked; 0 errors, 3 warnings.



Requires
--------
ohc-core-j8 (rpmlib, GLIBC filtered):
    java-headless
    jpackage-utils
    mvn(org.caffinitas.ohc:ohc-core)

ohc-javadoc (rpmlib, GLIBC filtered):
    jpackage-utils

ohc (rpmlib, GLIBC filtered):
    java-headless
    jpackage-utils
    mvn(com.google.guava:guava)
    mvn(net.java.dev.jna:jna)
    mvn(org.slf4j:slf4j-api)

ohc-benchmark (rpmlib, GLIBC filtered):
    java-headless
    jpackage-utils
    mvn(commons-cli:commons-cli)
    mvn(io.dropwizard.metrics:metrics-core)
    mvn(org.apache.commons:commons-math3)
    mvn(org.apache.logging.log4j:log4j-api)
    mvn(org.apache.logging.log4j:log4j-core)
    mvn(org.apache.logging.log4j:log4j-slf4j-impl)
    mvn(org.caffinitas.ohc:ohc-core)
    mvn(org.caffinitas.ohc:ohc-core-j8)

ohc-jmh (rpmlib, GLIBC filtered):
    java-headless
    jpackage-utils
    mvn(org.apache.logging.log4j:log4j-core)
    mvn(org.apache.logging.log4j:log4j-slf4j-impl)
    mvn(org.caffinitas.ohc:ohc-core)
    mvn(org.openjdk.jmh:jmh-core)

ohc-parent (rpmlib, GLIBC filtered):
    java-headless
    jpackage-utils



Provides
--------
ohc-core-j8:
    mvn(org.caffinitas.ohc:ohc-core-j8)
    mvn(org.caffinitas.ohc:ohc-core-j8:pom:)
    ohc-core-j8

ohc-javadoc:
    ohc-javadoc

ohc:
    mvn(org.caffinitas.ohc:ohc-core)
    mvn(org.caffinitas.ohc:ohc-core:pom:)
    ohc

ohc-benchmark:
    mvn(org.caffinitas.ohc:ohc-benchmark)
    mvn(org.caffinitas.ohc:ohc-benchmark:pom:)
    ohc-benchmark

ohc-jmh:
    mvn(org.caffinitas.ohc:ohc-jmh)
    mvn(org.caffinitas.ohc:ohc-jmh:pom:)
    ohc-jmh

ohc-parent:
    mvn(org.caffinitas.ohc:ohc-parent:pom:)
    ohc-parent



Source checksums
----------------
https://github.com/snazy/ohc/archive/0.4.2.tar.gz :
  CHECKSUM(SHA256) this package     : 67329a58527f8100ef42f4e22c7a187474048b0005a2093b6a024ba90c96c83d
  CHECKSUM(SHA256) upstream package : 67329a58527f8100ef42f4e22c7a187474048b0005a2093b6a024ba90c96c83d


Generated by fedora-review 0.6.0 (3c5c9d7) last change: 2015-05-20
Command line :/usr/bin/fedora-review -b 1306945 -m fedora-rawhide-x86_64
Buildroot used: fedora-rawhide-x86_64
Active plugins: Generic, Shell-api, Java
Disabled plugins: C/C++, Python, fonts, SugarActivity, Ocaml, Perl, Haskell, R, PHP, Ruby
Disabled flags: EXARCH, DISTTAG, EPEL5, BATCH, EPEL6

Comment 5 jiri vanek 2016-03-07 12:25:39 UTC
So teo questions - why the patch, and why not the launchers?

Nice pacakge anyway. Thanx!

Comment 6 Tomas Repik 2016-03-07 16:42:46 UTC
(In reply to jiri vanek from comment #1)
> Build fails on f24
I'm gonna aim for f24 of course, so yes I'm gonna work on this.

(In reply to jiri vanek from comment #5)
> So teo questions - why the patch?
It was there just temporarly until lz4-java was added to fedora.
it was added few days ago
The patch will be removed in the newer version

> and why not the launchers?
I'm gonna work on this as well.

Anyway thanks so far.

Comment 7 jiri vanek 2016-03-08 07:53:40 UTC
Ok. I will wait for new package and reveiew again (or watch diff:) )

Comment 8 Tomas Repik 2016-03-10 11:38:58 UTC
Should I also install these two scripts? batch-bench.sh and consolidate-output.sh
If yes, could you give me some advice on how to to that?

Comment 9 jiri vanek 2016-03-15 19:20:14 UTC
Originally I did not wont you to do so, but when you have asked I get tempted and take second look.

The batch-bench really looks like useful binary.  General rule for such a script is to rewrote them, but this one does not looks like best candidate to it.
Look to the generated launcher from your first package (The hystogram) and you can find that only think you need to replace in this huge script is generation of classpath and how java is found and launched. Long story short - sed on two lines if it goes well.

Second script looks like small tool. So it looks like it can be installed as it is.

Also I would recommand to add prefix to them - ohc-batch... and ohc-consolidat...

Feel free to take this as advance and exercise. Not an must.  (but after you forced me to do second reading of those two scripts it would be nice to have them :)

Comment 10 Tomas Repik 2016-03-16 13:31:56 UTC
Spec URL: https://trepik.fedorapeople.org/ohc/v1/ohc.spec
SRPM URL: https://trepik.fedorapeople.org/ohc/v1/ohc-0.4.2-3.fc23.src.rpm

* Wed Mar 16 2016 Tomas Repik <trepik> - 0.4.2-3
- launcher BenchmarkOHC installation
- benchmark scrips added

Comment 11 jiri vanek 2016-03-17 10:54:34 UTC
Hello! The launchers seems not to be working.
The  ohc-batch-bench is misisng dependences. You have put just main jar to classapth. You need all dependencies to..
Also I think that you can drop `which java` and just keep java. (but I may miss why you did it)
I changed : 
....
jar=/usr/share/java/guava.jar:/usr/share/java/jmh/jmh-core.jar:/usr/share/java/slf4j/nop.jar:/usr/share/java/slf4j/api.jar:/usr/share/java/slf4j/simple.jar:/usr/share/java/slf4j/slf4j-api.jar:/usr/share/java/slf4j/slf4j-nop.jar:/usr/share/java/slf4j/slf4j-simple.jar:/usr/share/java/metrics/metrics-core.jar:/usr/share/java/commons-math3.jar:/usr/share/java/log4j.jar:/usr/share/java/jna.jar:/usr/share/java/commons-cli.jar:/usr/share/java/apache-commons-cli.jar:/usr/share/java/apache-commons-math.jar:/usr/share/java/log4j/log4j-to-slf4j.jar:/usr/share/java/log4j/log4j-slf4j-impl.jar:/usr/share/java/log4j/log4j-iostreams.jar:/usr/share/java/log4j/log4j-jul.jar:/usr/share/java/log4j/log4j-core.jar:/usr/share/java/log4j/log4j-api.jar:/usr/share/java/log4j/log4j-1.2-api.jar:/usr/share/java/jopt-simple/jopt-simple.jar:/usr/share/java/ohc/ohc-benchmark.jar:/usr/share/java/ohc/ohc-jmh.jar:/usr/share/java/ohc/ohc-core.jar:/usr/share/java/ohc/ohc-core-j8.jar

#if [ ! -f $jar ] ; then
#    echo "Executable jar file $jar does not exist" > /dev/stderr
#    exit 1
#fi
java -cp $jar org.caffinitas.ohc.benchmark.BenchmarkOHC -h > /dev/null
if [ ! $? ] ; then
    echo "Cannot execute ohc-benchmark jar file $jar" > /dev/stderr
    exit 1
fi
...
To make it work. Please dont copypaste my chnagehs, they may nto be enough (best is to generate launcher for ohc-benchmark.jar package, and then copypaste generated classpath :)
Also you need to require "time" package , as the script is calling "time" (dont confuse it with embedded shell function time)

After that I had quite a lot of cvs files to tes second launcher, and that worked fine


One fatal issue I overlooked in first iteration
 - you have main classes in jar:
  - ohc-benchmark.jar have org.caffinitas.ohc.benchmark.BenchmarkOHC
  - thats forbidden by guidelines


Also there is:
Rpmlint (installed packages)
----------------------------
sh: /usr/bin/python: No such file or directory

But I was nto able to reproduce it.

Please fix the launcher and the main class ( I guess you need to remove maven attribute which is setting it)

Few more notes -
%jpackage_script - maybe prefix the launcher by ohc too?
chmod 755  - its better to specifi this in filles like http://pkgs.fedoraproject.org/cgit/rpms/java-1.8.0-openjdk.git/tree/java-1.8.0-openjdk.spec#n516 (just without ghosts :) )

Comment 12 Tomas Repik 2016-03-21 15:20:12 UTC
Spec URL: https://trepik.fedorapeople.org/ohc/v2/ohc.spec
SRPM URL: https://trepik.fedorapeople.org/ohc/v2/ohc-0.4.2-4.fc23.src.rpm

* Mon Mar 21 2016 Tomas Repik <trepik> - 0.4.2-4
- launchers renamed (ohc-batch-benchmark, ohc-benchmark, ohc-consolidate-output)
- %attr(775, root, root) instead of chmod 755
- classpath in ohc-benchmark updated
- ohc-batch-benchmark revised, but...

running ohc-batch-benchmark produced:
Warning Failed to load Java8 implementation ohc-core-j8 : java.lang.NoSuchMethodException: org.caffinitas.ohc.linked.UnsExt8.<init>(java.lang.Class)

CVS files are generated but the execution "never" ends

On the main class I found only:
JAR files MUST NOT include classh-path entry inside META-INF/MANIFEST.MF 

https://fedoraproject.org/wiki/Packaging:Java#No_class-path_in_MANIFEST.MF

Comment 13 jiri vanek 2016-03-22 11:49:34 UTC
> 
> On the main class I found only:
> JAR files MUST NOT include classh-path entry inside META-INF/MANIFEST.MF 
> 
> https://fedoraproject.org/wiki/Packaging:Java#No_class-path_in_MANIFEST.MF

Exactly. So please remove org.caffinitas.ohc.benchmark.BenchmarkOHC from ohc-benchmark.jar

Comment 14 jiri vanek 2016-03-22 11:56:26 UTC
Mia cupla! Mian class x class path. Is should stop smoking java leaves. 
Please ignore that note of mine unless you wish to not.

Comment 15 jiri vanek 2016-03-29 12:54:27 UTC
> running ohc-batch-benchmark produced:
> Warning Failed to load Java8 implementation ohc-core-j8 :
> java.lang.NoSuchMethodException:
> org.caffinitas.ohc.linked.UnsExt8.<init>(java.lang.Class)
> 
> CVS files are generated but the execution "never" ends
> 
> On the main class I found only:
> JAR files MUST NOT include classh-path entry inside META-INF/MANIFEST.MF 
> 
> https://fedoraproject.org/wiki/Packaging:Java#No_class-path_in_MANIFEST.MF


When I run it as root, it dies on various exception. As non root, it passes for me in mack/real system/VM

So I think your script is correct.May you try it under non root user/ out of mock?

Also - The missing class is in /usr/share/java/ohc/ohc-core-j8.jar - afaik this jar is on your classapth. So maybe yo have some wrong environment?

As for classpath - diffing my and yours - are you sure ohc-jmh. jar fo ot need to be on classapth?


I have no issues with package, and from my point of view it is approved. I will wait for review+ on your reply.

Comment 16 Tomas Repik 2016-03-30 10:31:12 UTC
(In reply to jiri vanek from comment #15)
> May you try it under non root user/ out of mock?
Doesn't matter whether run under root or non root, behaviour still the same. Same warning comes up when running ohc-benchmark.

> Also - The missing class is in /usr/share/java/ohc/ohc-core-j8.jar - afaik
> this jar is on your classapth. So maybe yo have some wrong environment?
I don't know how to find this out.
 
> As for classpath - diffing my and yours - are you sure ohc-jmh. jar fo ot
> need to be on classapth?
I tried including it in classpath but with no effect.

> I have no issues with package, and from my point of view it is approved. I
> will wait for review+ on your reply.
thanks

Comment 17 jiri vanek 2016-03-30 12:25:44 UTC
jsut fyi:
javap UnsExt8.class 
Compiled from "UnsExt8.java"
final class org.caffinitas.ohc.linked.UnsExt8 extends org.caffinitas.ohc.linked.UnsExt {
  org.caffinitas.ohc.linked.UnsExt8(sun.misc.Unsafe);
  long getAndPutLong(long, long, long);
  int getAndAddInt(long, long, int);
  long crc32(long, long, long);
}

Comment 18 jiri vanek 2016-03-30 12:47:36 UTC
Working for me on other and other machines....

Considering main purpose as library, package is APPROVED

Comment 19 Gwyn Ciesla 2016-04-01 01:51:13 UTC
Package request has been approved: https://admin.fedoraproject.org/pkgdb/package/rpms/ohc

Comment 20 Fedora Update System 2016-04-04 10:56:25 UTC
ohc-0.4.2-4.fc23 has been submitted as an update to Fedora 23. https://bodhi.fedoraproject.org/updates/FEDORA-2016-254a29f4e3

Comment 21 Fedora Update System 2016-04-05 15:53:52 UTC
ohc-0.4.2-4.fc23 has been pushed to the Fedora 23 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-254a29f4e3

Comment 22 Fedora Update System 2016-04-13 20:23:37 UTC
ohc-0.4.2-4.fc23 has been pushed to the Fedora 23 stable repository. If problems still persist, please make note of it in this bug report.

Comment 23 Fedora Update System 2016-06-22 06:46:25 UTC
ohc-0.4.2-4.fc24 has been submitted as an update to Fedora 24. https://bodhi.fedoraproject.org/updates/FEDORA-2016-413823b182

Comment 24 Fedora Update System 2016-06-22 23:03:01 UTC
ohc-0.4.2-4.fc24 has been pushed to the Fedora 24 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-413823b182

Comment 25 Fedora Update System 2016-06-30 21:27:31 UTC
ohc-0.4.2-4.fc24 has been pushed to the Fedora 24 stable repository. If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.