Bug 823122 - Review Request: zookeeper - A high-performance coordination service for distributed applications
Summary: Review Request: zookeeper - A high-performance coordination service for distr...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: Package Review
Version: rawhide
Hardware: All
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Björn 'besser82' Esser
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On: 957337
Blocks: bigdata-review 948589 968136 985087
TreeView+ depends on / blocked
 
Reported: 2012-05-19 09:21 UTC by gil cattaneo
Modified: 2013-10-05 12:07 UTC (History)
8 users (show)

Fixed In Version: zookeeper-3.4.5-5.fc18
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2013-06-29 18:40:53 UTC
Type: ---
Embargoed:
besser82: fedora-review+
gwync: fedora-cvs+


Attachments (Terms of Use)
Adds building/packaging of the zookeeper-test.jar (1.05 KB, patch)
2013-04-23 16:47 UTC, Robert Rati
no flags Details | Diff
systemd service file for zookeeper (342 bytes, application/octet-stream)
2013-04-24 18:42 UTC, Jeffrey C. Ollie
no flags Details
Updated patch for zookeeper-test jar (2.16 KB, patch)
2013-04-25 19:13 UTC, Robert Rati
no flags Details | Diff

Description gil cattaneo 2012-05-19 09:21:23 UTC
Spec URL: http://gil.fedorapeople.org/zookeeper.spec
SRPM URL: http://gil.fedorapeople.org/zookeeper-3.4.3-1.fc16.src.rpm
Description: ZooKeeper is a centralized service for
maintaining configuration information,
naming, providing distributed synchronization,
and providing group services.

Comment 1 Jeffrey C. Ollie 2012-10-12 14:29:08 UTC
Gil, are you still interested in packaging this?  If you are, could you update the spec to 3.4.4 and I'll dust off my reviewing skills and do a review.

Comment 2 gil cattaneo 2012-10-12 14:40:37 UTC
hi Jeffrey,
sure, now work on this, but if want take the ownership no problems
thanks

Comment 3 gil cattaneo 2012-10-12 17:13:49 UTC
Spec URL: http://gil.fedorapeople.org/zookeeper/1/zookeeper.spec
SRPM URL: http://gil.fedorapeople.org/zookeeper/1/zookeeper-3.4.4-1.fc16.src.rpm

- update to 3.4.4
- disable rat-lib and jdiff support

Comment 4 Jeffrey C. Ollie 2012-10-12 17:53:14 UTC
Have you tested this on F17?  On F17 I need to add 'hostname' to the list of BuildRequires.  Once I've done that I get the following during the %check:

/builddir/build/BUILD/zookeeper-3.4.4/build.xml:1133: Problem: failed to create task or type junit
Cause: the class org.apache.tools.ant.taskdefs.optional.junit.JUnitTask was not found.
        This looks like one of Ant's optional components.
Action: Check that the appropriate optional JAR exists in
        -/usr/share/ant/lib
        -/builddir/.ant/lib
        -a directory added on the command line with the -lib argument
Do not panic, this is a common problem.
The commonest cause is a missing JAR.
This is not a bug; it is a configuration problem
Total time: 42 seconds

Comment 5 gil cattaneo 2012-10-12 18:03:51 UTC
no i built only in my f16 system
this problem should fixed adding ant-junit as BR

Comment 7 Michael S. 2012-10-30 15:18:26 UTC
Doesn't build on f18, a require on hostname is missing. It comes from build.xml, when creating zookeeper.jar.

Comment 9 Michael S. 2012-10-30 22:47:29 UTC
Why is the rpm splitted in several rpms ? I see there is only 1 package on debian/ubuntu.

And I think you also miss a systemd file for the server.

Comment 10 gil cattaneo 2012-10-31 06:55:52 UTC
hi Michael,
isn't true. debian for e.g. split zookepeer in several packages
see http://packages.debian.org/source/sid/zookeeper
for now i haven't intention to add a systemd script, 
use only java package just for import:
 http://hbase.apache.org/
 http://incubator.apache.org/hama/
 http://whirr.apache.org/
 http://www.hibernate.org/subprojects/search.html

thanks

Comment 12 gil cattaneo 2013-04-04 02:13:14 UTC
Spec URL: http://gil.fedorapeople.org/zookeeper/4/zookeeper.spec
SRPM URL: http://gil.fedorapeople.org/zookeeper/4/zookeeper-3.4.5-1.fc18.src.rpm

- enable zktreeutil and zkpython sub-project

Comment 13 Robert Rati 2013-04-23 16:47:07 UTC
Created attachment 739085 [details]
Adds building/packaging of the zookeeper-test.jar

Some packages may rely upon the test sources from zookeeper.  The upstream distribution provides the ability to generate a jar of test sources with the test-jar target.  This patch adds that target and packages the resulting jar in the zookeeper-java sub-package.

Comment 14 gil cattaneo 2013-04-23 17:07:21 UTC
Spec URL: http://gil.fedorapeople.org/zookeeper/4/zookeeper.spec
SRPM URL: http://gil.fedorapeople.org/zookeeper/4/zookeeper-3.4.5-2.fc18.src.rpm

- building/packaging of the zookeeper-test.jar

Comment 15 Paul P Komkoff Jr 2013-04-24 18:36:32 UTC
The problem that I think I have with this package I think can be phrased as: where is the server?
if I install zookeeper, there's no server.
In zookeeper-java there's a server but no service unit.
I think the logical thing would be to have
zookeeper - server and systemd service unit
zookeeper-cli (or -tools) - cli and zktreeutil and probably loadtest
zookeeper-lib - .so libraries
zookeeper-devel - everything else needed for development

That way, there will be much less confusion on what to install/require, it'll be mostly along the lines of other stuff in fedora.

Comment 16 Jeffrey C. Ollie 2013-04-24 18:42:15 UTC
Created attachment 739568 [details]
systemd service file for zookeeper

(In reply to comment #15)
> The problem that I think I have with this package I think can be phrased as:
> where is the server?

I agree.  I've attached what I'm using for a systemd unit file.  The paths will need to be edited to suit Fedora packaging standards

Comment 17 Robert Rati 2013-04-25 19:13:43 UTC
Created attachment 740026 [details]
Updated patch for zookeeper-test jar

This patch updates the changes for the zookeeper-test jar.  It puts the test jar in it's own subpackage and creates a pom and fragments so it can be found by mvn-rpmbuild

Comment 18 gil cattaneo 2013-04-27 10:46:21 UTC
Spec URL: http://gil.fedorapeople.org/zookeeper/5/zookeeper.spec
SRPM URL: http://gil.fedorapeople.org/zookeeper/5/zookeeper-3.4.5-3.fc18.src.rpm

- built ZooInspector - Browser and Editor for ZooKeeper Instances (requires jtoaster rhbz#957337)
- added additional poms files

Comment 19 Björn 'besser82' Esser 2013-06-11 14:07:28 UTC
Package has some critical issues.

#####

Package Review
==============

Legend:
[x] = Pass
[!] = Fail
[-] = Not applicable
[?] = Not evaluated
[ ] = Manual review needed


Issues:
=======
- Spec-file builds daemon/server app without hardening-features
  Note: Missing: %global _hardened_build 1
  See: https://fedoraproject.org/wiki/Packaging/Guidelines#PIE

  ---> enable hardened-builds

- Fully versioned dependency in subpackages, if present.
  Note: Missing: Requires: %{name} = %{version}-%{release} in zookeeper-lib,
  zookeeper-lib-devel, zookeeper-java
  See: http://fedoraproject.org/wiki/Packaging/Guidelines#RequiringBasePackage

  ---> false positive:
         * main-pkg requires lib-subpkg
         * lib-subpkg will work without main-pkg
         * java-subpkg only provides interface to main-pkg-server
           and might be installed on other device

  ---> "real" issue:
         * lib-devel-subpkg should Requires: %{name}-lib%{?_isa} = %{version}-%{release}

- Packages have proper BuildRequires/Requires on jpackage-utils
  See: https://fedoraproject.org/wiki/Packaging:Java

  ---> false positive:
         * BR: jpackage-utils is present in spec
         * java-subpkg has proper auto-requires

- License field in the package spec

  ---> License: APL 2.0 and BSD

- Package consistently uses macros

  ---> %{name} hardcoded in:
         * URL:           http://zookeeper.apache.org/
         * Source3:       zookeeper.service
         * %build
         * files listed in python-%{name}

- Large documentation must go in a -doc subpackage.

  ---> please move %doc src/c/docs/html/* from libs-devel to libs-doc


===== MUST items =====

C/C++:
[x]: Package does not contain kernel modules.
[x]: Package contains no static executables.
[x]: Header files in -devel subpackage, if present.
[x]: ldconfig called in %post and %postun if required.
[x]: Package does not contain any libtool archives (.la)
[x]: Rpath absent or only used for internal libs.
[x]: Development (unversioned) .so files in -devel subpackage, if present.

Generic:
[x]: Package is licensed with an open-source compatible license and meets
     other legal requirements as defined in the legal section of Packaging
     Guidelines.
[x]: %build honors applicable compiler flags or justifies otherwise.
[x]: Package contains no bundled libraries without FPC exception.
[x]: Changelog in prescribed format.
[x]: Sources contain only permissible code or content.
[!]: Each %files section contains %defattr if rpm < 4.4
     Note: %defattr present but not needed

     ---> please remove

[-]: Package contains desktop file if it is a GUI application.
[x]: Development files must be in a -devel package
[x]: Package requires other packages for directories it uses.
[x]: Package uses nothing in %doc for runtime.
[x]: Package is not known to require ExcludeArch.
[!]: Fully versioned dependency in subpackages, if present.

     ---> see above

[!]: Package complies to the Packaging Guidelines

     ---> server app without PIE and full RELRO

[!]: License field in the package spec file matches the actual license.
     Note: Checking patched sources after %prep for licenses. Licenses found:
     "Apache (v2.0)", "BSD (2 clause)", "Unknown or generated", "*No
     copyright* Apache (v2.0)". 79 files have unknown license. Detailed output
     of licensecheck in
     /home/bjoern.esser/fedora/review/823122-zookeeper/licensecheck.txt

     ---> License is APL 2.0 and BSD

[x]: License file installed when any subpackage combination is installed.
[!]: Package consistently uses macro is (instead of hard-coded directory
     names).

     ---> %{name} hardcoded in:
            * URL:           http://zookeeper.apache.org/
            * Source3:       zookeeper.service
            * %build
            * files listed in python-%{name}

[x]: Package is named according to the Package Naming Guidelines.
[x]: Package does not generate any conflict.
[x]: Package obeys FHS, except libexecdir and /usr/target.
[-]: If the package is a rename of another package, proper Obsoletes and
     Provides are present.
[x]: Package must own all directories that it creates.
[x]: Package does not own files or directories owned by other packages.
[!]: Requires correct, justified where necessary.

     ---> lib-devel, see above

[x]: Spec file is legible and written in American English.
[x]: Package contains systemd file(s) if in need.
[x]: Useful -debuginfo package or justification otherwise.
[!]: Large documentation must go in a -doc subpackage.
     Note: Documentation size is 819200 bytes in 73 files.

     ---> please move %doc src/c/docs/html/* from libs-devel to libs-doc

[x]: All build dependencies are listed in BuildRequires, except for any that
     are listed in the exceptions section of Packaging Guidelines.
[x]: Package does not run rm -rf %{buildroot} (or $RPM_BUILD_ROOT) at the
     beginning of %install.
[x]: Macros in Summary, %description expandable at SRPM build time.
[x]: Package does not contain duplicates in %files.
[x]: Permissions on files are set properly.
[x]: If (and only if) the source package includes the text of the license(s)
     in its own file, then that file, containing the text of the license(s)
     for the package is included in %doc.
[x]: Package use %makeinstall only when make install' ' DESTDIR=... doesn't
     work.
[x]: Package is named using only allowed ASCII characters.
[x]: Package do not use a name that already exist
[x]: Package is not relocatable.
[x]: Sources used to build the package match the upstream source, as provided
     in the spec URL.
[x]: Spec file name must match the spec package %{name}, in the format
     %{name}.spec.
[x]: File names are valid UTF-8.
[x]: Packages must not store files under /srv, /opt or /usr/local
[x]: Package successfully compiles and builds into binary rpms on at least one
     supported primary architecture.
[x]: Package installs properly.
[x]: Rpmlint is run on all rpms the build produces.
     Note: There are rpmlint messages (see attachment).

Java:
[x]: Javadoc documentation files are generated and included in -javadoc
     subpackage
[x]: Javadoc subpackages should not have Requires: jpackage-utils
[x]: Javadocs are placed in %{_javadocdir}/%{name} (no -%{version} symlink)
[x]: Bundled jar/class files should be removed before build

Maven:
[x]: Pom files have correct Maven mapping
     Note: Some add_maven_depmap calls found. Please check if they are correct
     or update to latest guidelines
[x]: If package contains pom.xml files install it (including depmaps) even
     when building with ant
[x]: Old add_to_maven_depmap macro is not being used
[x]: Packages DOES NOT have Requires(post) and Requires(postun) on jpackage-
     utils for %update_maven_depmap macro
[x]: Package DOES NOT use %update_maven_depmap in %post/%postun
[x]: Packages use %{_mavenpomdir} instead of %{_datadir}/maven2/poms

===== SHOULD items =====

Generic:
[x]: Uses parallel make.
[-]: If the source package does not include license text(s) as a separate file
     from upstream, the packager SHOULD query upstream to include it.
[!]: Final provides and requires are sane (see attachments).

     ---> see above

[x]: Package functions as described.
[x]: Latest version is packaged.
[x]: Package does not include license text files separate from upstream.
[x]: Patches link to upstream bugs/comments/lists or are otherwise justified.
[x]: Scriptlets must be sane, if used.
[-]: Description and summary sections in the package spec file contains
     translations for supported Non-English languages, if available.
[x]: Package should compile and build into binary rpms on all supported
     architectures.
[x]: %check is present and all tests pass.
[x]: Packages should try to preserve timestamps of original installed files.
[x]: Packager, Vendor, PreReq, Copyright tags should not be in spec file
[x]: Sources can be downloaded from URI in Source: tag
[x]: Reviewer should test that the package builds in mock.
[x]: Buildroot is not present
[x]: Package has no %clean section with rm -rf %{buildroot} (or
     $RPM_BUILD_ROOT)
[x]: Dist tag is present.
[x]: No file requires outside of /etc, /bin, /sbin, /usr/bin, /usr/sbin.
[x]: SourceX tarball generation or download is documented.
[x]: SourceX is a working URL.
[x]: Spec use %global instead of %define.

Java:
[x]: Packages are noarch unless they use JNI
     Note: zookeeper subpackage is not noarch. Please verify manually
[x]: Package uses upstream build method (ant/maven/etc.)

===== EXTRA items =====

Generic:
[!]: Large data in /usr/share should live in a noarch subpackage if package is
     arched.

     ---> html-docs in lib-devel should be in lib-doc

[x]: Rpmlint is run on all installed packages.
     Note: There are rpmlint messages (see attachment).
[x]: Spec file according to URL is the same as in SRPM.


Rpmlint
-------
Checking: zookeeper-3.4.5-3.fc20.x86_64.rpm
          zookeeper-lib-3.4.5-3.fc20.x86_64.rpm
          zookeeper-lib-devel-3.4.5-3.fc20.x86_64.rpm
          zookeeper-java-3.4.5-3.fc20.noarch.rpm
          zookeeper-javadoc-3.4.5-3.fc20.noarch.rpm
zookeeper.x86_64: W: no-manual-page-for-binary load_gen
zookeeper.x86_64: W: no-manual-page-for-binary cli_mt
zookeeper.x86_64: W: no-manual-page-for-binary cli_st
zookeeper.x86_64: W: no-manual-page-for-binary zktreeutil
5 packages and 0 specfiles checked; 0 errors, 4 warnings.




Rpmlint (installed packages)
----------------------------
# rpmlint zookeeper-javadoc zookeeper zookeeper-lib zookeeper-lib-d 
evel zookeeper-java
zookeeper.x86_64: W: no-manual-page-for-binary load_gen
zookeeper.x86_64: W: no-manual-page-for-binary cli_mt
zookeeper.x86_64: W: no-manual-page-for-binary cli_st
zookeeper.x86_64: W: no-manual-page-for-binary zktreeutil
5 packages and 0 specfiles checked; 0 errors, 4 warnings.
# echo 'rpmlint-done:'



Requires
--------
zookeeper-javadoc (rpmlib, GLIBC filtered):
    jpackage-utils

zookeeper (rpmlib, GLIBC filtered):
    libc.so.6()(64bit)
    libdl.so.2()(64bit)
    libgcc_s.so.1()(64bit)
    libgcc_s.so.1(GCC_3.0)(64bit)
    liblog4cxx.so.10()(64bit)
    libm.so.6()(64bit)
    libpthread.so.0()(64bit)
    libstdc++.so.6()(64bit)
    libstdc++.so.6(CXXABI_1.3)(64bit)
    libxml2.so.2()(64bit)
    libxml2.so.2(LIBXML2_2.4.30)(64bit)
    libxml2.so.2(LIBXML2_2.6.0)(64bit)
    libz.so.1()(64bit)
    libzookeeper_mt.so.2()(64bit)
    libzookeeper_st.so.2()(64bit)
    rtld(GNU_HASH)

zookeeper-lib (rpmlib, GLIBC filtered):
    /sbin/ldconfig
    libc.so.6()(64bit)
    libm.so.6()(64bit)
    libpthread.so.0()(64bit)
    rtld(GNU_HASH)

zookeeper-lib-devel (rpmlib, GLIBC filtered):
    libzookeeper_mt.so.2()(64bit)
    libzookeeper_st.so.2()(64bit)
    zookeeper-lib

zookeeper-java (rpmlib, GLIBC filtered):
    checkstyle
    java
    jline
    jpackage-utils
    jtoaster
    junit
    log4j
    mockito
    netty
    slf4j



Provides
--------
zookeeper-javadoc:
    zookeeper-javadoc

zookeeper:
    zookeeper
    zookeeper(x86-64)

zookeeper-lib:
    libzookeeper_mt.so.2()(64bit)
    libzookeeper_st.so.2()(64bit)
    zookeeper-lib
    zookeeper-lib(x86-64)

zookeeper-lib-devel:
    zookeeper-lib-devel
    zookeeper-lib-devel(x86-64)

zookeeper-java:
    mvn(org.apache.zookeeper:zookeeper)
    mvn(org.apache.zookeeper:zookeeper-ZooInspector)
    mvn(org.apache.zookeeper:zookeeper-test)
    osgi(org.apache.hadoop.zookeeper)
    zookeeper-java



Source checksums
----------------
http://www.apache.org/dist/zookeeper/stable/zookeeper-3.4.5.tar.gz :
  CHECKSUM(SHA256) this package     : e92b634e99db0414c6642f6014506cc22eefbea42cc912b57d7d0527fb7db132
  CHECKSUM(SHA256) upstream package : e92b634e99db0414c6642f6014506cc22eefbea42cc912b57d7d0527fb7db132

Built with local dependencies:
    /home/bjoern.esser/fedora/review/rpm/jtoaster-1.0.5-1.fc20.noarch.rpm


Generated by fedora-review 0.4.1 (b2e211f) last change: 2013-04-29
Buildroot used: fedora-rawhide-x86_64
Command line :/usr/bin/fedora-review -L /home/bjoern.esser/fedora/review/rpm/ -m fedora-rawhide-x86_64 -b 823122

#####

Please fix and I'll grant review.

Comment 20 gil cattaneo 2013-06-11 14:49:58 UTC
Thanks!

Spec URL: http://gil.fedorapeople.org/zookeeper/6/zookeeper.spec
SRPM URL: http://gil.fedorapeople.org/zookeeper/6/zookeeper-3.4.5-4.fc18.src.rpm

- enabled hardened-builds
- fixed fully versioned dependency in subpackages (lib-devel and python)
- fixed License tag
- moved large documentation in lib-doc subpackage

Comment 21 Björn 'besser82' Esser 2013-06-11 15:10:41 UTC
Everything is fine now, but one little thing:

python-ZooKeeper.x86_64: E: non-standard-executable-perm /usr/lib64/python2.7/site-packages/zookeeper.so 0775L

adding

 pushd src/contrib/zkpython
 %{__python} src/python/setup.py build --build-base=$PWD/build \
 install --root=%{buildroot} ;\
+chmod 0755 %{buildroot}%{python_sitearch}/zookeeper.so
 popd

 find %{buildroot} -name '*.la' -exec rm -f {} ';'

should fix.

You can do this in SCM, then.

APPROVED!

Comment 22 gil cattaneo 2013-06-11 16:01:44 UTC
Thanks!

Spec URL: http://gil.fedorapeople.org/zookeeper.spec
SRPM URL: http://gil.fedorapeople.org/zookeeper-3.4.5-5.fc18.src.rpm

- fixed zookeeper.so non-standard-executable-perm

New Package SCM Request
=======================
Package Name: zookeeper
Short Description: A high-performance coordination service for distributed applications
Owners: gil
Branches: f18 f19 f20
InitialCC: java-sig

Comment 23 Gwyn Ciesla 2013-06-11 16:09:13 UTC
WARNING: Invalid branch f20 requested

Comment 24 gil cattaneo 2013-06-11 16:15:09 UTC
New Package SCM Request
=======================
Package Name: zookeeper
Short Description: A high-performance coordination service for distributed applications
Owners: gil
Branches: f18 f19
InitialCC: java-sig

Comment 25 Gwyn Ciesla 2013-06-11 16:27:13 UTC
Git done (by process-git-requests).

Comment 26 Fedora Update System 2013-06-12 19:41:03 UTC
zookeeper-3.4.5-5.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/zookeeper-3.4.5-5.fc19

Comment 27 Fedora Update System 2013-06-13 18:07:43 UTC
zookeeper-3.4.5-5.fc19 has been pushed to the Fedora 19 testing repository.

Comment 28 Fedora Update System 2013-06-14 12:28:51 UTC
zookeeper-3.4.5-6.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/zookeeper-3.4.5-6.fc19

Comment 29 Fedora Update System 2013-06-29 11:08:07 UTC
zookeeper-3.4.5-5.fc18 has been submitted as an update for Fedora 18.
https://admin.fedoraproject.org/updates/zookeeper-3.4.5-5.fc18

Comment 30 Fedora Update System 2013-06-29 18:40:53 UTC
zookeeper-3.4.5-6.fc19 has been pushed to the Fedora 19 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 31 Fedora Update System 2013-07-09 01:40:08 UTC
zookeeper-3.4.5-5.fc18 has been pushed to the Fedora 18 stable repository.


Note You need to log in before you can comment on or make changes to this bug.