Bug 986615 - Review Request: openstack-savanna - Apache Hadoop cluster management on OpenStack
Summary: Review Request: openstack-savanna - Apache Hadoop cluster management on OpenS...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: Package Review
Version: rawhide
Hardware: All
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Nobody's working on this, feel free to take it
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: bigdata-review
TreeView+ depends on / blocked
 
Reported: 2013-07-21 01:48 UTC by Matthew Farrellee
Modified: 2013-10-04 20:06 UTC (History)
4 users (show)

Fixed In Version: openstack-savanna-0.2-3.fc19
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2013-09-03 22:28:26 UTC
Type: ---
Embargoed:
p: fedora-review+
gwync: fedora-cvs+


Attachments (Terms of Use)

Description Matthew Farrellee 2013-07-21 01:48:52 UTC
Spec URL: http://matt.fedorapeople.org/savanna.spec
SRPM URL: http://matt.fedorapeople.org/savanna-0.2-1.fc19.src.rpm
Description: Savanna provides the ability to elastically manage Apache Hadoop clusters on OpenStack.
Fedora Account System Username: matt

Comment 1 Christopher Meng 2013-07-21 02:57:00 UTC
Why Requires(post): systemd-unit?

Comment 2 Matthew Farrellee 2013-07-21 20:11:08 UTC
(In reply to Christopher Meng from comment #1)
> Why Requires(post): systemd-unit?

Following the example of some of the openstack-* packages.

I see that systemd.spec Provides systemd-units, since Jan 2012 when it was merged w/ the main package.

Would it be more appropriate to require just "systemd"?

Comment 3 Christopher Meng 2013-07-22 00:44:53 UTC
Yes, please see:

http://fedoraproject.org/wiki/Packaging:ScriptletSnippets#Systemd

Comment 4 Matthias Runge 2013-07-22 10:14:20 UTC
Matt, please move everything build related to %build section. Esp. 

export PYTHONPATH=$PWD:${PYTHONPATH}
# Note: json warnings likely resolved w/ pygments 1.5 (not yet in Fedora)
sphinx-build doc/source html
rm -rf html/.{doctrees,buildinfo}

is something to be done during build and not at install section.

Comment 5 Matthew Farrellee 2013-07-22 10:40:08 UTC
(In reply to Christopher Meng from comment #3)
> Yes, please see:
> 
> http://fedoraproject.org/wiki/Packaging:ScriptletSnippets#Systemd

That's much nicer.

Done.

Comment 6 Matthew Farrellee 2013-07-22 10:40:39 UTC
(In reply to Matthias Runge from comment #4)
> Matt, please move everything build related to %build section. Esp. 
> 
> export PYTHONPATH=$PWD:${PYTHONPATH}
> # Note: json warnings likely resolved w/ pygments 1.5 (not yet in Fedora)
> sphinx-build doc/source html
> rm -rf html/.{doctrees,buildinfo}
> 
> is something to be done during build and not at install section.

Thanks for catching that.

Done.

Comment 7 Pádraig Brady 2013-07-23 14:35:28 UTC
Since this is an OpenStack specific package, please rename it openstack-savanna

Given the distributed nature of OpenStack it's probably best to allocate static UID. You can see the existing openstack set of IDs, so just pick the next one and use that in the package. Also file the ticket but no need to wait for it to be granted

Also when doing review updates, just post new links with new versions

thanks!

Comment 8 Matthew Farrellee 2013-07-23 18:15:02 UTC
(In reply to Pádraig Brady from comment #7)
> Since this is an OpenStack specific package, please rename it
> openstack-savanna

Done.

> Given the distributed nature of OpenStack it's probably best to allocate
> static UID. You can see the existing openstack set of IDs, so just pick the
> next one and use that in the package. Also file the ticket but no need to
> wait for it to be granted

Done, https://bugzilla.redhat.com/show_bug.cgi?id=987591

I'll wait for the uid/gid to be assigned before claiming it in the package.

 
> Also when doing review updates, just post new links with new versions

Done,

Spec URL: http://matt.fedorapeople.org/pkg/1/openstack-savanna.spec
SRPM URL: http://matt.fedorapeople.org/pkg/1/openstack-savanna-0.2-1.fc19.src.rpm

Comment 9 Matthew Farrellee 2013-08-09 18:50:00 UTC
FYI, I have retracted the static uid/gid request pending better real-world need - https://fedorahosted.org/fpc/ticket/322

Comment 10 Pádraig Brady 2013-08-09 19:13:04 UTC
Are you changing to sqlite by default?

sed -i -e "s,^connection=.*,connection=sqlite:///$HOME/savanna-server.db," $CONF

That would be a departure from other openstack services that use mysql by default

Comment 11 Matthew Farrellee 2013-08-09 19:25:08 UTC
Savanna currently uses sqlite by default. The sample conf file places the db in sqlite:////tmp/savanna-server.db. I'm simply changing the db location to be more stable storage.

Comment 12 Pádraig Brady 2013-08-09 19:31:54 UTC
The tests/ dir should be removed from the final rpm

Did you try the tests in a %check section?
Was there a particular issue causing them to fail?

Comment 13 Matthew Farrellee 2013-08-09 19:52:26 UTC
I've added an rm -rf to %install to remove the tests.

A run of setup.py test from %check fails w/ pkg_resources.VersionConflict: (SQLAlchemy 0.8.1 (/usr/lib64/python2.7/site-packages), Requirement.parse('sqlalchemy>=0.7,<=0.7.99'))

Savanna works w/ sqlalchemy 0.8.1, but it pulls the requirements.txt to match OpenStack's.

I could probably patch the sources for this release, but the next (already committed to savanna master) will have "SQLAlchemy>=0.7.8,<=0.8.99" and I could enable %check then. IIRC< it'll also require adding the Requires as BuildRequires.

Comment 14 Pádraig Brady 2013-08-09 20:05:24 UTC
In general one should remove the python runtime dependency stuff.
It's redundant and problematic with rpm.
so in %prep please:

  rm -f requirements.txt

If %check works then, please include.
Please post new links to srpms with bumped release when ready. thanks

Comment 15 Matthew Farrellee 2013-08-09 20:28:41 UTC
I've added an rm for tools/pip-require

%check also requires unittest2 and mock.

setup.py test runs both unit and system tests. The system tests fail trying to connect to a non-existent keystone service and reading non-existent /etc/savanna/savanna.conf.

I'll upload a new spec & srpm w/ release bumped that has %check commented out.

Comment 17 Pádraig Brady 2013-08-09 20:39:44 UTC
You get a build error unless

s/Needed by %check/Needed by %%check/

Comment 18 Matthew Farrellee 2013-08-09 20:47:41 UTC
rpm-build-4.11.1-1.fc19.x86_64 must be more forgiving, I'm not getting any errors.

I've updated the spec/srpm in place.

Comment 19 Pádraig Brady 2013-08-09 20:49:32 UTC
Looks good, thanks!

Package Review
==============

Key:
[x] = Pass
[!] = Fail
[-] = Not applicable
[?] = Not evaluated
[ ] = Manual review needed


===== MUST items =====

Generic:
[x]: Package is licensed with an open-source compatible license and meets
     other legal requirements as defined in the legal section of Packaging
     Guidelines.
[x]: All build dependencies are listed in BuildRequires, except for any that
     are listed in the exceptions section of Packaging Guidelines.
     Note: Using prebuilt rpms.
[x]: Package contains no bundled libraries without FPC exception.
[x]: Changelog in prescribed format.
[x]: Package does not run rm -rf %{buildroot} (or $RPM_BUILD_ROOT) at the
     beginning of %install.
     Note: rm -rf %{buildroot} present but not required
[x]: Sources contain only permissible code or content.
[-]: Package contains desktop file if it is a GUI application.
[-]: Development files must be in a -devel package
[x]: Package requires other packages for directories it uses.
[-]: Package uses nothing in %doc for runtime.
[x]: Package is not known to require ExcludeArch.
[x]: Package complies to the Packaging Guidelines
[x]: Package consistently uses macro is (instead of hard-coded directory
     names).
[x]: Package is named according to the Package Naming Guidelines.
[x]: Package does not generate any conflict.
[x]: Package obeys FHS, except libexecdir and /usr/target.
[-]: If the package is a rename of another package, proper Obsoletes and
     Provides are present.
[x]: Package must own all directories that it creates.
[x]: Package does not own files or directories owned by other packages.
[x]: Requires correct, justified where necessary.
[x]: Spec file is legible and written in American English.
[x]: Package contains systemd file(s) if in need.
[x]: Large documentation must go in a -doc subpackage.
     Note: Documentation size is 829440 bytes in 72 files.
[-]: Package successfully compiles and builds into binary rpms on at least one
     supported primary architecture.
[x]: %config files are marked noreplace or the reason is justified.
[x]: Each %files section contains %defattr if rpm < 4.4
[x]: Macros in Summary, %description expandable at SRPM build time.
[x]: Package does not contain duplicates in %files.
[x]: Permissions on files are set properly.
[x]: Fully versioned dependency in subpackages, if present.
[x]: Spec file lacks Packager, Vendor, PreReq tags.
[x]: If (and only if) the source package includes the text of the license(s)
     in its own file, then that file, containing the text of the license(s)
     for the package is included in %doc.
[x]: Package use %makeinstall only when make install' ' DESTDIR=... doesn't
     work.
[x]: Package is named using only allowed ASCII characters.
[x]: No %config files under /usr.
[x]: Package do not use a name that already exist
[x]: Package is not relocatable.
[x]: Sources used to build the package match the upstream source, as provided
     in the spec URL.
[x]: Spec file name must match the spec package %{name}, in the format
     %{name}.spec.
[x]: File names are valid UTF-8.
[x]: Packages must not store files under /srv, /opt or /usr/local
[x]: Rpmlint is run on all rpms the build produces.
     Note: There are rpmlint messages (see attachment).

Python:
[x]: Binary eggs must be removed in %prep
     Note: Cannot find any build in BUILD directory (--prebuilt option?)
[x]: Python eggs must not download any dependencies during the build process.
[x]: A package which is used by another package via an egg interface should
     provide egg info.
[x]: Package meets the Packaging Guidelines::Python
[x]: Package contains BR: python2-devel or python3-devel

===== SHOULD items =====

Generic:
[x]: Reviewer should test that the package builds in mock.
[-]: If the source package does not include license text(s) as a separate file
     from upstream, the packager SHOULD query upstream to include it.
[x]: Final provides and requires are sane (see attachments).
[x]: Package functions as described.
[x]: Latest version is packaged.
[x]: Package does not include license text files separate from upstream.
[x]: Description and summary sections in the package spec file contains
     translations for supported Non-English languages, if available.
[-]: Package should compile and build into binary rpms on all supported
     architectures.
[!]: %check is present and all tests pass.
[x]: Packages should try to preserve timestamps of original installed files.
[x]: Sources can be downloaded from URI in Source: tag
[x]: Buildroot is not present
[x]: Package has no %clean section with rm -rf %{buildroot} (or
     $RPM_BUILD_ROOT)
[x]: Dist tag is present.
[x]: No file requires outside of /etc, /bin, /sbin, /usr/bin, /usr/sbin.
[x]: SourceX tarball generation or download is documented.
[x]: SourceX is a working URL.
[x]: Spec use %global instead of %define.

===== EXTRA items =====

Generic:
[x]: Rpmlint is run on all installed packages.
     Note: Mock build failed
[x]: Spec file according to URL is the same as in SRPM.
     Note: Spec file as given by url is not the same as in SRPM (see attached
     diff).
[x]: Large data in /usr/share should live in a noarch subpackage if package is
     arched.



Rpmlint
-------
Checking: openstack-savanna-0.2-2.fc19.noarch.rpm
openstack-savanna.noarch: W: non-standard-gid /etc/savanna/savanna.conf savanna
openstack-savanna.noarch: E: non-readable /etc/savanna/savanna.conf 0640L
openstack-savanna.noarch: W: non-standard-uid /var/lib/savanna savanna
openstack-savanna.noarch: W: non-standard-gid /var/lib/savanna savanna
openstack-savanna.noarch: E: non-standard-dir-perm /var/lib/savanna 0700L
openstack-savanna.noarch: W: no-manual-page-for-binary savanna-db-manage
openstack-savanna.noarch: W: no-manual-page-for-binary savanna-api
openstack-savanna.noarch: W: percent-in-%post
openstack-savanna.noarch: W: percent-in-%preun
openstack-savanna.noarch: W: percent-in-%postun
1 packages and 0 specfiles checked; 2 errors, 8 warnings.



Requires
--------
openstack-savanna (rpmlib, GLIBC filtered):
    /bin/sh
    /usr/bin/python
    config(openstack-savanna)
    python(abi)
    python-alembic
    python-cinderclient
    python-eventlet
    python-flask
    python-iso8601
    python-jsonschema
    python-keystoneclient
    python-netaddr
    python-novaclient
    python-oslo-config
    python-paramiko
    shadow-utils
    systemd



Provides
--------
openstack-savanna:
    config(openstack-savanna)
    openstack-savanna



MD5-sum check
-------------
http://tarballs.openstack.org/savanna/savanna-0.2.tar.gz :
  CHECKSUM(SHA256) this package     : 1d28a008d4eb545582b17945b06f4ecb20329d5d2e135526c0f46f788ea40438
  CHECKSUM(SHA256) upstream package : 1d28a008d4eb545582b17945b06f4ecb20329d5d2e135526c0f46f788ea40438
Using local file /home/padraig/rhat/fedora-scm/openstack/openstack-savanna/openstack-savanna-api.service as upstream
file:///home/padraig/rhat/fedora-scm/openstack/openstack-savanna/openstack-savanna-api.service :
  CHECKSUM(SHA256) this package     : ffa8c2d5d57f5ec3e3336fc3ba39384e1edf491176a23bbfb660c256bf9e7f80
  CHECKSUM(SHA256) upstream package : ffa8c2d5d57f5ec3e3336fc3ba39384e1edf491176a23bbfb660c256bf9e7f80


Generated by fedora-review 0.4.0 (660ce56) last change: 2013-01-29
Buildroot used: fedora-19-x86_64
Command line :/usr/bin/fedora-review -n openstack-savanna -p

Comment 20 Matthew Farrellee 2013-08-09 21:00:25 UTC
New Package SCM Request
=======================
Package Name: openstack-savanna
Short Description: Apache Hadoop cluster management on OpenStack
Owners: matt
Branches: f19
InitialCC:

Comment 21 Pádraig Brady 2013-08-10 00:08:49 UTC
New Package SCM Request
=======================
Package Name: openstack-savanna
Short Description: Apache Hadoop cluster management on OpenStack
Owners: matt
Branches: f19 el6
InitialCC:

Comment 22 Gwyn Ciesla 2013-08-12 12:47:15 UTC
Git done (by process-git-requests).

Comment 23 Fedora Update System 2013-08-12 19:55:49 UTC
openstack-savanna-0.2-3.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/openstack-savanna-0.2-3.fc19

Comment 24 Fedora Update System 2013-08-15 02:40:47 UTC
openstack-savanna-0.2-3.fc19 has been pushed to the Fedora 19 testing repository.

Comment 25 Fedora Update System 2013-09-03 22:28:26 UTC
openstack-savanna-0.2-3.fc19 has been pushed to the Fedora 19 stable repository.


Note You need to log in before you can comment on or make changes to this bug.