Bug 864464

This package is required for the Continuous Integration (CI) system of Openstack.  The various projects are moving from using nosetests and a venv to tox in order to keep in sync with CI.


Just confirmed that it builds on mine.  Just needed to install the Python3 dependencies.

Installed the RPM and attempted to run Tox for Keystone.  



Installed:
  python-tox.noarch 0:1.4.2-1.f17ayoung

$ tox
Traceback (most recent call last):
  File "/usr/bin/tox", line 5, in <module>
    from pkg_resources import load_entry_point
  File "/usr/lib/python3.2/site-packages/pkg_resources.py", line 2736, in <module>
    working_set.require(__requires__)
  File "/usr/lib/python3.2/site-packages/pkg_resources.py", line 690, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/lib/python3.2/site-packages/pkg_resources.py", line 588, in resolve
    raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: tox==1.4.2

At Matthias' request, I installed the python3-tox RPM, and the tests ran to completion.

I can't believe this submission just happened today! I spent last week creating my first package for Fedora and I went for python-tox.

Anyway, I'll put links to my specs and srpms just in case they can be useful.

First the python 2 only version:

# SPEC:
http://lorenzogil.com/rpms/python-tox/python-tox.spec
# SRPM:
http://lorenzogil.com/rpms/python-tox/python-tox-1.4.2-1.fc17.src.rpm
# Koji build:
http://koji.fedoraproject.org/koji/taskinfo?taskID=4575039


Then the python 2 & 3 version:

http://lorenzogil.com/rpms/python-tox/python-tox-py3.spec


IMHO the difficult part on this package is the %check phase (which is avoided in Matthias spec for good reasons). Tox tries to fetch some packages from Pypi in the 'python setup.py test' command since the tox.ini file has the following deps:

[testenv]
commands=py.test --junitxml={envlogdir}/junit-{envname}.xml {posargs}
deps=pytest
     py

It also uses virtualenv when running the tests. It doesn't matter if you already have the python-virtualenv package installed. This package is on the BuildRequires because is also needed in the python setup.py build step.

About the python 3 version I ended removing it because it would Requires python3-virtualenv and it does not exist. You can see this dependency hidden in the following line: https://bitbucket.org/hpk42/tox/src/b6d3b82545e4/tox/_venv.py?at=default#cl-173

If that line is ever called with a python3 interpreter you will get this error:

>>> import imp
>>> imp.find_module('virtualenv')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named virtualenv


I hope this information helps.

By the way, my rpmlint warnings are pretty similar to Matthias', except for the %check phase.

[makerpm@ticotico rpmbuild]$ rpmlint SRPMS/python-tox-1.4.2-1.fc17.src.rpm 
python-tox.src: W: spelling-error Summary(en_US) virtualenv -> virtual
python-tox.src: W: spelling-error %description -l en_US virtualenv -> virtual
python-tox.src: W: spelling-error %description -l en_US frontend -> fronted, front end, front-end
1 packages and 0 specfiles checked; 0 errors, 3 warnings.

(In reply to comment #3)
> At Matthias' request, I installed the python3-tox RPM, and the tests ran to
> completion.

That's great, thank you!

(In reply to comment #4)
> 
> Then the python 2 & 3 version:
> 
> http://lorenzogil.com/rpms/python-tox/python-tox-py3.spec
> 
> 
> IMHO the difficult part on this package is the %check phase (which is
> avoided in Matthias spec for good reasons). Tox tries to fetch some packages
> from Pypi in the 'python setup.py test' command since the tox.ini file has
> the following deps:
> 
> [testenv]
> commands=py.test --junitxml={envlogdir}/junit-{envname}.xml {posargs}
> deps=pytest
>      py
> 
> It also uses virtualenv when running the tests. It doesn't matter if you
> already have the python-virtualenv package installed. This package is on the
> BuildRequires because is also needed in the python setup.py build step.
>
Two remarks to your spec:
- You're mixing sources and requirements. I'd recomment, not to use these here, but just to require those as build requirements.
Source1:        http://pypi.python.org/packages/source/p/py/py-1.4.9.zip
Source2:        http://pypi.python.org/packages/source/p/pytest/pytest-2.2.4.zip
Source3:        http://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.8.2.tar.gz

in prep you made:
cp -a . %{py3dir}
find %{py3dir} -name '*.py' | xargs sed -i '1s|^#!python|#!%{__python3}|'
%endif # with_python3

find -name '*.py' | xargs sed -i '1s|^#!python|#!%{__python}|'

One find is too much.

Update:
- enabled tests also
SPEC: http://www.matthias-runge.de/fedora/python-tox.spec
SRPM: http://www.matthias-runge.de/fedora/python-tox-1.4.2-2.fc17.src.rpm

(In reply to comment #8)
> Update:
> - enabled tests also
> SPEC: http://www.matthias-runge.de/fedora/python-tox.spec
> SRPM: http://www.matthias-runge.de/fedora/python-tox-1.4.2-2.fc17.src.rpm

Your new SRPM is failing at Koji:

http://koji.fedoraproject.org/koji/taskinfo?taskID=4577449

The reason is the one I mentioned in my first comment: koji does can't do HTTP requests and Tox test suite does a bunch of them. Unless you trick it to believe there is a local package index inside the BUILD directory. Which is exactly what my package is doing with this lines:

Source1:        http://pypi.python.org/packages/source/p/py/py-1.4.9.zip
Source2:        http://pypi.python.org/packages/source/p/pytest/pytest-2.2.4.zip
Source3:        http://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.8.2.tar.gz

These are not build requirements. They are requirements to run tox test suite, not to build Tox.

As I said before, it is not trivial to run Tox tests inside Koji. I spent several hours to debug them.

Lorenzo,

BuildRequires means to build the RPM, as opposed to Requires, which is just for run time. Even the tox testsuite stage is considered part of the rpmbuild, and thus external packages like pypy and such should be used from the external system, not explicitly included as source.

Can you modify your spec to use the files from the base system, possibly via a symlink, instead of importing the sources?

(In reply to comment #10)
> Lorenzo,
> 
> BuildRequires means to build the RPM, as opposed to Requires, which is just
> for run time. Even the tox testsuite stage is considered part of the
> rpmbuild, and thus external packages like pypy and such should be used from
> the external system, not explicitly included as source.
> 
> Can you modify your spec to use the files from the base system, possibly via
> a symlink, instead of importing the sources?

Unfortunately Tox needs 'source distributions' of pytest, py and virtualenv. A Python 'source distribution' is different from a package installed on a system. For example, a source distribution contains the setup.py file, which is not included in the base system when you install the packages python-py, pytest or python-virtualenv.

Maybe the easiest solution would be to skip the %test phase for this package.

By the way, we still have the issue of the python3-tox package, which depends on the non existing package python3-virtualenv. Adam, you said that the tests run with the python3-tox package but I wonder if the tests does not cover the code path I spotted in the comment #4

(In reply to comment #9)

> Your new SRPM is failing at Koji:
> 
> http://koji.fedoraproject.org/koji/taskinfo?taskID=4577449
> 
> The reason is the one I mentioned in my first comment: koji does can't do
> HTTP requests and Tox test suite does a bunch of them. Unless you trick it
> to believe there is a local package index inside the BUILD directory. Which
> is exactly what my package is doing with this lines:
> 
Ah, I should've checked that, my bad.

Adding python-virtualenv as buildrequirement (to my version of that package) solves that problem (partly). Currently, I disabled python3 tests. I requested co-maintainership in the python-virtualenv-package to provide a python3-build.


SPEC: http://www.matthias-runge.de/fedora/python-tox.spec
SRPM: http://www.matthias-runge.de/fedora/python-tox-1.4.2-3.fc17.src.rpm

koji-scratchbuild: http://koji.fedoraproject.org/koji/taskinfo?taskID=4580712

and also:
[mrunge@turing SPECS]$ rpmlint ./python-tox.spec ../SRPMS/python-tox-1.4.2-3.fc17.src.rpm ../RPMS/noarch/python3-tox-1.4.2-3.fc17.noarch.rpm ../RPMS/noarch/python-tox-1.4.2-3.fc17.noarch.rpm 
./python-tox.spec:93: W: macro-in-comment %{__python}
./python-tox.spec:94: W: macro-in-comment %endif
python-tox.src: W: spelling-error Summary(en_US) Virtualenv -> Virtual
python-tox.src: W: spelling-error %description -l en_US virtualenv -> virtual
python-tox.src: W: spelling-error %description -l en_US frontend -> fronted, front end, front-end
python-tox.src:93: W: macro-in-comment %{__python}
python-tox.src:94: W: macro-in-comment %endif
python3-tox.noarch: W: spelling-error Summary(en_US) Virtualenv -> Virtual
python3-tox.noarch: W: spelling-error %description -l en_US virtualenv -> virtual
python3-tox.noarch: W: spelling-error %description -l en_US frontend -> fronted, front end, front-end
python-tox.noarch: W: spelling-error Summary(en_US) Virtualenv -> Virtual
python-tox.noarch: W: spelling-error %description -l en_US virtualenv -> virtual
python-tox.noarch: W: spelling-error %description -l en_US frontend -> fronted, front end, front-end
python-tox.noarch: W: no-manual-page-for-binary tox
3 packages and 1 specfiles checked; 0 errors, 14 warnings.

I tried building your new package in Koji and it still fails if the target is f17:

koji build --scratch f17 python-tox-1.4.2-3.fc17.src.rpm
http://koji.fedoraproject.org/koji/taskinfo?taskID=4580787

Your build is succesful because the target is rawhide and apparently, in rawhide the %check phase is not run. You can check that in your logs for that taskID.

If you see my build with your package, the relevant lines are these ones:

Downloading/unpacking pytest
  Cannot fetch index base URL http://pypi.python.org/simple/
  Could not find any downloads that satisfy the requirement pytest
No distributions at all found for pytest
Storing complete log in /builddir/.pip/pip.log
ERROR: could not install deps [pytest, py]



Also, I think python-virtualenv should be a Requires also.

Interesting

fails: f18: http://koji.fedoraproject.org/koji/taskinfo?taskID=4580792 
succeeds: rawhide: http://koji.fedoraproject.org/koji/taskinfo?taskID=4580798

The latter also executes check.

Executing(%check): /bin/sh -e /var/tmp/rpm-tmp.g7sz4A
+ umask 022
+ cd /builddir/build/BUILD
+ cd tox-1.4.2
+ exit 0

Agreed, python-virtualenv should be requirement.

update: conditionalize tests, as they require internet connection, and koji builders are not allowed to connect to the net.

To build with tests, run rpmbuild --with internet -ba <spec>

SPEC: http://www.matthias-runge.de/fedora/python-tox.spec
SRPM: http://www.matthias-runge.de/fedora/python-tox-1.4.2-4.fc17.src.rpm

I agree with Matthias' 1.4.2-4 solution about only running the tests when having Internet connection.

I think that's the best compromise in this case.

I built the rpm with the python-tox-1.4.2-4.fc17.src.rpm and installed it on my Fedora 17 box. After installing it, the tox script in /bin/tox looks like this:

#!/usr/bin/python3
# EASY-INSTALL-ENTRY-SCRIPT: 'tox==1.4.2','console_scripts','tox'
__requires__ = 'tox==1.4.2'
import sys
from pkg_resources import load_entry_point

if __name__ == '__main__':
    sys.exit(
        load_entry_point('tox==1.4.2', 'console_scripts', 'tox')()
    )

And if you try to use it you get this error:

Traceback (most recent call last):
  File "/usr/bin/tox", line 5, in <module>
    from pkg_resources import load_entry_point
  File "/usr/lib/python3.2/site-packages/pkg_resources.py", line 2736, in <module>
    working_set.require(__requires__)
  File "/usr/lib/python3.2/site-packages/pkg_resources.py", line 690, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/lib/python3.2/site-packages/pkg_resources.py", line 588, in resolve
    raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: tox==1.4.2

The tox libraries are installed in /usr/lib/python2.7/site-packages/tox/ but the tox script has the wrong shebang: #!/usr/bin/python3

To fix this problem we can do two things:

 A) Don't build with python3 if we are not creating the python3 package until we get the python3-virtualenv dependency
 B) Fix the shebangs in the %prep step as I did in my package with the 'find' commands.

(In reply to comment #17)

> The tox libraries are installed in /usr/lib/python2.7/site-packages/tox/ but
> the tox script has the wrong shebang: #!/usr/bin/python3
> 
> To fix this problem we can do two things:
> 
>  A) Don't build with python3 if we are not creating the python3 package
> until we get the python3-virtualenv dependency
>  B) Fix the shebangs in the %prep step as I did in my package with the
> 'find' commands.

Waah, this is nasty. 
Since both packages provide /usr/bin/tox, I'll skip the python3-part.

SPEC: http://www.matthias-runge.de/fedora/python-tox.spec
SRPM: http://www.matthias-runge.de/fedora/python-tox-1.4.2-5.fc17.src.rpm

Package Review
==============

Key:
[x] = Pass
[!] = Fail
[-] = Not applicable
[?] = Not evaluated
[ ] = Manual review needed



===== MUST items =====

Generic:
[x]: Package is licensed with an open-source compatible license and meets
     other legal requirements as defined in the legal section of Packaging
     Guidelines.
[x]: Package successfully compiles and builds into binary rpms on at least one
     supported primary architecture.
[-]: %build honors applicable compiler flags or justifies otherwise.
[x]: All build dependencies are listed in BuildRequires, except for any that
     are listed in the exceptions section of Packaging Guidelines.
[x]: Package contains no bundled libraries.
[x]: Changelog in prescribed format.
[x]: Package does not run rm -rf %{buildroot} (or $RPM_BUILD_ROOT) at the
     beginning of %install.
[x]: Sources contain only permissible code or content.
[x]: Each %files section contains %defattr if rpm < 4.4
[-]: Macros in Summary, %description expandable at SRPM build time.
[-]: Package contains desktop file if it is a GUI application.
[-]: Development files must be in a -devel package
[-]: Package requires other packages for directories it uses. 
[x]: Package uses nothing in %doc for runtime.
[x]: Package is not known to require ExcludeArch.
[x]: Package does not contain duplicates in %files.
[x]: Permissions on files are set properly.
[x]: Package complies to the Packaging Guidelines
[x]: Spec file lacks Packager, Vendor, PreReq tags.
[-]: Large documentation files are in a -doc subpackage, if required.
[x]: If (and only if) the source package includes the text of the license(s)
     in its own file, then that file, containing the text of the license(s)
[!]: License field in the package spec file matches the actual license.
     Note: Checking patched sources after %prep for licenses. Licenses found:
     "MIT/X11 (BSD like)", "Unknown or generated". 2 files have unknown
     license. Detailed output of licensecheck in /home/ndipanov/864464-python-
     tox/licensecheck.txt
[x]: Package consistently uses macro is (instead of hard-coded directory
     names).
[x]: Package is named using only allowed ASCII characters.
[x]: Package is named according to the Package Naming Guidelines.
[x]: Package does not generate any conflict.
     Note: Package contains no Conflicts: tag(s)
[x]: Package do not use a name that already exist
[x]: Package obeys FHS, except libexecdir and /usr/target.
[x]: If the package is a rename of another package, proper Obsoletes and
     Provides are present.
[x]: Package must own all directories that it creates.
[x]: Package does not own files or directories owned by other packages.
[x]: Package installs properly.
[x]: Package is not relocatable.
[x]: Requires correct, justified where necessary.
[x]: CheckResultdir
[x]: Rpmlint is run on all rpms the build produces.
     Note: There are rpmlint messages (see attachment).
[x]: Sources used to build the package match the upstream source, as provided
     in the spec URL.
[x]: Spec file is legible and written in American English.
[x]: Spec file name must match the spec package %{name}, in the format
     %{name}.spec.
[x]: Package contains systemd file(s) if in need.
[x]: File names are valid UTF-8.

Python:
[x]: Package contains BR: python2-devel or python3-devel

===== SHOULD items =====

Generic:
[x]: Reviewer should test that the package builds in mock.
[x]: Buildroot is not present
[x]: Package has no %clean section with rm -rf %{buildroot} (or
     $RPM_BUILD_ROOT)
[-]: If the source package does not include license text(s) as a separate file
     from upstream, the packager SHOULD query upstream to include it.
[x]: Dist tag is present.
[x]: No file requires outside of /etc, /bin, /sbin, /usr/bin, /usr/sbin.
[x]: Final provides and requires are sane (rpm -q --provides and rpm -q
     --requires).
[x]: Package functions as described.
[x]: Latest version is packaged.
[x]: Package does not include license text files separate from upstream.
[x]: The placement of pkgconfig(.pc) files are correct.
[x]: SourceX tarball generation or download is documented.
[!]: SourceX / PatchY prefixed with %{name}.
     Note: Source0 (tox-1.4.2.zip)
[x]: SourceX is a working URL.
[-]: Description and summary sections in the package spec file contains
     translations for supported Non-English languages, if available.
[x]: Package should compile and build into binary rpms on all supported
     architectures.
[x]: %check is present and all tests pass.
[x]: Packages should try to preserve timestamps of original installed files.
[x]: Spec use %global instead of %define.

===== EXTRA items =====

Generic:
[x]: Rpmlint is run on all installed packages.
     Note: There are rpmlint messages (see attachment).
[x]: Spec file according to URL is the same as in SRPM.


Rpmlint
-------
Checking: python-tox-1.4.2-5.fc19.src.rpm
          python-tox-1.4.2-5.fc19.noarch.rpm
python-tox.src: W: spelling-error Summary(en_US) Virtualenv -> Virtual
python-tox.src: W: spelling-error %description -l en_US virtualenv -> virtual
python-tox.src: W: spelling-error %description -l en_US frontend -> fronted, front end, front-end
python-tox.src:56: W: macro-in-comment %if
python-tox.src:57: W: macro-in-comment %{__python}
python-tox.src:58: W: macro-in-comment %endif
python-tox.noarch: W: spelling-error Summary(en_US) Virtualenv -> Virtual
python-tox.noarch: W: spelling-error %description -l en_US virtualenv -> virtual
python-tox.noarch: W: spelling-error %description -l en_US frontend -> fronted, front end, front-end
python-tox.noarch: W: no-manual-page-for-binary tox
2 packages and 0 specfiles checked; 0 errors, 10 warnings.




Rpmlint (installed packages)
----------------------------
# rpmlint python-tox
python-tox.noarch: W: spelling-error Summary(en_US) Virtualenv -> Virtual
python-tox.noarch: W: spelling-error %description -l en_US virtualenv -> virtual
python-tox.noarch: W: spelling-error %description -l en_US frontend -> fronted, front end, front-end
python-tox.noarch: W: no-manual-page-for-binary tox
1 packages and 0 specfiles checked; 0 errors, 4 warnings.
# echo 'rpmlint-done:'



Requires
--------
python-tox-1.4.2-5.fc19.noarch.rpm (rpmlib, GLIBC filtered):

    /usr/bin/python
    python(abi) = 2.7

Provides
--------
python-tox-1.4.2-5.fc19.noarch.rpm:

    python-tox = 1.4.2-5.fc19



MD5-sum check
-------------
http://pypi.python.org/packages/source/t/tox/tox-1.4.2.zip :
  CHECKSUM(SHA256) this package     : a8a5b3ad5ff0907c13203c5b36085f8dafbdd367e0c9211aa24797990a6f3d2f
  CHECKSUM(SHA256) upstream package : a8a5b3ad5ff0907c13203c5b36085f8dafbdd367e0c9211aa24797990a6f3d2f


Generated by fedora-review 0.3.0 (c78e275) last change: 2012-09-24

=========================

Mathias note that there is one file that is under MIT liscence (toxbootstrap.py
) so maybe this should be fixed with the upstream or dealt with somehow?

Otherwise looks good!

N.

Nicola, thank you for your review.

I totally agree, I should be more specific regarding the license

--- python-tox-5.spec	2012-10-18 08:57:10.391143083 +0200
+++ python-tox.spec	2012-10-18 09:00:54.379344127 +0200
@@ -7,10 +7,11 @@
 %global pypiname tox
 Name:           python-tox
 Version:        1.4.2
-Release:        5%{?dist}
+Release:        6%{?dist}
 Summary:        Virtualenv-based automation of test activities
 
-License:        GPLv2+
+# file toxbootstrap.py is licensed under MIT License
+License:        GPLv2+ and MIT
 URL:            http://codespeak.net/tox
 Source0:        http://pypi.python.org/packages/source/t/%{pypiname}/%{pypiname}-%{version}.zip
 #md5=dc52acf22eb57eaa33e704f8a54e7b34
@@ -71,6 +72,9 @@
 
 
 %changelog
+* Thu Oct 18 2012 Matthias Runge <mrunge> - 1.4.2-6
+- change license to GPLv2+ and MIT
+
 * Tue Oct 16 2012 Matthias Runge <mrunge> - 1.4.2-5
 - totally disable python3 support for now
 

New 
SPEC: http://www.matthias-runge.de/fedora/python-tox.spec
SRPM: http://www.matthias-runge.de/fedora/python-tox-1.4.2-6.fc17.src.rpm

Package APPROVED.

Looks good Matthias.

Nikola, thanks again for the review!

New Package SCM Request
=======================
Package Name: python-tox
Short Description: virtualenv-based automation of test activities 
Owners: mrunge
Branches: f18 f17 el6

Git done (by process-git-requests).

python-tox-1.4.2-6.fc18 has been submitted as an update for Fedora 18.
https://admin.fedoraproject.org/updates/python-tox-1.4.2-6.fc18

python-tox-1.4.2-6.fc17 has been submitted as an update for Fedora 17.
https://admin.fedoraproject.org/updates/python-tox-1.4.2-6.fc17

python-tox-1.4.2-6.el6 has been submitted as an update for Fedora EPEL 6.
https://admin.fedoraproject.org/updates/python-tox-1.4.2-6.el6

Package python-tox-1.4.2-6.fc18:
* should fix your issue,
* was pushed to the Fedora 18 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing python-tox-1.4.2-6.fc18'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2012-16408/python-tox-1.4.2-6.fc18
then log in and leave karma (feedback).