Bug 1662777 - Review Request: python-pytest-randomly - Pytest plugin to randomly order tests and control random.seed
Summary: Review Request: python-pytest-randomly - Pytest plugin to randomly order test...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: Package Review
Version: rawhide
Hardware: All
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Zbigniew Jędrzejewski-Szmek
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-01-01 22:19 UTC by Dan Callaghan
Modified: 2020-02-27 17:30 UTC (History)
4 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2020-02-27 16:44:34 UTC
Type: ---
Embargoed:
zbyszek: fedora-review+


Attachments (Terms of Use)

Description Dan Callaghan 2019-01-01 22:19:45 UTC
Spec URL: https://fedorapeople.org/~dcallagh/python-pytest-randomly/python-pytest-randomly.spec
SRPM URL: https://fedorapeople.org/~dcallagh/python-pytest-randomly/python-pytest-randomly-1.2.3-1.fc30.src.rpm
Description: Pytest-randomly is a pytest plugin to randomly order tests and control random.seed.
Fedora Account System Username: dcallagh

Comment 1 Hirotaka Wakabayashi 2019-01-23 14:41:20 UTC
Hello, this is an unofficial review. Please read it for your reference.

# Summary

1. Rpmlint on the src.rpm reported 1 errors, 0 warnings
2. Rpmlint on the binary rpm reported 2 errors, 0 warnings
3. %check should not be failed

# Details

## 1. Rpmlint on the src.rpm reported 1 errors, 0 warnings

no-changelogname-tag error found. You need add a changelog entry.
https://docs.fedoraproject.org/en-US/packaging-guidelines/#changelogs

Here is the output in my fedora29 environment:
```
python-pytest-randomly.src: E: no-changelogname-tag
There is no %changelog tag in your spec file. To insert it, just insert a
'%changelog' in your spec file and rebuild it.

1 packages and 0 specfiles checked; 1 errors, 0 warnings.
```

## 2. Rpmlint on the binary rpm reported 2 errors, 0 warnings

no-changelogname-tag error and non-readable error found.

1. no-changelogname-tag
You need add a changelog entry.
https://docs.fedoraproject.org/en-US/packaging-guidelines/#changelogs

2. non-readable
pytest_randomly.cpython-37-PYTEST.pyc is non-readable from users. Permissions
on files MUST be set properly.
https://docs.fedoraproject.org/en-US/packaging-guidelines/#_file_permissions

Here is the output in my fedora29 environment:
```
python3-pytest-randomly.noarch: E: no-changelogname-tag
There is no %changelog tag in your spec file. To insert it, just insert a
'%changelog' in your spec file and rebuild it.

python3-pytest-randomly.noarch: E: non-readable /usr/lib/python3.7/site-packages/__pycache__/pytest_randomly.cpython-37-PYTEST.pyc 600
The file can't be read by everybody. Review if this is expected.

1 packages and 0 specfiles checked; 2 errors, 0 warnings.
```

## 3. %check should not be failed

%check failed and the status seems to be currently ignored.
tests should be executed and I think they should be executed correctly.
https://docs.fedoraproject.org/en-US/packaging-guidelines/#_test_suites

Here is the output of 'rpmbuild -ra python-pytest-randomly-1.2.3-1.fc29.src.rpm'.
```
...snip...

+ pytest-3 tests/
============================= test session starts ==============================
platform linux -- Python 3.7.2, pytest-3.6.4, py-1.5.4, pluggy-0.6.0
Using --randomly-seed=1548246897
rootdir: /home/vagrant/rpmbuild/BUILD/pytest-randomly-1.2.3, inifile:
plugins: randomly-1.2.3
collected 20 items

tests/test_it.py ..........F...F.....                                    [100%]

...snip...
```

Thanks in advance,
Hirotaka Wakabayashi

Comment 2 Zbigniew Jędrzejewski-Szmek 2019-02-10 16:45:28 UTC
Looks good in general. 

> pytest_randomly.cpython-37-PYTEST.pyc

Is this file created by pytest? It looks like it shouldn't be part of the package at all.

The changelog is indeed missing.

The spec file and srpm don't match anymore. Please post an updated spec file and srpm.

Comment 3 Elliott Sales de Andrade 2019-09-05 09:03:20 UTC
You should set PYTHONDONTWRITEBYTECODE=1 when running tests so that you don't get pytest bytecode in your rpms.

Comment 4 Dan Callaghan 2019-12-29 22:27:46 UTC
Thanks for the tip about PYTHONDONTWRITEBYTECODE=1. Also it turns out the bug in python-faker affecting the tests has now been fixed so they pass cleanly.

Here is an updated version with the tests properly enabled, no pytest bytecode, and a proper %changelog:

https://fedorapeople.org/~dcallagh/python-pytest-randomly/python-pytest-randomly.spec
https://fedorapeople.org/~dcallagh/python-pytest-randomly/python-pytest-randomly-1.2.3-2.fc32.src.rpm

Comment 5 Zbigniew Jędrzejewski-Szmek 2019-12-30 19:26:46 UTC
Oops, upstream is at version 3.2.0 now. Please update!

Comment 6 Dan Callaghan 2020-01-12 05:52:28 UTC
Oh yes indeed.

I updated the package but noticed that upstream accidentally dropped the test suite from the latest tarballs. I filed a PR for it:
https://github.com/pytest-dev/pytest-randomly/pull/223

Let's see if they roll a new release for it soon...

Comment 7 Zbigniew Jędrzejewski-Szmek 2020-01-12 08:49:44 UTC
Please post a new spec file and srpm.

Comment 8 Dan Callaghan 2020-01-12 10:42:58 UTC
Yep I was just waiting to see upstream's response about the missing tests before I post the new spec.

As I expected, they are resistant to including tests in the source tarball from PyPI and suggest I should be using the Github tarballs instead. I guess nowadays everyone sees PyPI as basically a binary distribution platform and not a source hosting platform...

Comment 10 Zbigniew Jędrzejewski-Szmek 2020-01-30 09:02:54 UTC
+ package name is OK
+ latest version
+ license is acceptable for Fedora (BSD 3-clause)
+ license is specified correctly
+ builds and installs OK
+ rpmlint finds no issues

Package is APPROVED.

Comment 11 Igor Raits 2020-02-17 12:52:28 UTC
(fedscm-admin):  The Pagure repository was created at https://src.fedoraproject.org/rpms/python-pytest-randomly

Comment 12 Fedora Update System 2020-02-19 22:22:43 UTC
FEDORA-2020-26bda81e92 has been submitted as an update to Fedora 31. https://bodhi.fedoraproject.org/updates/FEDORA-2020-26bda81e92

Comment 13 Fedora Update System 2020-02-20 05:45:19 UTC
python-pytest-randomly-3.2.1-1.fc31 has been pushed to the Fedora 31 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-26bda81e92

Comment 14 Fedora Update System 2020-02-20 06:42:26 UTC
python-pytest-randomly-3.2.1-1.fc30 has been pushed to the Fedora 30 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-a76fa3f47e

Comment 15 Fedora Update System 2020-02-27 16:44:34 UTC
python-pytest-randomly-3.2.1-1.fc30 has been pushed to the Fedora 30 stable repository. If problems still persist, please make note of it in this bug report.

Comment 16 Fedora Update System 2020-02-27 17:30:18 UTC
python-pytest-randomly-3.2.1-1.fc31 has been pushed to the Fedora 31 stable repository. If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.