Bug 1473130 - Adjust test execution settings in Python %check section
Adjust test execution settings in Python %check section
Status: NEW
Product: Fedora
Classification: Fedora
Component: python3 (Show other bugs)
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Charalampos Stratakis
Fedora Extras Quality Assurance
: FutureFeature
Depends On:
  Show dependency treegraph
Reported: 2017-07-20 01:37 EDT by Nick Coghlan
Modified: 2018-06-14 17:33 EDT (History)
8 users (show)

See Also:
Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description Nick Coghlan 2017-07-20 01:37:12 EDT
The Python spec file currently runs the test suite using the "--verbose" flag, which makes it hard to find the details of which tests actually failed when something goes wrong.

Two other flags are likely to produce easier to read results (these can both be set at the same time):

-w: Immediately re-runs failed tests in verbose mode
-W: Prints captured output when a test fails

Another potentially useful option for the %check section may be "--failfast", which will abort the test run as soon as any test fails, rather than always continuing to run the full test suite.

Some other ideas potentially worth exploring:

* using the `--findleaks` option forces serialisation of test execution on a single CPU. If it is omitted (effectively reclassifying leak hunting in the Python runtime as an integration test rather than a unit test), then the `-j` option could instead be passed to run tests in parallel across multiple CPUs, which may significantly reduce the overall RPM build time (since the test suite gets run twice, once against the debug binaries, once against the release binaries). If we did this, then it would make sense to instead set up a Taskotron task to replace it in the Fedora release pipeline.

* the `--slowest` option lists the 10 slowest tests at the end of the run
Comment 1 Charalampos Stratakis 2017-07-20 12:02:38 EDT
First step done for F27 and F26 [0]

[0] http://pkgs.fedoraproject.org/cgit/rpms/python3.git/commit/?id=5b6a3e0e34f46702773ad1082df3b782c1a8b7ac
Comment 2 Jan Kurik 2017-08-15 02:25:57 EDT
This bug appears to have been reported against 'rawhide' during the Fedora 27 development cycle.
Changing version to '27'.
Comment 3 Petr Viktorin 2018-01-18 09:08:54 EST
[x] -wW was added
[x] --failfast is not worth it
[ ] --findleaks removal is waiting for Taskotron or Standard test interface
[x] --slowest is there already
Comment 4 Fedora End Of Life 2018-02-20 10:33:43 EST
This bug appears to have been reported against 'rawhide' during the Fedora 28 development cycle.
Changing version to '28'.
Comment 5 Petr Viktorin 2018-04-19 09:59:21 EDT
Miro is looking into running integration tests first.

Note You need to log in before you can comment on or make changes to this bug.