Red Hat Bugzilla – Bug 1473130
Adjust test execution settings in Python %check section
Last modified: 2018-06-14 17:33:11 EDT
The Python spec file currently runs the test suite using the "--verbose" flag, which makes it hard to find the details of which tests actually failed when something goes wrong.
Two other flags are likely to produce easier to read results (these can both be set at the same time):
-w: Immediately re-runs failed tests in verbose mode
-W: Prints captured output when a test fails
Another potentially useful option for the %check section may be "--failfast", which will abort the test run as soon as any test fails, rather than always continuing to run the full test suite.
Some other ideas potentially worth exploring:
* using the `--findleaks` option forces serialisation of test execution on a single CPU. If it is omitted (effectively reclassifying leak hunting in the Python runtime as an integration test rather than a unit test), then the `-j` option could instead be passed to run tests in parallel across multiple CPUs, which may significantly reduce the overall RPM build time (since the test suite gets run twice, once against the debug binaries, once against the release binaries). If we did this, then it would make sense to instead set up a Taskotron task to replace it in the Fedora release pipeline.
* the `--slowest` option lists the 10 slowest tests at the end of the run
First step done for F27 and F26 
This bug appears to have been reported against 'rawhide' during the Fedora 27 development cycle.
Changing version to '27'.
[x] -wW was added
[x] --failfast is not worth it
[ ] --findleaks removal is waiting for Taskotron or Standard test interface
[x] --slowest is there already
This bug appears to have been reported against 'rawhide' during the Fedora 28 development cycle.
Changing version to '28'.
Miro is looking into running integration tests first.