Bug 1075088

Wouldn't it make more sense to run all the tests with make check?

Hi David,
didn't see the make check target. I will rework the patch.  I tried to run it and this is what I got (plus a few more Buildrequires). 


$ cat tests/test-suite.log 
==========================================
   anaconda 21.26: tests/test-suite.log
==========================================

# TOTAL: 7
# PASS:  5
# SKIP:  1
# XFAIL: 0
# FAIL:  1
# XPASS: 0
# ERROR: 0

.. contents:: :depth: 2

FAIL: pylint/runpylint
======================

pylint reports the following issues:
************* Module ../scripts/makeupdates
F0401:711,4: get_rpm_from_Koji_thread: Unable to import 'koji'
************* Module ../pyanaconda/network.py
I0011:709,0: : Locally disabling E1101
W1201:1118,20: apply_kickstart_from_pre_section: Specify string format arguments as logging function parameters
************* Module ../pyanaconda/anaconda.py
F0401:164,8: Anaconda.dumpState: Unable to import 'meh'
F0401:165,8: Anaconda.dumpState: Unable to import 'meh.dump'
************* Module ../pyanaconda/rescue.py
F0401: 46,0: : Unable to import 'meh.ui.text'
************* Module ../anaconda
I0011:398,0: : Locally disabling W0621
I0011:467,0: : Locally disabling F0401
I0011:723,0: : Locally disabling W0611
F0401: 39,4: : Unable to import 'coverage'
************* Module ../pyanaconda/exception.py
I0011:176,0: : Locally disabling W0702
F0401: 24,0: : Unable to import 'meh'
F0401: 25,0: : Unable to import 'meh.handler'
F0401: 26,0: : Unable to import 'meh.dump'
E1002: 62,4: AnacondaExceptionHandler.run_handleException: Use of super on an old style class
E1002: 74,4: AnacondaExceptionHandler.handleException: Use of super on an old style class
E1002: 74,4: AnacondaExceptionHandler.handleException: Use of super on an old style class
E1101: 98,12: AnacondaExceptionHandler.handleException: Instance of 'AnacondaExceptionHandler' has no 'intf' member
E1101:168,66: AnacondaExceptionHandler.postWriteHook: Instance of 'AnacondaExceptionHandler' has no 'exnFile' member
E1101:169,32: AnacondaExceptionHandler.postWriteHook: Instance of 'AnacondaExceptionHandler' has no 'exnFile' member
E1101:171,69: AnacondaExceptionHandler.postWriteHook: Instance of 'AnacondaExceptionHandler' has no 'exnFile' member
E1101:264,4: initExceptionHandling: Instance of 'AnacondaExceptionHandler' has no 'install' member
************* Module ../pyanaconda/ui/tui/__init__.py
F0401: 35,0: : Unable to import 'meh.ui.text'
************* Module ../tests/lib/translatepo.py
F0401: 30,4: : Unable to import 'polib'
ERROR:root:Could not find any typelib for Keybinder
************* Module ../pyanaconda/ui/gui/__init__.py
I0011:223,0: : Locally disabling W0221
************* Module ../pyanaconda/ui/gui/__init__.py
F0401: 22,0: : Unable to import 'meh.ui.gui'
************* Module ../pyanaconda/packaging/dnfpayload.py
F0401: 43,4: : Unable to import 'dnf'
F0401: 44,4: : Unable to import 'dnf.exceptions'
F0401: 45,4: : Unable to import 'dnf.repo'
F0401: 46,4: : Unable to import 'dnf.callback'
E1002:118,4: PayloadRPMDisplay.__init__: Use of super on an old style class
E1101:125,21: PayloadRPMDisplay.event: Instance of 'PayloadRPMDisplay' has no 'PKG_INSTALL' member
E1101:135,23: PayloadRPMDisplay.event: Instance of 'PayloadRPMDisplay' has no 'TRANS_POST' member

SKIP: storage/run_storage_tests.py
==================================

You must be root to run the storage tests; skipping.


Does the pylint section look OK or I'm missing some requirements (looking at the import errors) ?

Another question: is the test-suite.log file available for download from Koji in case one needs to investigate failures ? If not we have to redirect all output to stderr

Created attachment 873422 [details]
New patch using make check

With this patch the pylint check fails which ultimately halts the anaconda build.

Do we really want to run the PyLint check during build ? Considering how long it runs (even  though it is multi-threaded), the number of dependencies it requires & the number false positives that still crop up.

On the other hand if it was not build-blocking and the log could be downloaded from Koji, it might be useful.

All the pylint failures in comment 2 are missing buildrequires. If you want to run checks with the build, I say we should include pylint, since pylint and the addons we've written for it are currently the most extensive checking we do for the anaconda code.

If you run make check with VERBOSE=1 it should print the output of test-suite.log

Created attachment 873586 [details]
execute make check in %check and add more BuildRequires for pylint

python-mock also needs to be in the buildrequires for the unit tests. Other than that it looks like everything is there. Just make the patch against anaconda.spec.in instead of anaconda.spec.

Created attachment 875850 [details]
Add python-mock and patch against anaconda.spec.in

Btw I started getting the following error with anaconda-21.27-1.fc21 which I didn't see before:

pylint reports the following issues:
error: XDG_RUNTIME_DIR not set in the environment.


The test system isn't running X server.

Yeah, I started seeing that too. Fixed in commit 380570aeebbe860e1a0c69374e203e8f6a5c32ba

Copied the patch to anaconda-patches to see if anyone else has any comments.

I just noticed the hard-coded python-blivet version, so I went ahead and changed your patch to move that (and the other hard-coded version that was already there) into a macro. Pushed as e84068c658a7cdadf620e97ab1c059e27924e8d4.

Reverted the commit. We did not find %check failures blocking the build to be particularly helpful.