Bug 1377125 - python-hacking doesn't work with flake8 3.x
Summary: python-hacking doesn't work with flake8 3.x
Keywords:
Status: CLOSED EOL
Alias: None
Product: Fedora
Classification: Fedora
Component: python-hacking
Version: 26
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Matthias Runge
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-09-18 19:12 UTC by Ville Skyttä
Modified: 2018-05-29 11:40 UTC (History)
4 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2018-05-29 11:40:53 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
Port to pycodestyle/flake8-2.6.x (7.01 KB, patch)
2016-10-05 20:13 UTC, Ville Skyttä
no flags Details | Diff


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1607942 0 None None None 2016-09-22 07:57:21 UTC
Launchpad 1652409 0 None None None 2016-12-24 01:24:20 UTC

Description Ville Skyttä 2016-09-18 19:12:59 UTC
I've updated flake8 to 3.0.4 in rawhide, and after doing that found that current python-hacking doesn't work with it.

The first step towards a fix is to add dependencies to pep8 as flake8 will no longer pull it in (it uses pycodestyle instead). But that doesn't seem enough, looks like code changes are needed.

hacking 0.11.0 is available, but judging from requirements.txt contents it may not work with flake8 3.x either (has flake8<2.6.0,>=2.5.4).

Comment 1 Matthias Runge 2016-09-22 07:57:22 UTC
Looking at hacking upstream, it seems updating flake8 was a bit early. 

The issue is known on both upstreams https://gitlab.com/pycqa/flake8/issues/153
but hasn't been fixed.

Comment 2 Ville Skyttä 2016-09-22 09:20:25 UTC
Yes, it was a bit early, my bad as already noted. I intend to limit it to F-26 only at least until dependents have been fixed.

Not sure what you mean by hasn't been fixed -- the gitlab issue has been closed, it points to https://review.openstack.org/335965 which has status merged, and has indeed been merged as https://github.com/openstack-dev/hacking/commit/c8f3bc26a29bb04641ed5bf967b1dce632c97a0e. Maybe 0.11.0 + that change would be enough -- want me to try it out + do the update in rawhide if it appears to work?

Comment 3 Matthias Runge 2016-09-22 09:53:17 UTC
Builds still fail due to test issues:

Failed to import test module: hacking.tests.test_doctest
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/unittest2/loader.py", line 456, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/lib/python2.7/site-packages/unittest2/loader.py", line 395, in _get_module_from_name
    __import__(name)
  File "hacking/tests/test_doctest.py", line 18, in <module>
    from flake8 import engine
ImportError: cannot import name engine

Failed to import test module: hacking.tests.test_local
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/unittest2/loader.py", line 456, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/lib/python2.7/site-packages/unittest2/loader.py", line 395, in _get_module_from_name
    __import__(name)
  File "hacking/tests/test_local.py", line 16, in <module>
    from flake8 import engine
ImportError: cannot import name engine
Non-zero exit code (2) from test listing.
error: testr failed (3)


Not sure if it makes sense to pull all kinds of patches together now or if it's better to wait for a release fixing the issues.

Comment 4 Matthias Runge 2016-09-22 10:14:30 UTC
It fails the same way with latest master snapshot.

Comment 5 Ville Skyttä 2016-09-22 10:30:18 UTC
I'll try to find time to take a peek at fixing this in hacking master soonish unless someone beats me to it.

Comment 6 Ville Skyttä 2016-10-05 20:13:59 UTC
Created attachment 1207675 [details]
Port to pycodestyle/flake8-2.6.x

The attached patch (against upstream git) paves way towards proper flake8 3.x support by porting to pycodestyle and flake8 2.6.x. Lightly tested, test suite passes, seems to work.

But this is not enough for flake8 3.x. On a quick peek the hacking/tests/test_doctest.py and hacking/tests/test_local.py test files need substantial changes to work with 3.x. But other than those two files, it appears on a very quick peek that no other changes would be needed; at least some checks start to work.

So I guess upstream git master + this patch + requirements.txt tweak for 3.x + simply rm those two mentioned test files in the specfile's %prep would result in at least somewhat functional hacking with flake8 3.x. Haven't tested building such a package but rather played around in virtualenvs, so as said this is bit of a guess.

Although I suppose the attached patch should be good for upstream as-is, I haven't submitted it because they require new contributors to jump through more hoops than what I'm willing to do right now just for the purpose of getting this one-off patch applied.

Comment 7 Adam Williamson 2016-12-23 22:58:06 UTC
I've spent most of today bashing my head against this, as I'm working on the Python 3.6.

The good news: hacking *mostly* does actually work with flake8 3.x, at least AFAICT. I have figured out how to make the test_doctest test suite work, and correct the test_local test suite for flake8 3.x.

The bad news: the functionality that test_local tests - the 'local-check' feature of hacking - does not work with flake8 3.x and cannot really *possibly* work with it (at least, not with my Python skills).

flake8 changed extensively between 2.x and 3.x. It is no longer really a 'wrapper' around pep8/pycodestyle as it claims to be; it's more of an entire alternate loader for pycodestyle-type checks. It doesn't actually pycodestyle's check loading or running code at all. All it does is import some checks from pycodestyle as plugins to itself. It does all the checker loading and running itself.

This fundamentally breaks hacking's 'local-check' feature, AFAICT. hacking tries to implement this 'local-check' feature by loading the specified checks into pep8/pycodestyle, by importing the check function and running `pep8.register_check(checker)` . This, technically, still *works* fine: the check does indeed get loaded into pep8 (or pycodestyle, if you tweak it to import that instead).

However, it doesn't *achieve* anything at all, because flake8 ignores the pep8 check register and builds its own. It has its own effectively hardcoded list of pycodestyle built-in checks to load (all the ones listed in its setup.py). So it doesn't run the local checks at all.

I can't see any viable way for hacking to implement this feature on flake8 3.x, because the only mechanism flake8 actually has for loading checks is via entry points, and I don't think it's logically possible for something that has *itself* been loaded as an entry point-based plugin to flake8 - hacking - to somehow create a *new* entry point for flake8 and cause it to load that. I don't see how that could work. So unless flake8 grows some alternative way to register checks, I think this feature is just doomed. Maybe someone smarter than me can see a fix I'm missing, but I'm stumped.

This, at least, is the state of affairs so far as I can figure it out. I'm going to do a build which just disables the test for local-check and includes a note about it somewhere, as that's no worse than what we have right now, and report this mess upstream somewhere if I can figure out where...

Comment 8 Adam Williamson 2016-12-24 01:24:20 UTC
I've filed https://bugs.launchpad.net/hacking/+bug/1652409 for the issue with the local-check feature, and done a Rawhide build with my patch to make the tests work with flake8 3.x: https://koji.fedoraproject.org/koji/taskinfo?taskID=17043639

Note I have tested that the tests also work with flake8 2.5.5 by building the same SPRM on Fedora 25 locally, but I haven't sent an official build for F25, as that's kinda out of my scope.

I also tested manually that running flake8 and python3-flake8 actually *works* on both F25 and Rawhide, and produces what looks like sane output, including checks from hacking.

Don't know if we want to keep this open to track the local-check feature issue, or close it.

Comment 9 Adam Williamson 2016-12-24 01:25:04 UTC
If someone could help send my patch upstream, btw, that'd be great. I saw the gigantic series of hoops one is required to jump through, and thought "yeesh".

Comment 10 Fedora End Of Life 2017-02-28 10:19:45 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 26 development cycle.
Changing version to '26'.

Comment 11 Fedora End Of Life 2018-05-03 08:28:15 UTC
This message is a reminder that Fedora 26 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 26. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '26'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version'
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not
able to fix it before Fedora 26 is end of life. If you would still like
to see this bug fixed and are able to reproduce it against a later version
of Fedora, you are encouraged  change the 'version' to a later Fedora
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.

Comment 12 Fedora End Of Life 2018-05-29 11:40:53 UTC
Fedora 26 changed to end-of-life (EOL) status on 2018-05-29. Fedora 26
is no longer maintained, which means that it will not receive any
further security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.


Note You need to log in before you can comment on or make changes to this bug.