Bug 1791216

Summary: python-hypothesis fails to build with Python 3.9 with tests enabled: hypothesis.extra.pandas.impl imports Iterable, OrderedDict from collections
Product: [Fedora] Fedora Reporter: Miro Hrončok <mhroncok>
Component: python-hypothesisAssignee: Miro Hrončok <mhroncok>
Status: CLOSED DEFERRED QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: rawhideCC: cstratak, hugovk+redhatbugzilla, igor.raits, mhroncok, michel, mplch, pingou, python-sig
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-02-28 09:09:02 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1785415    

Description Miro Hrončok 2020-01-15 09:05:33 UTC
python-hypothesis fails to build with Python 3.9.0a2 with testes enabled:

==================================== ERRORS ====================================
_ ERROR collecting hypothesis-python/tests/pandas/test_argument_validation.py __
ImportError while importing test module '/builddir/build/BUILD/hypothesis-hypothesis-python-4.23.8/hypothesis-python/tests/pandas/test_argument_validation.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/pandas/test_argument_validation.py:20: in <module>
    import hypothesis.extra.pandas as pdst
../../../BUILDROOT/python-hypothesis-4.23.8-6.fc32.x86_64/usr/lib/python3.9/site-packages/hypothesis/extra/pandas/__init__.py:20: in <module>
    from hypothesis.extra.pandas.impl import (
../../../BUILDROOT/python-hypothesis-4.23.8-6.fc32.x86_64/usr/lib/python3.9/site-packages/hypothesis/extra/pandas/impl.py:20: in <module>
    from collections import Iterable, OrderedDict
E   ImportError: cannot import name 'Iterable' from 'collections' (/usr/lib64/python3.9/collections/__init__.py)
______ ERROR collecting hypothesis-python/tests/pandas/test_data_frame.py ______
ImportError while importing test module '/builddir/build/BUILD/hypothesis-hypothesis-python-4.23.8/hypothesis-python/tests/pandas/test_data_frame.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/pandas/test_data_frame.py:23: in <module>
    import hypothesis.extra.pandas as pdst
../../../BUILDROOT/python-hypothesis-4.23.8-6.fc32.x86_64/usr/lib/python3.9/site-packages/hypothesis/extra/pandas/__init__.py:20: in <module>
    from hypothesis.extra.pandas.impl import (
../../../BUILDROOT/python-hypothesis-4.23.8-6.fc32.x86_64/usr/lib/python3.9/site-packages/hypothesis/extra/pandas/impl.py:20: in <module>
    from collections import Iterable, OrderedDict
E   ImportError: cannot import name 'Iterable' from 'collections' (/usr/lib64/python3.9/collections/__init__.py)
_______ ERROR collecting hypothesis-python/tests/pandas/test_indexes.py ________
ImportError while importing test module '/builddir/build/BUILD/hypothesis-hypothesis-python-4.23.8/hypothesis-python/tests/pandas/test_indexes.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/pandas/test_indexes.py:25: in <module>
    import hypothesis.extra.pandas as pdst
../../../BUILDROOT/python-hypothesis-4.23.8-6.fc32.x86_64/usr/lib/python3.9/site-packages/hypothesis/extra/pandas/__init__.py:20: in <module>
    from hypothesis.extra.pandas.impl import (
../../../BUILDROOT/python-hypothesis-4.23.8-6.fc32.x86_64/usr/lib/python3.9/site-packages/hypothesis/extra/pandas/impl.py:20: in <module>
    from collections import Iterable, OrderedDict
E   ImportError: cannot import name 'Iterable' from 'collections' (/usr/lib64/python3.9/collections/__init__.py)
________ ERROR collecting hypothesis-python/tests/pandas/test_series.py ________
ImportError while importing test module '/builddir/build/BUILD/hypothesis-hypothesis-python-4.23.8/hypothesis-python/tests/pandas/test_series.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/pandas/test_series.py:24: in <module>
    import hypothesis.extra.pandas as pdst
../../../BUILDROOT/python-hypothesis-4.23.8-6.fc32.x86_64/usr/lib/python3.9/site-packages/hypothesis/extra/pandas/__init__.py:20: in <module>
    from hypothesis.extra.pandas.impl import (
../../../BUILDROOT/python-hypothesis-4.23.8-6.fc32.x86_64/usr/lib/python3.9/site-packages/hypothesis/extra/pandas/impl.py:20: in <module>
    from collections import Iterable, OrderedDict
E   ImportError: cannot import name 'Iterable' from 'collections' (/usr/lib64/python3.9/collections/__init__.py)


The import now must be done from collections.abc.

Our hypothesis version is quite old, but this is not yet even fixed in upstream master.

For the build logs, see:
https://copr-be.cloud.fedoraproject.org/results/@python/python3.9/fedora-rawhide-x86_64/01144829-python-hypothesis/

For all our attempts to build python-hypothesis with Python 3.9, see:
https://copr.fedorainfracloud.org/coprs/g/python/python3.9/package/python-hypothesis/

Testing and mass rebuild of packages is happening in copr. You can follow these instructions to test locally in mock if your package builds with Python 3.9:
https://copr.fedorainfracloud.org/coprs/g/python/python3.9/

Let us know here if you have any questions.

Python 3.9 will be included in Fedora 33. To make that update smoother, we're building Fedora packages with early pre-releases of Python 3.9.
A build failure prevents us from testing all dependent packages (transitive [Build]Requires), so it's important for us to get it fixed soon.
We'd appreciate help from the people who know this package best, but if you don't want to work on this now, let us know so we can try to work around it on our side.

Comment 2 Miro Hrončok 2020-01-22 11:02:04 UTC
> Our hypothesis version is quite old, but this is not yet even fixed in upstream master.

Oh, damn it, I've seen:

    from collections import OrderedDict

And I was like: not fixed. Need some rest, OrderedDict is not an abstract base class.

Thanks Hugo.

Comment 4 Ben Cotton 2020-02-11 17:29:17 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 32 development cycle.
Changing version to 32.

Comment 5 Miro Hrončok 2020-02-28 09:09:02 UTC
In Python 3.9.0a4 we have managed to revert the change that disables importing abstract base classes (collections.abc) directly form the regular collections module. It was reverted to ease transition for projects that did not yet actually dropped support for Python 2.7. The removal is scheduled for Python 3.10 instead.

We are mass closing all the bug reports about build failures that were caused by this. Expect a new Bugzilla report in case this package fails to build with Python 3.9 for a different reason as well.

In case you haven't done this already, we highly recommend to talk to your upstream and get this fixed anyway, because this will return in Python 3.10 and Python upstream is very unlikely to postpone this once more. In case you've already done that, thank you (this message is mass posted to all the relevant Bugzillas).