Created attachment 1562674 [details] Full log from Copr python-joblib-0.13.0 fails to build with Python 3.8 It seems that test_hashing fails on various test cases and also test_numpy_pickle Example snippet: ________ test_hashes_stay_the_same[This is a string to hash-expected0] _________ to_hash = 'This is a string to hash' expected = {'py2': '80436ada343b0d79a99bfd8883a96e45', 'py3': '71b3f47df22cb19431d85d92d0b230b2'} @parametrize('to_hash,expected', [('This is a string to hash', {'py2': '80436ada343b0d79a99bfd8883a96e45', 'py3': '71b3f47df22cb19431d85d92d0b230b2'}), (u"C'est l\xe9t\xe9", {'py2': '2ff3a25200eb6219f468de2640913c2d', 'py3': '2d8d189e9b2b0b2e384d93c868c0e576'}), ((123456, 54321, -98765), {'py2': '50d81c80af05061ac4dcdc2d5edee6d6', 'py3': 'e205227dd82250871fa25aa0ec690aa3'}), ([random.Random(42).random() for _ in range(5)], {'py2': '1a36a691b2e2ba3a9df72de3dccf17ea', 'py3': 'a11ffad81f9682a7d901e6edc3d16c84'}), ([3, 'abc', None, TransportableException('foo', ValueError)], {'py2': 'adb6ba84990ee5e462dc138383f11802', 'py3': '994f663c64ba5e64b2a85ebe75287829'}), ({'abcde': 123, 'sadfas': [-9999, 2, 3]}, {'py2': 'fc9314a39ff75b829498380850447047', 'py3': 'aeda150553d4bb5c69f0e69d51b0e2ef'})]) def test_hashes_stay_the_same(to_hash, expected): # We want to make sure that hashes don't change with joblib # version. For end users, that would mean that they have to # regenerate their cache from scratch, which potentially means # lengthy recomputations. # Expected results have been generated with joblib 0.9.2 py_version_str = 'py3' if PY3_OR_LATER else 'py2' > assert hash(to_hash) == expected[py_version_str] E AssertionError: assert '115916c98f74...58a5c7184c94c' == '71b3f47df22cb...85d92d0b230b2' E - 115916c98f747117d0458a5c7184c94c E + 71b3f47df22cb19431d85d92d0b230b2 Full log attached
Please respond.
I can update to the latest joblib, but is python 3 only. scikit-learn depends on it and it still has the pyhton2 subpackage. Can I drop the python2 subpackage of scikit-learn?
scikit-learn has no Python 2 subpackage in rawhide, neither does joblib.
(In reply to Miro Hrončok from comment #3) > scikit-learn has no Python 2 subpackage in rawhide, neither does joblib. Great! I have updated to 0.13.2 https://koji.fedoraproject.org/koji/taskinfo?taskID=36182499 Could you check if you still get the same problem?
+ /usr/bin/python3 setup.py build '--executable=/usr/bin/python3 -s' Traceback (most recent call last): File "setup.py", line 6, in <module> import joblib File "/builddir/build/BUILD/joblib-0.13.2/joblib/__init__.py", line 119, in <module> from .parallel import Parallel File "/builddir/build/BUILD/joblib-0.13.2/joblib/parallel.py", line 28, in <module> from ._parallel_backends import (FallbackToBackend, MultiprocessingBackend, File "/builddir/build/BUILD/joblib-0.13.2/joblib/_parallel_backends.py", line 22, in <module> from .executor import get_memmapping_executor File "/builddir/build/BUILD/joblib-0.13.2/joblib/executor.py", line 14, in <module> from .externals.loky.reusable_executor import get_reusable_executor File "/builddir/build/BUILD/joblib-0.13.2/joblib/externals/loky/__init__.py", line 12, in <module> from .backend.reduction import set_loky_pickler File "/builddir/build/BUILD/joblib-0.13.2/joblib/externals/loky/backend/reduction.py", line 125, in <module> from joblib.externals import cloudpickle # noqa: F401 File "/builddir/build/BUILD/joblib-0.13.2/joblib/externals/cloudpickle/__init__.py", line 3, in <module> from .cloudpickle import * File "/builddir/build/BUILD/joblib-0.13.2/joblib/externals/cloudpickle/cloudpickle.py", line 152, in <module> _cell_set_template_code = _make_cell_set_template_code() File "/builddir/build/BUILD/joblib-0.13.2/joblib/externals/cloudpickle/cloudpickle.py", line 133, in _make_cell_set_template_code return types.CodeType( TypeError: an integer is required (got type bytes) CodeType has a new signature in 3.8. See https://docs.python.org/3.8/whatsnew/3.8.html - search for CodeType.
In fact, cloudpickle is a bundled package (0.8.0). Even the upstream version of cloudpickle has exactly the same problem https://github.com/cloudpipe/cloudpickle/blob/master/cloudpickle/cloudpickle.py#L367
The linked code is not used on 3.8. Please, just unbundle cloudpickle and you should be good.
I have unbundled cloudpickle in the last joblib https://koji.fedoraproject.org/koji/buildinfo?buildID=1311731
Thanks. The recent failure is: https://copr-be.cloud.fedoraproject.org/results/@python/python3.8/fedora-rawhide-x86_64/00967852-python-joblib/builder-live.log You can try to build the package in mock: http://copr.fedorainfracloud.org/coprs/g/python/python3.8/
This bug appears to have been reported against 'rawhide' during the Fedora 31 development cycle. Changing version to '31'.
The coordinated rebuild of Python 3.8 has started in the `f32-python` side tag. If you figure out how to rebuild this package, please don't rebuild it in regular rawhide, but use the side tag instead: on branch master: $ fedpkg build --target=f32-python To wait for a build to show up in the side tag, do: $ koji wait-repo f32-python --build=<nvr> Where <nvr> is name-version-release of the source package, e.g. python-foo-1.1-2.fc32. An updated mock config is posted at: http://copr.fedorainfracloud.org/coprs/g/python/python3.8/ Note that it will take a while before the essential packages are rebuilt, so don't expect all your dependencies to be available right away. Thanks. Let us know if you need up to date info, or if you have any questions. PS this message is mass posted to all the bugs that block the PYTHON38 bug. If this is also a Fedora 31 FTBFS bug and you manage to fix it, you can do a f31 build as usual: on branch f31: $ fedpkg build
The f32-python side tag has been merged. In order to rebuild the package, do it in regular rawhide, but please wait until python3-3.8 is tagged: $ koji wait-repo f32-build --build python3-3.8.0~b3-3.fc32 If your built already started in f32-python, after it is finished, please tag it to rawhide with: $ koji tag-build f32-pending <nvr> For example: $ koji tag-build f32-pending libreoffice-6.3.0.4-3.fc32 Thanks! (This comment is mass posted to all bugzillas blocking the PYTHON38 tracking bug.)
(Python 3.8 has landed in the rawhide buildroot.)