python-unittest2 fails to build with Python 3.9.0a2. Error message: running install Traceback (most recent call last): File "/builddir/build/BUILD/unittest2-1.1.0/setup.py", line 87, in <module> setup(**params) File "/usr/lib/python3.9/site-packages/setuptools/__init__.py", line 145, in setup return distutils.core.setup(**attrs) File "/usr/lib64/python3.9/distutils/core.py", line 148, in setup dist.run_commands() File "/usr/lib64/python3.9/distutils/dist.py", line 966, in run_commands self.run_command(cmd) File "/usr/lib64/python3.9/distutils/dist.py", line 984, in run_command cmd_obj.ensure_finalized() File "/usr/lib64/python3.9/distutils/cmd.py", line 107, in ensure_finalized self.finalize_options() File "/usr/lib/python3.9/site-packages/setuptools/command/install.py", line 38, in finalize_options orig.install.finalize_options(self) File "/usr/lib64/python3.9/distutils/command/install.py", line 292, in finalize_options 'dist_fullname': self.distribution.get_fullname(), File "/usr/lib64/python3.9/distutils/dist.py", line 1166, in get_fullname return "%s-%s" % (self.get_name(), self.get_version()) File "/builddir/build/BUILD/unittest2-1.1.0/setup.py", line 15, in __str__ from unittest2 import __version__ as VERSION File "/builddir/build/BUILD/unittest2-1.1.0/unittest2/__init__.py", line 40, in <module> from unittest2.collector import collector File "/builddir/build/BUILD/unittest2-1.1.0/unittest2/collector.py", line 3, in <module> from unittest2.loader import defaultTestLoader File "/builddir/build/BUILD/unittest2-1.1.0/unittest2/loader.py", line 13, in <module> from unittest2 import case, suite, util File "/builddir/build/BUILD/unittest2-1.1.0/unittest2/case.py", line 18, in <module> from unittest2 import result File "/builddir/build/BUILD/unittest2-1.1.0/unittest2/result.py", line 10, in <module> from unittest2.compatibility import wraps File "/builddir/build/BUILD/unittest2-1.1.0/unittest2/compatibility.py", line 143, in <module> class ChainMap(collections.MutableMapping): AttributeError: module 'collections' has no attribute 'MutableMapping' This is due to MutableMapping being now accessible only through collections.abc. For the build logs, see: https://copr-be.cloud.fedoraproject.org/results/@python/python3.9/fedora-rawhide-x86_64/01141957-python-unittest2/ For all our attempts to build python-unittest2 with Python 3.9, see: https://copr.fedorainfracloud.org/coprs/g/python/python3.9/package/python-unittest2/ Testing and mass rebuild of packages is happening in copr. You can follow these instructions to test locally in mock if your package builds with Python 3.9: https://copr.fedorainfracloud.org/coprs/g/python/python3.9/ Let us know here if you have any questions. Python 3.9 will be included in Fedora 33, but the initial bootstrapping has already started. A build failure this early in the bootstrap sequence blocks us very much.
This can be fixed with something like: diff --git a/unittest2/compatibility.py b/unittest2/compatibility.py index 9e5f1a5..0eedfa7 100644 --- a/unittest2/compatibility.py +++ b/unittest2/compatibility.py @@ -1,3 +1,4 @@ +import collections.abc as abc import collections import os import sys @@ -140,7 +141,7 @@ except ImportError: ### ChainMap (helper for configparser and string.Template) ######################################################################## -class ChainMap(collections.MutableMapping): +class ChainMap(abc.MutableMapping): ''' A ChainMap groups multiple dicts (or other mappings) together to create a single, updateable view.
Why would we fix this? Shouldn't we fix the 20 packages that incorrectly require unittest2 and make unittest2 EPEL-only? For some background, unittest2 is a backport of unittest for Python 2.6 since significant changes were added in 2.7. So why would we need this for Python 3.9? We don't. I checked a few of these and for most of them we can just remove the line in the spec file for unittest2. The packages either don't use it or they have 2.6 compatibility and do a conditional import. One or two may need a small patch. All of these are srpms except python3-testtools. $ dnf repoquery --disablerepo=* --enablerepo=fedora{,-source} --releasever=rawhide --whatrequires python3-unittest2 --refresh ProDy-0:1.10.10-4.fc32.src beaker-0:27.0-1.fc32.src fedpkg-0:1.37-10.fc32.src obs-service-tar_scm-0:0.10.10-2.fc32.src pyephem-0:3.7.6.0-17.fc32.src python-agate-0:1.6.1-7.fc32.src python-billiard-1:3.6.1.0-3.fc32.src python-case-0:1.5.3-4.fc32.src python-funcsigs-0:1.0.2-17.fc32.src python-leather-0:0.3.3-13.gite85dd30.fc32.src python-linecache2-0:1.0.0-24.fc32.src python-mimerender-0:0.6.0-5.fc32.src python-pyclipper-0:1.1.0-8.fc32.src python-pynwb-0:1.1.2-1.fc32.src python-rsa-0:3.4.2-12.fc32.src python-social-auth-core-0:1.7.0-8.fc32.src python-testtools-0:2.3.0-15.fc32.src python-traceback2-0:1.4.0-25.fc32.src python3-testtools-0:2.3.0-15.fc32.noarch tuned-0:2.13.0-1.fc32.src
> Shouldn't we fix the 20 packages that incorrectly require unittest2 and make unittest2 EPEL-only? Yes! I've contacted the maintainers repeatedly about this, but this is what we have (left). Somebody can do a heroic effort to fix all of them, but it won't be me. In the mean time, before somebody does, let's fix this one instead?
> A build failure this early in the bootstrap sequence blocks us very much. Which dependencies are the blocking ones?
We bootstrap to fedpkg - in order to have a working rawhide development environment once the side tag is merged. So the obvious one is fedpkg, but that has been fixed: https://src.fedoraproject.org/rpms/fedpkg/c/245d4d31c29352f975c74a92d4ba2cb650e6f9b7?branch=master testrepository and testscenario need testtools are needed to build pbr and runtime for fixtures. pbr and testools combined are needed very much, mostly by the oslo stack, but also other packages. $ repoquery --repo=rawhide{,-source} --whatrequires python3-fixtures | pkgname dlrn python-bashate python-congressclient python-debtcollector python-gear python-keystoneauth1 python-keystoneclient python-linecache2 python-magnumclient python-mox3 python-openstackclient python-os-client-config python-osc-lib python-oslo-concurrency python-oslo-config python-oslo-context python-oslo-db python-oslo-i18n python-oslo-utils python-pyvmomi python-requests-mock python-sphinxcontrib-spelling python-stestr python-subunit2sql python-tackerclient python-testrepository python-testresources python-testscenarios python-tosca-parser python-traceback2 python3-congressclient-tests python3-keystoneclient-tests python3-magnumclient-tests python3-mox3 python3-octaviaclient-tests python3-osc-lib-tests python3-oslo-concurrency-tests python3-oslo-db-tests python3-oslo-utils-tests python3-oslotest python3-pifpaf python3-stestr python3-tackerclient-tests-unit python3-testrepository python3-testtools subunit $ repoquery --repo=rawhide{,-source} --whatrequires python3-pbr | pkgname ara bandit dib-utils diskimage-builder diskimage-builder dlrn git-pull-request git-review guake guake heat-cfntools heat-cfntools nodepool nodepool pylast python-ZEO python-aioresponses python-aodhclient python-automaton python-barbicanclient python-bashate python-cinderclient python-cliff python-congressclient python-cotyledon python-couchbase python-cradox python-daiquiri python-debtcollector python-designateclient python-distroinfo python-django-helpdesk python-fedmsg-meta-fedora-infrastructure python-fixtures python-freecell_solver python-futurist python-gabbi python-gear python-gerritlib python-gertty python-glanceclient python-gnocchiclient python-hacking python-heatclient python-ironicclient python-jenkins python-jenkins-job-builder python-jira python-jsonpath-rw-ext python-keystoneauth1 python-keystoneclient python-ldappool python-linecache2 python-lockfile python-magnumclient python-manilaclient python-migrate python-mistralclient python-molecule python-mox3 python-murano-pkg-check python-neutronclient python-novaclient python-octaviaclient python-openstack-doc-tools python-openstackclient python-openstackdocstheme python-openstacksdk python-os-client-config python-os-service-types python-os-testr python-osc-lib python-oslo-concurrency python-oslo-config python-oslo-context python-oslo-db python-oslo-i18n python-oslo-log python-oslo-serialization python-oslo-sphinx python-oslo-utils python-oslotest python-pifpaf python-positional python-pyghmi python-pymod2pkg python-pysol-cards python-renderspec python-reno python-requests-mock python-requests-unixsocket python-requestsexceptions python-rsd-lib python-saharaclient python-shade python-sphinx-click python-sphinxcontrib-apidoc python-sphinxcontrib-fulltoc python-sphinxcontrib-pecanwsme python-sphinxcontrib-spelling python-stestr python-stevedore python-subunit2sql python-sushy python-swiftclient python-tackerclient python-taskflow python-tenacity python-testinfra python-testscenarios python-testtools python-tosca-parser python-traceback2 python-troveclient python-virtualbmc python-virtualenvwrapper python-vitrageclient python-yaql python-zaqarclient python-zuul-sphinx python3-aodhclient python3-ara python3-automaton python3-barbicanclient python3-bashate python3-cinderclient python3-congressclient python3-debtcollector python3-designateclient python3-distroinfo python3-django-helpdesk python3-dlrn python3-etcd3gw python3-fixtures python3-freecell_solver python3-futurist python3-gabbi python3-gear python3-gerritlib python3-gertty python3-glanceclient python3-gnocchiclient python3-hacking python3-heatclient python3-ironicclient python3-jenkins python3-jira python3-jsonpath-rw-ext python3-keystoneauth1 python3-keystoneclient python3-magnumclient python3-manilaclient python3-migrate python3-mistralclient python3-molecule python3-mox3 python3-murano-pkg-check python3-neutronclient python3-novaclient python3-octaviaclient python3-openstack-doc-tools python3-openstackclient python3-openstackdocstheme python3-openstacksdk python3-os-service-types python3-os-testr python3-osc-lib python3-oslo-concurrency python3-oslo-config python3-oslo-context python3-oslo-db python3-oslo-log python3-oslo-sphinx python3-pifpaf python3-positional python3-pymod2pkg python3-pysol-cards python3-pywbem python3-rdopkg python3-reno python3-rsd-lib python3-rsd-lib-tests python3-rsdclient python3-saharaclient python3-shade python3-sphinx-click python3-sphinxcontrib-apidoc python3-stestr python3-stevedore python3-subunit2sql python3-sushy python3-tackerclient python3-tackerclient-tests-unit python3-taskflow python3-testinfra python3-testscenarios python3-testtools python3-troveclient python3-virtualbmc python3-vitrageclient python3-yaql python3-zaqarclient python3-zuul-sphinx pywbem rdopkg
> pbr and testools combined I meant: pbr and fixtures combined
I'll start by trying to remove the dependency in python-pbr.
Does python-pbr have a dependency on unittest2? It wasn't in the list. Can one of you guys with commit access remove python-unittest2 and python-traceback2 from python-testtools? That should go a long way in fixing the dependencies.
Happy to do a pull request if needed, but in testing looks like a simple fix.
> Does python-pbr have a dependency on unittest2? Yeah, pbr doesn't. It must fail for some other reason. > Happy to do a pull request if needed, but in testing looks like a simple fix. If you already have the local patch, please do.
(In reply to Avram Lubkin from comment #8) > Does python-pbr have a dependency on unittest2? It wasn't in the list. pbr has: BuildRequires: python3-testrepository BuildRequires: python3-testresources BuildRequires: python3-testscenarios It has a transitive dependency on unittest2.
PR created for python-testtools https://src.fedoraproject.org/rpms/python-testtools/pull-request/5
Initial list had 19 packages (python-testtools was listed as rpm and srpm). fedpkg is done, so that's 19. python-traceback2 seems to be only used by python-testtools and python-unittest so it should be easy to drop. If python-testtools is done that should fix a lot of dependencies and leaves the list at 17. Most of these are easy, just removing a line in the spec. It's possible that something might require a patch, but I haven't seen it yet. If they will get merged, I can work on PRs today for some more of them.
Count is one off. I meant we're at 18 now, after python-testtools patch is merged, that's an effective 16.
(In reply to Miro Hrončok from comment #3) > > Shouldn't we fix the 20 packages that incorrectly require unittest2 and make unittest2 EPEL-only? > > Yes! I've contacted the maintainers repeatedly about this, but this is what > we have (left). Somebody can do a heroic effort to fix all of them, but it > won't be me. > > In the mean time, before somebody does, let's fix this one instead? I can fix unittest2 in the meanwhile to remove some pressure on the heroic effort.
Thanks, Alfredo.
I was advocating we don't fix it. I wouldn't call the effort heroic. It's 17 packages. We should be able to knock this out.
> I was advocating we don't fix it This is not a good reason not to fix it. Please, don't do that.
Explain why this is not a good reason. This package should be retired. The dependencies are incorrect and are either ignored or will cause the tests to use a version of unittest that doesn't match the standard library of the Python version being used. It's technical debt, so we should fix it. By fixing unittest2 it removes the incentive to do it quickly.
(In reply to Avram Lubkin from comment #19) > Explain why this is not a good reason. This package should be retired. The > dependencies are incorrect and are either ignored or will cause the tests to > use a version of unittest that doesn't match the standard library of the > Python version being used. It's technical debt, so we should fix it. By > fixing unittest2 it removes the incentive to do it quickly. I agree it's technical debt but iiuc some packages may need upstream changes to replace imports on unittest2 that will need time to get released?, i still think it's good to fix it in unittest2 until we can remove it everywhere.
Almost all of these are simple spec file fixes. There may be a couple that need a patch. If we run into a roadblock then it would make sense to carry unittest2 forward, but doesn't it make more sense to see if we hit a roadblock than to assume we will?
> Explain why this is not a good reason. Because the failure is blocking us now. We need to carry a downstream-downstream patch in order to proceed at the moment. This effort (getting rid of unittest2 for good) is very well needed, due time and appreciated, but it is not blocking anything now. It is like refusing to fix a leaking pipe because you need a new boiler anyway. Sure, let's use the leaking pipe as a reason to actually go and get a new boiler, but at least duct tape it or turn off the water before you go shopping.
Your metaphor is flawed. In this case the boiler is python-unittest2. It's old and you don't need it any more, but it's plumbed into a bunch of rooms even though the building also has a new boiler. The old boiler is broken and you want to fix it instead of re-plumbing the rooms. I get it, but it's why these things stick around for years. It's ok for things to be broken in rawhide, that's what it's for. So what if we need a downstream patch? We have a lot of those.
Rawhide is not for being broken. Broken rawhide blocks us. I need a downstream patch. Alfredo offered to carry it. Please let him. Until that happens I need to carry a local patch over the Fedora package.
Oh, I thought you were talking about a patch for testtools. No one is stopping him .I just think it's a waste.
Thanks. You are most likely correct an I see your point.
https://src.fedoraproject.org/rpms/python-unittest2/pull-request/2 should be fine. I just rebuilt it using python 3.9 from the copr.
https://bugzilla.redhat.com/show_bug.cgi?id=1789416 for retirement of python-traceback2.
ProDy-0:1.10.10-4.fc32.src dist-git beaker-0:27.0-1.fc32.src koji fedpkg-0:1.37-10.fc32.src already fixed by Miro obs-service-tar_scm-0:0.10.10-2.fc32.src dist-git pyephem-0:3.7.6.0-17.fc32.src https://src.fedoraproject.org/rpms/pyephem/pull-request/1 python-agate-0:1.6.1-7.fc32.src koji python-billiard-1:3.6.1.0-3.fc32.src koji python-case-0:1.5.3-4.fc32.src koji python-funcsigs-0:1.0.2-17.fc32.src koji python-leather-0:0.3.3-13.gite85dd30.fc32.src koji python-linecache2-0:1.0.0-24.fc32.src koji python-mimerender-0:0.6.0-5.fc32.src koji python-pyclipper-0:1.1.0-8.fc32.src koji python-pynwb-0:1.1.2-1.fc32.src koji python-rsa-0:3.4.2-12.fc32.src koji python-social-auth-core-0:1.7.0-8.fc32.src koji python-testtools-0:2.3.0-15.fc32.src TBD python-traceback2-0:1.4.0-25.fc32.src https://bugzilla.redhat.com/show_bug.cgi?id=1789416 tuned-0:2.13.0-1.fc32.src koji (koji means built in koji without BR:unittest2, dist-git means pushed but not built)
Thanks for all that! I did start looking at a patch for testtools. python-rsa https://src.fedoraproject.org/rpms/python-rsa/pull-request/2
Fixed. Feel free to open a separate bug for the retirement, but please don't reopen this one unless there is a failure with 3.9, we use the bugzillas for tracking. Thanks.