Bug 1791769 - clufter fails to build with Python 3.9: ValueError became an ImportError
Summary: clufter fails to build with Python 3.9: ValueError became an ImportError
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: clufter
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Jan Pokorný [poki]
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: F33FTBFS PYTHON39
TreeView+ depends on / blocked
 
Reported: 2020-01-16 12:48 UTC by Miro Hrončok
Modified: 2020-06-11 11:48 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-06-11 11:48:22 UTC
Type: Bug


Attachments (Terms of Use)

Description Miro Hrončok 2020-01-16 12:48:37 UTC
clufter fails to build with Python 3.9.0a2.

Traceback (most recent call last):
  File "/builddir/build/BUILD/clufter-0.77.2/./run-dev", line 87, in <module>
    from . import __main__
  File "/builddir/build/BUILD/clufter-0.77.2/clufter/__main__.py", line 11, in <module>
    from .main import run
  File "/builddir/build/BUILD/clufter-0.77.2/clufter/main.py", line 33, in <module>
    from .command_manager import CommandManager
  File "/builddir/build/BUILD/clufter-0.77.2/clufter/command_manager.py", line 16, in <module>
    from .command import commands, CommandAlias
  File "/builddir/build/BUILD/clufter-0.77.2/clufter/command.py", line 8, in <module>
    from collections import MutableMapping
ImportError: cannot import name 'MutableMapping' from 'collections' (/usr/lib64/python3.9/collections/__init__.py)

See https://docs.python.org/3.9/whatsnew/3.9.html#removed

"The abstract base classes in collections.abc no longer are exposed in the regular collections module. This will help create a clearer distinction between the concrete classes and the abstract base classes."

For the build logs, see:
https://copr-be.cloud.fedoraproject.org/results/@python/python3.9/fedora-rawhide-x86_64/01148355-clufter/

For all our attempts to build clufter with Python 3.9, see:
https://copr.fedorainfracloud.org/coprs/g/python/python3.9/package/clufter/

Testing and mass rebuild of packages is happening in copr. You can follow these instructions to test locally in mock if your package builds with Python 3.9:
https://copr.fedorainfracloud.org/coprs/g/python/python3.9/

Let us know here if you have any questions.

Python 3.9 will be included in Fedora 33. To make that update smoother, we're building Fedora packages with early pre-releases of Python 3.9.
A build failure prevents us from testing all dependent packages (transitive [Build]Requires), so it's important for us to get it fixed soon.
We'd appreciate help from the people who know this package best, but if you don't want to work on this now, let us know so we can try to work around it on our side.

Comment 1 Jan Pokorný [poki] 2020-01-16 13:23:26 UTC
Thanks for the heads-up and details.

> it's important for us to get it fixed soon

Looking...

Comment 2 Jan Pokorný [poki] 2020-01-18 00:54:45 UTC
https://koji.fedoraproject.org/koji/taskinfo?taskID=40681936
->  clufter-0.77.2-6.fc32

Miro, one really subtle thing coming to 3.9, if you happen to read
this (not that ValueError did not use to be suspicious to me from
the beginning per the original comments, but still, rather unexpected
something like that would occur this relatively late to the major
version):

https://pagure.io/clufter/c/2b1b834972a8834410d9e0c348c9aeda07c010af?branch=next

Comment 3 Miro Hrončok 2020-01-18 09:27:53 UTC
What do you mean by "relatively late"? Python 3.X and 3.X+1 subtle incompatibilities are not getting any lower with higher X.

Anyway, I suppose the ValueError was a bug, now fixed.

Comment 4 Jan Pokorný [poki] 2020-01-18 22:06:50 UTC
I've meant, I find it rather wild west when something established since
at least 2.6 receives this "tweak" as late as in 3.9.  Now, someone
starts developing with 3.9, only for a surprise moment at the innocent
place like this when deployed with say 3.8 (because that's what's
supported in their distro of choice, for instance).

Maybe 3.x line will go on virtually forever and cognitive load
on breaking changes will not decline in time and everyone's fine
with that, but IMHO 2.x set some expectations that incompatibilities
are very very rare after 2.6.0, even such a change would likely be
no-go between 2.6 these two.  I dunno, just feelings, based on not
expecting something like this coming without warning (unlike with
the former problem you've raised, there was some head time about
it eventually happening, at least, if only I had paid attention) :-)

Also ... proactive defence against library/environment changes like
that (once/if one gets comfortable that these are apparently inevitable)
is one of the most terrible ideas perhaps ... to routinely start
excepting the most generaly exception classes (actual antipattern?).
Do Python core devs want to send such a signal and teach the crowds
this is an encouraged way of forward compatibility maintenance?
It doesn't look to me this was judged taking all these possible
implications into account, but afterall, who am I?

(purely rhetorical to justify my stance that perhaps owed some
explaining, no more replies insisted on, thanks a lot for pushing
this forward in such as graceful manner)

Comment 5 Miro Hrončok 2020-01-18 23:45:44 UTC
Victor, do you know where this was changed and whether it is a known difference between 3.8 and 3.9?

Comment 6 Miro Hrončok 2020-01-18 23:48:00 UTC
Oh, never mind, found it, first thing on https://docs.python.org/3.9/whatsnew/3.9.html

"__import__() now raises ImportError instead of ValueError, which used to occur when a relative import went past its top-level package."

https://bugs.python.org/issue37444

Comment 7 Jan Pokorný [poki] 2020-01-20 00:09:39 UTC
Yep, what was also explicitly linked in the commit msg.
That assumes your attention will survive all those items,
if you have enough discipline to review that to begin with.

Comment 8 Hugo van Kemenade 2020-01-20 21:07:24 UTC
Please see PR https://github.com/jnpkrn/clufter/pull/1 for the collections.abc fixes.

Comment 9 Jan Pokorný [poki] 2020-01-21 20:42:21 UTC
Hugo, thanks for the effort ... but now I plain regret that
I don't appear to do a good job of announcing:

- GitHub serves me rather as a mirror and Travis CI runner
  (that's why I also disabled issues there), primary is pagure.io
  hosting: https://pagure.io/clufter
  
  -> I've just fixed that (again, my apologies!)

- master branch serves to host stable code at the latest release
  snapshot, development happens on "next" branch ("Development version",
  very rarely, I even force-push there -- never to master, and
  only master hosts the tags)

  -> have marked next as "default branch"
     (I don't think there was this option back then and haven't
     looked at the settings until now)

  -> for good measure, did the same at pagure.io primary hosting

  -> will add more to the README file

That will hopefully explain (or in hindsight be explained by) the
https://pagure.io/clufter/c/2b1b834972a8834410d9e0c348c9aeda07c010af?branch=next
link in [comment 2] above.

Now to your changes, they in fact redo the pre-existing commit
I already had in next by that time:

https://github.com/jnpkrn/clufter/commit/b83e3091a7febd715cc0502dc154e43edb2d44f1
if you want GH, otherwise
https://pagure.io/clufter/c/b83e3091a7febd715cc0502dc154e43edb2d44f1?branch=next

(Also note that this alone wasn't sufficient for a successful
test suite run, see ValueError/ImportError discussed above).

That being said, Hugo, to retain at least something of your effort,
I'll be grateful if you can amend that pull request to only
contain that 3.9 addition to .travis.yml file, and retarget it
against next branch.  Thanks in advance!
It will make it to master right before the next release.

And thanks for understanding, sorry about this perhaps unexpected
obstacle, I deserve my portion of non-transparency shame here.

Comment 10 Hugo van Kemenade 2020-01-22 09:46:46 UTC
No problem! I've updated the PR, cheers!

Comment 11 Jan Pokorný [poki] 2020-01-22 10:53:43 UTC
Gladly taken, thanks!

Comment 12 Jan Pokorný [poki] 2020-01-22 10:56:50 UTC
(didn't mean to flip the status, something annoying going on)

Comment 13 Ben Cotton 2020-02-11 17:29:20 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 32 development cycle.
Changing version to 32.

Comment 14 Fedora Admin XMLRPC Client 2020-04-19 16:29:04 UTC
This package has changed maintainer in the Fedora.
Reassigning to the new maintainer of this component.

Comment 15 Miro Hrončok 2020-05-25 13:02:05 UTC
This comment is mass posted to all bugs blocking the Python 3.9 tracker, sorry if it is not 100 % relevant. When in doubt, please ask.


The Python 3.9 rebuild is in progress in a Koji side tag.

If you fix this bug, please don't rebuild the package in regular rawhide, but do it in the side tag with:

    $ fedpkg build --target=f33-python

The rebuild is progressing slowly and it is possible this package won't have all the required build dependencies yet. If that's the case, please just leave the fix committed and pushed and we will eventually rebuild it for you.

You are not asked to go and try rebuild all the missing dependencies yourself. If you know there is a bootstrap loop in the dependencies, let me know and we can untangle it together.

If you want to test your fix or reproduce the failure, you can still use the Copr repo mentioned in the initial comment of this bug: https://copr.fedorainfracloud.org/coprs/g/python/python3.9/

Comment 16 Miro Hrončok 2020-05-29 07:11:56 UTC
Python 3.9 update: The f33-python side tag is currently being merged.

New builds in f33-python are no longer possible, but python3 is not yet updated to Python 3.9 in rawhide. You can check when Python is Python 3.9 with:

    $ koji wait-repo f33-build --build python3.9-3.9.0~b1-3.fc3

And build the packages normally after that.

Comment 17 Miro Hrončok 2020-06-11 11:48:22 UTC
This is a bulk close of Python 3.9 bugzillas of packages that successfully built.
If this remained open for a reason, I am sorry and feel free to reopen.


Note You need to log in before you can comment on or make changes to this bug.