Bug 1964955 - pyproj fails to build with Python 3.10: TypeError: invalid Flag 'GeodIntermediateFlag' -- missing values: 4, 8, 32, 64, 128, 512, 1024, 2048
Summary: pyproj fails to build with Python 3.10: TypeError: invalid Flag 'GeodIntermed...
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: pyproj
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Jos de Kloe
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: PYTHON3.10
TreeView+ depends on / blocked
 
Reported: 2021-05-26 12:24 UTC by Tomáš Hrnčiar
Modified: 2021-05-26 19:53 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-05-26 19:53:15 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Tomáš Hrnčiar 2021-05-26 12:24:24 UTC
pyproj fails to build with Python 3.10.0b1.

I am not sure if this is connected with Python 3.10, but scratch build on rawhide passed. 

Configuration error:
There is a programmable error in your configuration file:

Traceback (most recent call last):
  File "/usr/lib/python3.10/site-packages/sphinx/config.py", line 327, in eval_config_file
    execfile_(filename, namespace)
  File "/usr/lib/python3.10/site-packages/sphinx/util/pycompat.py", line 88, in execfile_
    exec(code, _globals)
  File "/builddir/build/BUILD/pyproj-3.1.0/docs/conf.py", line 23, in <module>
    import pyproj
  File "/builddir/build/BUILD/pyproj-3.1.0/build/lib.linux-x86_64-3.10/pyproj/__init__.py", line 55, in <module>
    from pyproj.crs import CRS  # noqa: F401
  File "/builddir/build/BUILD/pyproj-3.1.0/build/lib.linux-x86_64-3.10/pyproj/crs/__init__.py", line 7, in <module>
    from pyproj._crs import (  # noqa: F401
  File "pyproj/_crs.pyx", line 12, in init pyproj._crs
  File "/builddir/build/BUILD/pyproj-3.1.0/build/lib.linux-x86_64-3.10/pyproj/crs/enums.py", line 4, in <module>
    from pyproj.enums import BaseEnum
  File "/builddir/build/BUILD/pyproj-3.1.0/build/lib.linux-x86_64-3.10/pyproj/enums.py", line 141, in <module>
    class GeodIntermediateFlag(IntFlag):
  File "/usr/lib64/python3.10/enum.py", line 544, in __new__
    raise TypeError(
TypeError: invalid Flag 'GeodIntermediateFlag' -- missing values: 4, 8, 32, 64, 128, 512, 1024, 2048

For the build logs, see:
https://copr-be.cloud.fedoraproject.org/results/@python/python3.10/fedora-rawhide-x86_64/02207050-pyproj/

For all our attempts to build pyproj with Python 3.10, see:
https://copr.fedorainfracloud.org/coprs/g/python/python3.10/package/pyproj/

Testing and mass rebuild of packages is happening in copr. You can follow these instructions to test locally in mock if your package builds with Python 3.10:
https://copr.fedorainfracloud.org/coprs/g/python/python3.10/

Let us know here if you have any questions.

Python 3.10 will be included in Fedora 35. To make that update smoother, we're building Fedora packages with early pre-releases of Python 3.10.
A build failure prevents us from testing all dependent packages (transitive [Build]Requires), so if this package is required a lot, it's important for us to get it fixed soon.
We'd appreciate help from the people who know this package best, but if you don't want to work on this now, let us know so we can try to work around it on our side.

Comment 1 Jos de Kloe 2021-05-26 19:07:10 UTC
Hi Tomáš,

thanks a lot for reporting this issue.
Using the COPR test possibility that you provided I could reproduce the problem on my local machine.

I think this is caused by some undocumented behaviour in python 3.10.0b1 (or maybe I could just not find it in the documentation or maybe it is a bug in this python version?).
Anyway, thanks to the COPR test possibility I could quickly zoom in to the problem. It seems that it is no longer allowed in this python version to define an enum.IntFlag subclass with a sequence of non consecutive bit values. And this is how IntFlag is used at the moment in the pyproj/enum.py module. If I add dummy flag constants to ensure the full range of bits between the minimum and maximum used value is defined then the code runs just fine again, also for python 3.10.0b1, in my local mock test.

I will now kick off a koji build for regular rawhide to ensure I did not break anything there.

Comment 2 Miro Hrončok 2021-05-26 19:15:59 UTC
This change seem to have introduced the behavior: https://github.com/python/cpython/commit/7aaeb2a3d682ecba125c33511e4b4796021d2f82

Comment 3 Miro Hrončok 2021-05-26 19:32:42 UTC
I've reported https://bugs.python.org/issue44242

Comment 4 Jos de Kloe 2021-05-26 19:36:39 UTC
The koji build in rawhide succeeded for python 3.9.5, so you can try to re-build pyproj now in COPR.

Miro, thanks for reporting this to bugs.python.org.
From your answer I get that this is actual new behavior for python3.10.0b1 and not a bug in itself, but more a lack in the documentation, so I will inform upstream about this.

From the change in git it seems that another way to solve this would be to use something like:

class GeodIntermediateFlag(IntFlag, boundary=enum.KEEP):

in stead of:

class GeodIntermediateFlag(IntFlag):

But actually I am not sure if I like that. I find that a lot less obvious to understand when just reading the code.

Comment 5 Miro Hrončok 2021-05-26 19:53:15 UTC
Also, boundary=enum.KEEP is not yet available in Python <= 3.9, so you would need to use something a bit far-fetched for compatibility, e.g.:

try:
    from enum import IntFlag, KEEP
    INTFLAG_KWARGS = {'boundary': KEEP}
except ImportError:
    INTFLAG_KWARGS = {}


class GeodIntermediateFlag(IntFlag, **INTFLAG_KWARGS):
    ...


I would not jump into conclusions whether or not this is a bug or just missing documentation. Backwards incompatible changes in Python have rules and boundary=enum.KEEP might just as well become the default.



However, your solution with UNUSED enum values gets the job done, so I'll close this as fixed. Thanks!


Note You need to log in before you can comment on or make changes to this bug.