Bug 2247014 - numpy fails to build with Python 3.13: vendored meson fails with AttributeError: module 'importlib.resources' has no attribute 'path'
Summary: numpy fails to build with Python 3.13: vendored meson fails with AttributeErr...
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: numpy
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Miro Hrončok
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: PYTHON3.13
TreeView+ depends on / blocked
 
Reported: 2023-10-30 11:59 UTC by Karolina Surma
Modified: 2023-11-20 17:21 UTC (History)
7 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2023-11-20 17:13:06 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Fedora Package Sources numpy pull-request 40 0 None None None 2023-11-20 10:56:26 UTC
Github numpy numpy issues 25035 0 None open BUG: error: implicit declaration of function ‘_Py_HashDouble’ (building with Python 3.13~alpha1) 2023-10-30 16:13:32 UTC
Github numpy numpy issues 25036 0 None open MAINT: declare <ctype.h> explicitly 2023-10-30 16:13:32 UTC
Github numpy numpy pull 25198 0 None open MAINT: Replace deprecated ctypes.ARRAY(item_type, size) with item_type * size 2023-11-20 10:41:31 UTC

Description Karolina Surma 2023-10-30 11:59:47 UTC
numpy fails to build with Python 3.13.0a1.

+ /usr/bin/python3 -Bs /usr/lib/rpm/redhat/pyproject_wheel.py -Csetup-args=-Dblas=flexiblas -Csetup-args=-Dlapack=lapack /builddir/build/BUILD/numpy-1.26.0/pyproject-wheeldir
Processing /builddir/build/BUILD/numpy-1.26.0
  Preparing metadata (pyproject.toml): started
  Running command Preparing metadata (pyproject.toml)
  + /usr/bin/python3 /builddir/build/BUILD/numpy-1.26.0/vendored-meson/meson/meson.py setup /builddir/build/BUILD/numpy-1.26.0 /builddir/build/BUILD/numpy-1.26.0/.mesonpy-n_1_0z7t/build -Dbuildtype=release -Db_ndebug=if-release -Db_vscrt=md -Dblas=flexiblas -Dlapack=lapack --native-file=/builddir/build/BUILD/numpy-1.26.0/.mesonpy-n_1_0z7t/build/meson-python-native-file.ini
  Traceback (most recent call last):
    File "/builddir/build/BUILD/numpy-1.26.0/vendored-meson/meson/mesonbuild/mesonmain.py", line 194, in run
      return options.run_func(options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/builddir/build/BUILD/numpy-1.26.0/vendored-meson/meson/mesonbuild/msetup.py", line 358, in run
      app.generate()
    File "/builddir/build/BUILD/numpy-1.26.0/vendored-meson/meson/mesonbuild/msetup.py", line 183, in generate
      return self._generate(env, capture, vslite_ctx)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/builddir/build/BUILD/numpy-1.26.0/vendored-meson/meson/mesonbuild/msetup.py", line 228, in _generate
      intr.run()
    File "/builddir/build/BUILD/numpy-1.26.0/vendored-meson/meson/mesonbuild/interpreter/interpreter.py", line 2997, in run
      super().run()
    File "/builddir/build/BUILD/numpy-1.26.0/vendored-meson/meson/mesonbuild/interpreterbase/interpreterbase.py", line 164, in run
      self.evaluate_codeblock(self.ast, start=1)
    File "/builddir/build/BUILD/numpy-1.26.0/vendored-meson/meson/mesonbuild/interpreterbase/interpreterbase.py", line 190, in evaluate_codeblock
      raise e
    File "/builddir/build/BUILD/numpy-1.26.0/vendored-meson/meson/mesonbuild/interpreterbase/interpreterbase.py", line 182, in evaluate_codeblock
      self.evaluate_statement(cur)
    File "/builddir/build/BUILD/numpy-1.26.0/vendored-meson/meson/mesonbuild/interpreterbase/interpreterbase.py", line 198, in evaluate_statement
      self.assignment(cur)
    File "/builddir/build/BUILD/numpy-1.26.0/vendored-meson/meson/mesonbuild/interpreterbase/interpreterbase.py", line 635, in assignment
      value = self.evaluate_statement(node.value)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/builddir/build/BUILD/numpy-1.26.0/vendored-meson/meson/mesonbuild/interpreterbase/interpreterbase.py", line 200, in evaluate_statement
      return self.method_call(cur)
             ^^^^^^^^^^^^^^^^^^^^^
    File "/builddir/build/BUILD/numpy-1.26.0/vendored-meson/meson/mesonbuild/interpreterbase/interpreterbase.py", line 550, in method_call
      res = obj.method_call(method_name, args, kwargs)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/builddir/build/BUILD/numpy-1.26.0/vendored-meson/meson/mesonbuild/interpreter/interpreterobjects.py", line 811, in method_call
      ret = method(state, args, kwargs)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/builddir/build/BUILD/numpy-1.26.0/vendored-meson/meson/mesonbuild/interpreterbase/decorators.py", line 109, in wrapped
      ret = f(*wrapped_args, **wrapped_kwargs)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/builddir/build/BUILD/numpy-1.26.0/vendored-meson/meson/mesonbuild/interpreterbase/decorators.py", line 277, in wrapper
      return f(*nargs, **wrapped_kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/builddir/build/BUILD/numpy-1.26.0/vendored-meson/meson/mesonbuild/interpreterbase/decorators.py", line 596, in wrapper
      return f(*wrapped_args, **wrapped_kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/builddir/build/BUILD/numpy-1.26.0/vendored-meson/meson/mesonbuild/modules/python.py", line 437, in find_installation
      python = self._find_installation_impl(state, display_name, name_or_path, required)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/builddir/build/BUILD/numpy-1.26.0/vendored-meson/meson/mesonbuild/modules/python.py", line 393, in _find_installation_impl
      if python.sanity(state):
         ^^^^^^^^^^^^^^^^^^^^
    File "/builddir/build/BUILD/numpy-1.26.0/vendored-meson/meson/mesonbuild/modules/python.py", line 74, in sanity
      ret = super().sanity()
            ^^^^^^^^^^^^^^^^
    File "/builddir/build/BUILD/numpy-1.26.0/vendored-meson/meson/mesonbuild/dependencies/python.py", line 114, in sanity
      with importlib.resources.path('mesonbuild.scripts', 'python_info.py') as f:
           ^^^^^^^^^^^^^^^^^^^^^^^^
  AttributeError: module 'importlib.resources' has no attribute 'path'


The deprecated importlib.resources methods were removed from Python 3.13:
- contents()
- is_resource()
- open_binary()
- open_text()
- path()
- read_binary()
- read_text()
Use files() instead. Refer to importlib-resources: Migrating from Legacy for migration advice.
https://importlib-resources.readthedocs.io/en/latest/using.html#migrating-from-legacy


https://docs.python.org/3.13/whatsnew/3.13.html

For the build logs, see:
https://copr-be.cloud.fedoraproject.org/results/@python/python3.13/fedora-rawhide-x86_64/06578837-numpy/

For all our attempts to build numpy with Python 3.13, see:
https://copr.fedorainfracloud.org/coprs/g/python/python3.13/package/numpy/

Testing and mass rebuild of packages is happening in copr.
You can follow these instructions to test locally in mock if your package builds with Python 3.13:
https://copr.fedorainfracloud.org/coprs/g/python/python3.13/

Let us know here if you have any questions.

Python 3.13 is planned to be included in Fedora 41.
To make that update smoother, we're building Fedora packages with all pre-releases of Python 3.13.
A build failure prevents us from testing all dependent packages (transitive [Build]Requires),
so if this package is required a lot, it's important for us to get it fixed soon.

We'd appreciate help from the people who know this package best,
but if you don't want to work on this now, let us know so we can try to work around it on our side.

Comment 1 Miro Hrončok 2023-10-30 12:09:43 UTC
See also bz2246092 and https://github.com/mesonbuild/meson/issues/12401


I wonder if the vendored meson here could be unbundled... ? They appear to "simply" use a git submodule without custom changes in it.

Comment 2 Miro Hrončok 2023-10-30 13:45:26 UTC
> They appear to "simply" use a git submodule without custom changes in it.

Oh, the submodule goes to https://github.com/numpy/meson which actually is a fork.

Comment 3 Karolina Surma 2023-10-30 16:13:33 UTC
Patching the vendored meson and through backporting relevant upstream/main commits it's currently stuck at:

1. missing <ctype.h> declaration, added: https://github.com/numpy/numpy/pull/25036
2. removed ‘_Py_HashDouble’, reported upstream: https://github.com/numpy/numpy/issues/25035

Comment 4 Miro Hrončok 2023-11-20 10:18:25 UTC
At this point, numpy builds (without tests) with:

https://copr-dist-git.fedorainfracloud.org/cgit/@python/python3.13/numpy.git/plain/don-t-use-the-removed-importlib.patch?id=7452c637ffcf4dbfb1b00bbf1bd3c83c11d1a540

However, there are still some test failures:



=================================== FAILURES ===================================
____________________ TestLowlevelAPIAccess.test_loop_access ____________________

self = <numpy.core.tests.test_ufunc.TestLowlevelAPIAccess object at 0x7f6f21955a60>

    @pytest.mark.skipif(not hasattr(ct, "pythonapi"),
            reason="`ctypes.pythonapi` required for capsule unpacking.")
    def test_loop_access(self):
        # This is a basic test for the full strided loop access
>       data_t = ct.ARRAY(ct.c_char_p, 2)

self       = <numpy.core.tests.test_ufunc.TestLowlevelAPIAccess object at 0x7f6f21955a60>

../../../../BUILDROOT/numpy-1.26.0-2.fc40.x86_64/usr/lib64/python3.13/site-packages/numpy/core/tests/test_ufunc.py:2926: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/lib64/python3.13/ctypes/__init__.py:318: in ARRAY
    warnings._deprecated("ctypes.ARRAY", remove=(3, 15))
        len        = 2
        typ        = <class 'ctypes.c_char_p'>
        warnings   = <module 'warnings' from '/usr/lib64/python3.13/warnings.py'>
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

name = 'ctypes.ARRAY'
message = '{name!r} is deprecated and slated for removal in Python {remove}'

    def _deprecated(name, message=_DEPRECATED_MSG, *, remove, _version=sys.version_info):
        """Warn that *name* is deprecated or should be removed.
    
        RuntimeError is raised if *remove* specifies a major/minor tuple older than
        the current Python version or the same version but past the alpha.
    
        The *message* argument is formatted with *name* and *remove* as a Python
        version tuple (e.g. (3, 11)).
    
        """
        remove_formatted = f"{remove[0]}.{remove[1]}"
        if (_version[:2] > remove) or (_version[:2] == remove and _version[3] != "alpha"):
            msg = f"{name!r} was slated for removal after Python {remove_formatted} alpha"
            raise RuntimeError(msg)
        else:
            msg = message.format(name=name, remove=remove_formatted)
>           warn(msg, DeprecationWarning, stacklevel=3)
E           DeprecationWarning: 'ctypes.ARRAY' is deprecated and slated for removal in Python 3.15

_version   = sys.version_info(major=3, minor=13, micro=0, releaselevel='alpha', serial=1)
message    = '{name!r} is deprecated and slated for removal in Python {remove}'
msg        = "'ctypes.ARRAY' is deprecated and slated for removal in Python 3.15"
name       = 'ctypes.ARRAY'
remove     = (3, 15)
remove_formatted = '3.15'

/usr/lib64/python3.13/warnings.py:529: DeprecationWarning
_____________ test_deprecate_help_indentation[old_func4-old_func4] _____________

old_func = <function old_func4 at 0x7f6f1f7177e0>
new_func = <function old_func4 at 0x7f6f1f717880>

    @pytest.mark.skipif(sys.flags.optimize == 2, reason="-OO discards docstrings")
    @pytest.mark.parametrize('old_func, new_func', [
        (old_func4, new_func4),
        (old_func5, new_func5),
        (old_func6, new_func6),
    ])
    def test_deprecate_help_indentation(old_func, new_func):
        _compare_docs(old_func, new_func)
        # Ensure we don't mess up the indentation
        for knd, func in (('old', old_func), ('new', new_func)):
            for li, line in enumerate(func.__doc__.split('\n')):
                if li == 0:
                    assert line.startswith('    ') or not line.startswith(' '), knd
                elif line:
>                   assert line.startswith('    '), knd
E                   AssertionError: old
E                   assert False
E                    +  where False = <built-in method startswith of str object at 0x7f6f1a84ae70>('    ')
E                    +    where <built-in method startswith of str object at 0x7f6f1a84ae70> = 'Further info.'.startswith

func       = <function old_func4 at 0x7f6f1f7177e0>
knd        = 'old'
li         = 2
line       = 'Further info.'
new_func   = <function old_func4 at 0x7f6f1f717880>
old_func   = <function old_func4 at 0x7f6f1f7177e0>

../../../../BUILDROOT/numpy-1.26.0-2.fc40.x86_64/usr/lib64/python3.13/site-packages/numpy/lib/tests/test_utils.py:104: AssertionError
_____________ test_deprecate_help_indentation[old_func5-old_func5] _____________

old_func = <function old_func5 at 0x7f6f1f717920>
new_func = <function old_func5 at 0x7f6f1f7179c0>

    @pytest.mark.skipif(sys.flags.optimize == 2, reason="-OO discards docstrings")
    @pytest.mark.parametrize('old_func, new_func', [
        (old_func4, new_func4),
        (old_func5, new_func5),
        (old_func6, new_func6),
    ])
    def test_deprecate_help_indentation(old_func, new_func):
        _compare_docs(old_func, new_func)
        # Ensure we don't mess up the indentation
        for knd, func in (('old', old_func), ('new', new_func)):
            for li, line in enumerate(func.__doc__.split('\n')):
                if li == 0:
                    assert line.startswith('    ') or not line.startswith(' '), knd
                elif line:
>                   assert line.startswith('    '), knd
E                   AssertionError: old
E                   assert False
E                    +  where False = <built-in method startswith of str object at 0x7f6ef7d27530>('    ')
E                    +    where <built-in method startswith of str object at 0x7f6ef7d27530> = 'Bizarre indentation.'.startswith

func       = <function old_func5 at 0x7f6f1f717920>
knd        = 'old'
li         = 2
line       = 'Bizarre indentation.'
new_func   = <function old_func5 at 0x7f6f1f7179c0>
old_func   = <function old_func5 at 0x7f6f1f717920>

../../../../BUILDROOT/numpy-1.26.0-2.fc40.x86_64/usr/lib64/python3.13/site-packages/numpy/lib/tests/test_utils.py:104: AssertionError
_____________ test_deprecate_help_indentation[old_func6-old_func6] _____________

old_func = <function old_func6 at 0x7f6f1f717a60>
new_func = <function old_func6 at 0x7f6f1f717b00>

    @pytest.mark.skipif(sys.flags.optimize == 2, reason="-OO discards docstrings")
    @pytest.mark.parametrize('old_func, new_func', [
        (old_func4, new_func4),
        (old_func5, new_func5),
        (old_func6, new_func6),
    ])
    def test_deprecate_help_indentation(old_func, new_func):
        _compare_docs(old_func, new_func)
        # Ensure we don't mess up the indentation
        for knd, func in (('old', old_func), ('new', new_func)):
            for li, line in enumerate(func.__doc__.split('\n')):
                if li == 0:
                    assert line.startswith('    ') or not line.startswith(' '), knd
                elif line:
>                   assert line.startswith('    '), knd
E                   AssertionError: old
E                   assert False
E                    +  where False = <built-in method startswith of str object at 0x7f6f1a840730>('    ')
E                    +    where <built-in method startswith of str object at 0x7f6f1a840730> = 'Also in PEP-257.'.startswith

func       = <function old_func6 at 0x7f6f1f717a60>
knd        = 'old'
li         = 1
line       = 'Also in PEP-257.'
new_func   = <function old_func6 at 0x7f6f1f717b00>
old_func   = <function old_func6 at 0x7f6f1f717a60>

../../../../BUILDROOT/numpy-1.26.0-2.fc40.x86_64/usr/lib64/python3.13/site-packages/numpy/lib/tests/test_utils.py:104: AssertionError
______________________ test_deprecate_preserve_whitespace ______________________

    @pytest.mark.skipif(sys.flags.optimize == 2, reason="-OO discards docstrings")
    def test_deprecate_preserve_whitespace():
>       assert_('\n        Bizarre' in new_func5.__doc__)
E       AssertionError


../../../../BUILDROOT/numpy-1.26.0-2.fc40.x86_64/usr/lib64/python3.13/site-packages/numpy/lib/tests/test_utils.py:116: AssertionError

-----------------------------------------------------

One of the failures is caused by DeprecationWarning: 'ctypes.ARRAY' is deprecated and slated for removal in Python 3.15 -- we should look if upstream has fixed this, but otherwise not treating DeprecationWarning as errors is a way to unblock this.


All other failures seem to be caused by docstring dedentation: https://github.com/python/cpython/issues/81283 -- we should look if upstream has fixed this and offer a fix ourselves if needed.

Comment 5 Miro Hrončok 2023-11-20 10:28:42 UTC
The tests that fail due to the docstring dedentation were removed in https://github.com/numpy/numpy/pull/24154

The tested function was deprecated and while doing that, the tests were removed entirely. I don't quite understand that approach, but that is what upstream did. We should probably just deselect the tests until we update to a newer version.

Comment 6 Miro Hrončok 2023-11-20 10:56:26 UTC
https://src.fedoraproject.org/rpms/numpy/pull-request/40 should do the trick

Comment 7 Gwyn Ciesla 2023-11-20 16:59:53 UTC
Thank you, merged and building.

Comment 8 Miro Hrončok 2023-11-20 17:13:06 UTC
No need to build anything, that's why I did not include a changelog entry. Sorry about not making that clear.

Comment 9 Gwyn Ciesla 2023-11-20 17:21:22 UTC
Oh well. No worries. :)


Note You need to log in before you can comment on or make changes to this bug.