Bug 1903075 - python-zarr fails to build with Python 3.10
Summary: python-zarr fails to build with Python 3.10
Keywords:
Status: NEW
Alias: None
Product: Fedora
Classification: Fedora
Component: python-zarr
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Elliott Sales de Andrade
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On: 1904386
Blocks: PYTHON3.10
TreeView+ depends on / blocked
 
Reported: 2020-12-01 09:29 UTC by Tomáš Hrnčiar
Modified: 2021-02-09 16:24 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
Type: Bug


Attachments (Terms of Use)

Description Tomáš Hrnčiar 2020-12-01 09:29:35 UTC
python-zarr fails to build with Python 3.10.0a2.

Error log is too long to be pasted here. There is many test failures. Please see build logs.

For the build logs, see:
https://copr-be.cloud.fedoraproject.org/results/@python/python3.10/fedora-rawhide-x86_64/01806940-python-zarr/

For all our attempts to build python-zarr with Python 3.10, see:
https://copr.fedorainfracloud.org/coprs/g/python/python3.10/package/python-zarr/

Testing and mass rebuild of packages is happening in copr. You can follow these instructions to test locally in mock if your package builds with Python 3.10:
https://copr.fedorainfracloud.org/coprs/g/python/python3.10/

Let us know here if you have any questions.

Python 3.10 will be included in Fedora 35. To make that update smoother, we're building Fedora packages with early pre-releases of Python 3.10.
A build failure prevents us from testing all dependent packages (transitive [Build]Requires), so if this package is required a lot, it's important for us to get it fixed soon.
We'd appreciate help from the people who know this package best, but if you don't want to work on this now, let us know so we can try to work around it on our side.

Comment 1 Miro Hrončok 2020-12-01 16:08:06 UTC
Many failures are:

E       AttributeError: '_cffi_backend.buffer' object has no attribute 'tobytes'

Comment 2 Victor Stinner 2020-12-02 11:22:49 UTC
> E       AttributeError: '_cffi_backend.buffer' object has no attribute 'tobytes'

I checked cffi archives and it seems like this type never had a tobytes() method. I guess that previously, zarr didn't get a '_cffi_backend.buffer' object, but another type.

Maybe buffer[:] or bytes(buffer) can be used instead.

I don't understand what is the "ffi" module in the CFFI documentation. I found ffi.buffer documentation, not sure if it's related:
https://cffi.readthedocs.io/en/latest/ref.html#ffi-buffer-ffi-from-buffer

Comment 3 Tomáš Hrnčiar 2020-12-03 07:45:11 UTC
New version of zarr - 2.6.1 is now available. I tried to rebuild it in our Python3.10 COPR, but the errors are still present.

https://copr.fedorainfracloud.org/coprs/g/python/python3.10/build/1810021/

Comment 4 Victor Stinner 2020-12-03 14:03:43 UTC
I reported the test failures to upstream: https://github.com/zarr-developers/zarr-python/issues/669

Comment 5 Matthias Bussonnier 2020-12-03 17:33:44 UTC
That is likely an issue with Python-LMDB; 

My redhat-foo is not that good, but it looks like from googling that Python-lmdb on EPEL is 0.92 (from 2016), and that there was many releases (8) since then, last one August 28 2020. Can you confirm your version of Python-lmdb ? 

Note that zarr-python failure due to lmdb are non-fatal as they will affect only people using lmdb stores. You can also just not-install python-lmdb when running the test suite; as it will automatically skip the lmdb test when it is not present.

I'm happy to add a conditional version check in zarr  on Python and Python-LMDB version that skip the test on 3.10 with a known incompatible version of LMDB if that can help.

Comment 6 Petr Viktorin 2020-12-03 20:04:06 UTC
Thank you for the pointer! Indeed, python-lmdb is at 0.92 even in Fedora.

Here's a PR to update it to 1.0.0: https://src.fedoraproject.org/rpms/python-lmdb/pull-request/1

Comment 7 Petr Viktorin 2020-12-03 22:50:59 UTC
Indeed, python-zarr builds with Python 3.10 and python-lmdb 1.0.0: https://copr.fedorainfracloud.org/coprs/g/python/python3.10/build/1810977/

Comment 8 Matthias Bussonnier 2020-12-03 23:07:30 UTC
Ok, that is great. We've also added a workaround on zarr-python, so next release might not need python-lmdb 1.0.0 to work. 
Merging the workaround have closed the upstream issue; and as you point out that with LMDB 1.0.0 build appear to succeed; I'm going to assume it's ok to keep the upstream bug closed. Let me know if you need to re-open, and/or if there is anything we can do to make your life easier like notifying you on new release, or including specific patches/files in the source.

Comment 9 Petr Viktorin 2020-12-04 09:51:17 UTC
Yes, it's definitely OK to leave it closed. Thank you for investigating this one!

We get release notifications from PyPI. (It's set up for zarr; I've recommended it to Fedora's lmdb maintainer as well.)

Comment 10 Ben Cotton 2021-02-09 15:29:20 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 34 development cycle.
Changing version to 34.


Note You need to log in before you can comment on or make changes to this bug.