Bug 1897208 - petsc4py fails to build with Python 3.10: Py_SIZE= needs to be replaced with Py_SET_SIZE
Summary: petsc4py fails to build with Python 3.10: Py_SIZE= needs to be replaced with ...
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Fedora
Classification: Fedora
Component: petsc4py
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Antonio T. sagitter
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: PYTHON3.10
TreeView+ depends on / blocked
 
Reported: 2020-11-12 15:05 UTC by Tomáš Hrnčiar
Modified: 2020-11-18 19:50 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-11-18 19:50:10 UTC
Type: Bug

Attachments (Terms of Use)
Add an attachment (proposed patch, testcase, etc.)


Links
System ID Private Priority Status Summary Last Updated
Gitlab petsc/petsc4py/-/issues/9 0 None None None 2020-11-13 11:12:38 UTC

Description Tomáš Hrnčiar 2020-11-12 15:05:04 UTC 
petsc4py fails to build with Python 3.10.0a2.

src/petsc4py.PETSc.c:2604:23: error: lvalue required as left operand of assignment
 2604 |         Py_SIZE(list) = len+1;
      |                       ^
src/petsc4py.PETSc.c: In function ‘__Pyx_ListComp_Append’:
src/petsc4py.PETSc.c:2921:23: error: lvalue required as left operand of assignment
 2921 |         Py_SIZE(list) = len+1;
      |                       ^
In file included from src/PETSc.c:4:
src/petsc4py.PETSc.c: In function ‘__pyx_tp_dealloc_8petsc4py_5PETSc_Comm’:
src/petsc4py.PETSc.c:302802:5: error: lvalue required as increment operand
302802 |     ++Py_REFCNT(o);
       |     ^~
src/petsc4py.PETSc.c:302804:5: error: lvalue required as decrement operand
302804 |     --Py_REFCNT(o);
       |     ^~
src/petsc4py.PETSc.c: In function ‘__pyx_tp_dealloc_8petsc4py_5PETSc_Object’:
src/petsc4py.PETSc.c:303020:5: error: lvalue required as increment operand
303020 |     ++Py_REFCNT(o);
       |     ^~
src/petsc4py.PETSc.c:303022:5: error: lvalue required as decrement operand
303022 |     --Py_REFCNT(o);
       |     ^~
src/petsc4py.PETSc.c: In function ‘__pyx_tp_dealloc_8petsc4py_5PETSc_SF’:
src/petsc4py.PETSc.c:303908:5: error: lvalue required as increment operand
303908 |     ++Py_REFCNT(o);
       |     ^~
src/petsc4py.PETSc.c:303910:5: error: lvalue required as decrement operand
303910 |     --Py_REFCNT(o);
       |     ^~
src/petsc4py.PETSc.c: In function ‘__pyx_tp_dealloc_8petsc4py_5PETSc_Section’:
src/petsc4py.PETSc.c:304750:5: error: lvalue required as increment operand
304750 |     ++Py_REFCNT(o);
       |     ^~
src/petsc4py.PETSc.c:304752:5: error: lvalue required as decrement operand
304752 |     --Py_REFCNT(o);
       |     ^~
src/petsc4py.PETSc.c: In function ‘__pyx_tp_dealloc_8petsc4py_5PETSc__IS_buffer’:
src/petsc4py.PETSc.c:307767:5: error: lvalue required as increment operand
307767 |     ++Py_REFCNT(o);
       |     ^~
src/petsc4py.PETSc.c:307769:5: error: lvalue required as decrement operand
307769 |     --Py_REFCNT(o);
       |     ^~
src/petsc4py.PETSc.c: In function ‘__pyx_tp_dealloc_8petsc4py_5PETSc__Vec_buffer’:
src/petsc4py.PETSc.c:307904:5: error: lvalue required as increment operand
307904 |     ++Py_REFCNT(o);
       |     ^~
src/petsc4py.PETSc.c:307906:5: error: lvalue required as decrement operand
307906 |     --Py_REFCNT(o);
       |     ^~
src/petsc4py.PETSc.c: In function ‘__pyx_tp_dealloc_8petsc4py_5PETSc__DMComposite_access’:
src/petsc4py.PETSc.c:308561:5: error: lvalue required as increment operand
308561 |     ++Py_REFCNT(o);
       |     ^~
src/petsc4py.PETSc.c:308563:5: error: lvalue required as decrement operand
308563 |     --Py_REFCNT(o);
       |     ^~
src/petsc4py.PETSc.c: In function ‘__pyx_tp_dealloc_8petsc4py_5PETSc_Options’:
src/petsc4py.PETSc.c:308702:5: error: lvalue required as increment operand
308702 |     ++Py_REFCNT(o);
       |     ^~
src/petsc4py.PETSc.c:308704:5: error: lvalue required as decrement operand
308704 |     --Py_REFCNT(o);
       |     ^~
src/petsc4py.PETSc.c: In function ‘__pyx_tp_dealloc_array’:
src/petsc4py.PETSc.c:310700:5: error: lvalue required as increment operand
310700 |     ++Py_REFCNT(o);
       |     ^~
src/petsc4py.PETSc.c:310702:5: error: lvalue required as decrement operand
310702 |     --Py_REFCNT(o);
       |     ^~
src/petsc4py.PETSc.c: In function ‘__pyx_tp_dealloc_memoryview’:
src/petsc4py.PETSc.c:311011:5: error: lvalue required as increment operand
311011 |     ++Py_REFCNT(o);
       |     ^~
src/petsc4py.PETSc.c:311013:5: error: lvalue required as decrement operand
311013 |     --Py_REFCNT(o);
       |     ^~
src/petsc4py.PETSc.c: In function ‘__pyx_tp_dealloc__memoryviewslice’:
src/petsc4py.PETSc.c:311261:5: error: lvalue required as increment operand
311261 |     ++Py_REFCNT(o);
       |     ^~
src/petsc4py.PETSc.c:311263:5: error: lvalue required as decrement operand
311263 |     --Py_REFCNT(o);
       |     ^~
src/petsc4py.PETSc.c: In function ‘__Pyx_decode_c_bytes’:
src/petsc4py.PETSc.c:329649:9: warning: ‘PyUnicode_FromUnicode’ is deprecated [-Wdeprecated-declarations]
329649 |         return PyUnicode_FromUnicode(NULL, 0);
       |         ^~~~~~
In file included from /usr/include/python3.10/unicodeobject.h:1046,
                 from /usr/include/python3.10/Python.h:97,
                 from src/petsc4py.PETSc.c:4,
                 from src/PETSc.c:4:
/usr/include/python3.10/cpython/unicodeobject.h:551:42: note: declared here
  551 | Py_DEPRECATED(3.3) PyAPI_FUNC(PyObject*) PyUnicode_FromUnicode(
      |                                          ^~~~~~~~~~~~~~~~~~~~~
In file included from src/PETSc.c:4:
src/petsc4py.PETSc.c: In function ‘__Pyx__PyList_PopIndex’:
src/petsc4py.PETSc.c:330605:24: error: lvalue required as left operand of assignment
330605 |             Py_SIZE(L) -= 1;
       |                        ^~
src/petsc4py.PETSc.c: In function ‘__Pyx_ParseOptionalKeywords’:
src/petsc4py.PETSc.c:331043:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
331043 |                     (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
       |                     ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
                 from /usr/include/python3.10/Python.h:97,
                 from src/petsc4py.PETSc.c:4,
                 from src/PETSc.c:4:
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
  446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
      |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from src/PETSc.c:4:
src/petsc4py.PETSc.c:331043:21: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
331043 |                     (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
       |                     ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
                 from /usr/include/python3.10/Python.h:97,
                 from src/petsc4py.PETSc.c:4,
                 from src/PETSc.c:4:
/usr/include/python3.10/cpython/unicodeobject.h:580:45: note: declared here
  580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
      |                                             ^~~~~~~~~~~~~~~~~~~
In file included from src/PETSc.c:4:
src/petsc4py.PETSc.c:331043:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
331043 |                     (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
       |                     ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
                 from /usr/include/python3.10/Python.h:97,
                 from src/petsc4py.PETSc.c:4,
                 from src/PETSc.c:4:
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
  446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
      |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from src/PETSc.c:4:
src/petsc4py.PETSc.c:331043:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
331043 |                     (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
       |                     ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
                 from /usr/include/python3.10/Python.h:97,
                 from src/petsc4py.PETSc.c:4,
                 from src/PETSc.c:4:
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
  446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
      |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from src/PETSc.c:4:
src/petsc4py.PETSc.c:331043:21: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
331043 |                     (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
       |                     ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
                 from /usr/include/python3.10/Python.h:97,
                 from src/petsc4py.PETSc.c:4,
                 from src/PETSc.c:4:
/usr/include/python3.10/cpython/unicodeobject.h:580:45: note: declared here
  580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
      |                                             ^~~~~~~~~~~~~~~~~~~
In file included from src/PETSc.c:4:
src/petsc4py.PETSc.c:331043:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
331043 |                     (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
       |                     ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
                 from /usr/include/python3.10/Python.h:97,
                 from src/petsc4py.PETSc.c:4,
                 from src/PETSc.c:4:
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
  446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
      |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from src/PETSc.c:4:
src/petsc4py.PETSc.c:331059:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
331059 |                         (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
       |                         ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
                 from /usr/include/python3.10/Python.h:97,
                 from src/petsc4py.PETSc.c:4,
                 from src/PETSc.c:4:
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
  446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
      |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from src/PETSc.c:4:
src/petsc4py.PETSc.c:331059:25: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
331059 |                         (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
       |                         ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
                 from /usr/include/python3.10/Python.h:97,
                 from src/petsc4py.PETSc.c:4,
                 from src/PETSc.c:4:
/usr/include/python3.10/cpython/unicodeobject.h:580:45: note: declared here
  580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
      |                                             ^~~~~~~~~~~~~~~~~~~
In file included from src/PETSc.c:4:
src/petsc4py.PETSc.c:331059:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
331059 |                         (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
       |                         ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
                 from /usr/include/python3.10/Python.h:97,
                 from src/petsc4py.PETSc.c:4,
                 from src/PETSc.c:4:
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
  446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
      |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from src/PETSc.c:4:
src/petsc4py.PETSc.c:331059:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
331059 |                         (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
       |                         ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
                 from /usr/include/python3.10/Python.h:97,
                 from src/petsc4py.PETSc.c:4,
                 from src/PETSc.c:4:
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
  446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
      |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from src/PETSc.c:4:
src/petsc4py.PETSc.c:331059:25: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
331059 |                         (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
       |                         ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
                 from /usr/include/python3.10/Python.h:97,
                 from src/petsc4py.PETSc.c:4,
                 from src/PETSc.c:4:
/usr/include/python3.10/cpython/unicodeobject.h:580:45: note: declared here
  580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
      |                                             ^~~~~~~~~~~~~~~~~~~
In file included from src/PETSc.c:4:
src/petsc4py.PETSc.c:331059:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
331059 |                         (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
       |                         ^
In file included from /usr/include/python3.10/unicodeobject.h:1046,
                 from /usr/include/python3.10/Python.h:97,
                 from src/petsc4py.PETSc.c:4,
                 from src/PETSc.c:4:
/usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
  446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
      |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from src/PETSc.c:4:
src/petsc4py.PETSc.c: In function ‘__Pyx_PyList_Pop’:
src/petsc4py.PETSc.c:331906:20: error: lvalue required as left operand of assignment
331906 |         Py_SIZE(L) -= 1;
       |                    ^~
src/petsc4py.PETSc.c: In function ‘__Pyx_decode_c_string’:
src/petsc4py.PETSc.c:332131:9: warning: ‘PyUnicode_FromUnicode’ is deprecated [-Wdeprecated-declarations]
332131 |         return PyUnicode_FromUnicode(NULL, 0);
       |         ^~~~~~
In file included from /usr/include/python3.10/unicodeobject.h:1046,
                 from /usr/include/python3.10/Python.h:97,
                 from src/petsc4py.PETSc.c:4,
                 from src/PETSc.c:4:
/usr/include/python3.10/cpython/unicodeobject.h:551:42: note: declared here
  551 | Py_DEPRECATED(3.3) PyAPI_FUNC(PyObject*) PyUnicode_FromUnicode(
      |                                          ^~~~~~~~~~~~~~~~~~~~~

https://docs.python.org/3.10/whatsnew/changelog.html#id11

bpo-39573: Convert Py_REFCNT() and Py_SIZE() macros to static inline functions. They cannot be used as l-value anymore: use Py_SET_REFCNT() and Py_SET_SIZE() to set an object reference count and size. This change is backward incompatible on purpose, to prepare the C API for an opaque PyObject structure.


For the build logs, see:
https://copr-be.cloud.fedoraproject.org/results/@python/python3.10/fedora-rawhide-x86_64/01757218-petsc4py/

For all our attempts to build petsc4py with Python 3.10, see:
https://copr.fedorainfracloud.org/coprs/g/python/python3.10/package/petsc4py/

Testing and mass rebuild of packages is happening in copr. You can follow these instructions to test locally in mock if your package builds with Python 3.10:
https://copr.fedorainfracloud.org/coprs/g/python/python3.10/

Let us know here if you have any questions.

Python 3.10 will be included in Fedora 35. To make that update smoother, we're building Fedora packages with early pre-releases of Python 3.10.
A build failure prevents us from testing all dependent packages (transitive [Build]Requires), so if this package is required a lot, it's important for us to get it fixed soon.
We'd appreciate help from the people who know this package best, but if you don't want to work on this now, let us know so we can try to work around it on our side.
Comment 1 Miro Hrončok 2020-11-13 12:03:43 UTC 
A suggested in the upstream issue, the C sources need to be regenerated.

I've tried adding this in %prep:

rm -v $(grep -rl '/\* Generated by Cython')



It get's past the compilation, but than it fail sin %check:



+ /usr/bin/python3 setup.py test
running test
Traceback (most recent call last):
  File "/builddir/build/BUILD/petsc4py-3.13.0/petsc4py-openmpi/setup.py", line 270, in <module>
    main()
  File "/builddir/build/BUILD/petsc4py-3.13.0/petsc4py-openmpi/setup.py", line 267, in main
    run_setup()
  File "/builddir/build/BUILD/petsc4py-3.13.0/petsc4py-openmpi/setup.py", line 119, in run_setup
    setup(packages     = ['petsc4py',
  File "/usr/lib/python3.10/site-packages/setuptools/__init__.py", line 153, in setup
    return distutils.core.setup(**attrs)
  File "/usr/lib64/python3.10/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/lib64/python3.10/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/usr/lib64/python3.10/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/builddir/build/BUILD/petsc4py-3.13.0/petsc4py-openmpi/setup.py", line 258, in run_testsuite
    err = main(args)
  File "test/runtests.py", line 203, in main
    package = import_package(options, pkgname)
  File "test/runtests.py", line 84, in import_package
    package = __import__(pkgname)
ModuleNotFoundError: No module named 'petsc4py'
Comment 2 Antonio T. sagitter 2020-11-13 14:17:32 UTC 
petsc4py should be fixed for Python-3.10 with last build release.

Thank you Miro.
Comment 3 Miro Hrončok 2020-11-13 14:31:17 UTC 
Antonio, can you please drop the conditional? Regenerating the C sources is always the right thing to do:

https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/#_packages_using_cython
Comment 4 Antonio T. sagitter 2020-11-14 11:11:38 UTC 
Fixed.

If i regenerate the C sources in ARM arch, OpenMPI tests strangely fail.
Comment 5 Miro Hrončok 2020-11-18 17:57:10 UTC 
The change was reverted in Python 3.10, because it caused too much trouble (the amount of affected Fedora packages was too big). Keeping this open for now, unit  anew alpha version is out to test it out.

This comment is mass posted in all relevant bugzillas. If you already worked upstream to fix the problem, please let them know about the revert.

https://github.com/python/cpython/commit/0e2ac21dd
Comment 6 Miro Hrončok 2020-11-18 19:50:10 UTC 
I've patched Python 3.10 in our testing copr to include the revert. This package built fine. Sorry for the noise.
Note You need to log in before you can comment on or make changes to this bug.