Bug 2069158 - python-giacpy fails to build with Python 3.11: error: cannot convert PyObject {aka _object} to PyTracebackObject {aka _traceback} in assignment
Summary: python-giacpy fails to build with Python 3.11: error: cannot convert PyObject...
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: python-giacpy
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Antonio T. sagitter
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: PYTHON3.11
TreeView+ depends on / blocked
 
Reported: 2022-03-28 11:56 UTC by Tomáš Hrnčiar
Modified: 2022-06-22 11:49 UTC (History)
4 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2022-06-22 11:49:14 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Tomáš Hrnčiar 2022-03-28 11:56:53 UTC
python-giacpy fails to build with Python 3.11.0a6.

giacpy/giacpy.cpp: In function ‘int __Pyx_PyBytes_Equals(PyObject*, PyObject*, int)’:
giacpy/giacpy.cpp:160921:43: warning: ‘PyBytesObject::ob_shash’ is deprecated [-Wdeprecated-declarations]
160921 |             hash1 = ((PyBytesObject*)s1)->ob_shash;
       |                                           ^~~~~~~~
In file included from /usr/include/python3.11/bytesobject.h:62,
                 from /usr/include/python3.11/Python.h:50,
                 from giacpy/giacpy.cpp:30:
/usr/include/python3.11/cpython/bytesobject.h:7:35: note: declared here
    7 |     Py_DEPRECATED(3.11) Py_hash_t ob_shash;
      |                                   ^~~~~~~~
giacpy/giacpy.cpp:160921:43: warning: ‘PyBytesObject::ob_shash’ is deprecated [-Wdeprecated-declarations]
160921 |             hash1 = ((PyBytesObject*)s1)->ob_shash;
       |                                           ^~~~~~~~
/usr/include/python3.11/cpython/bytesobject.h:7:35: note: declared here
    7 |     Py_DEPRECATED(3.11) Py_hash_t ob_shash;
      |                                   ^~~~~~~~
giacpy/giacpy.cpp:160921:43: warning: ‘PyBytesObject::ob_shash’ is deprecated [-Wdeprecated-declarations]
160921 |             hash1 = ((PyBytesObject*)s1)->ob_shash;
       |                                           ^~~~~~~~
/usr/include/python3.11/cpython/bytesobject.h:7:35: note: declared here
    7 |     Py_DEPRECATED(3.11) Py_hash_t ob_shash;
      |                                   ^~~~~~~~
giacpy/giacpy.cpp:160922:43: warning: ‘PyBytesObject::ob_shash’ is deprecated [-Wdeprecated-declarations]
160922 |             hash2 = ((PyBytesObject*)s2)->ob_shash;
       |                                           ^~~~~~~~
/usr/include/python3.11/cpython/bytesobject.h:7:35: note: declared here
    7 |     Py_DEPRECATED(3.11) Py_hash_t ob_shash;
      |                                   ^~~~~~~~
giacpy/giacpy.cpp:160922:43: warning: ‘PyBytesObject::ob_shash’ is deprecated [-Wdeprecated-declarations]
160922 |             hash2 = ((PyBytesObject*)s2)->ob_shash;
       |                                           ^~~~~~~~
/usr/include/python3.11/cpython/bytesobject.h:7:35: note: declared here
    7 |     Py_DEPRECATED(3.11) Py_hash_t ob_shash;
      |                                   ^~~~~~~~
giacpy/giacpy.cpp:160922:43: warning: ‘PyBytesObject::ob_shash’ is deprecated [-Wdeprecated-declarations]
160922 |             hash2 = ((PyBytesObject*)s2)->ob_shash;
       |                                           ^~~~~~~~
/usr/include/python3.11/cpython/bytesobject.h:7:35: note: declared here
    7 |     Py_DEPRECATED(3.11) Py_hash_t ob_shash;
      |                                   ^~~~~~~~
giacpy/giacpy.cpp: In function ‘PyObject* __Pyx_Coroutine_SendEx(__pyx_CoroutineObject*, PyObject*, int)’:
giacpy/giacpy.cpp:164276:42: error: cannot convert ‘PyObject*’ {aka ‘_object*’} to ‘PyTracebackObject*’ {aka ‘_traceback*’} in assignment
164276 |         tb = __Pyx__ExceptionGetTraceback(exc_state->exc_value);
       |              ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
       |                                          |
       |                                          PyObject* {aka _object*}
error: command '/usr/bin/gcc' failed with exit code 1
error: Bad exit status from /var/tmp/rpm-tmp.triq6H (%build)


https://docs.python.org/3.11/whatsnew/3.11.html

For the build logs, see:
https://copr-be.cloud.fedoraproject.org/results/@python/python3.11/fedora-rawhide-x86_64/03845061-python-giacpy/

For all our attempts to build python-giacpy with Python 3.11, see:
https://copr.fedorainfracloud.org/coprs/g/python/python3.11/package/python-giacpy/

Testing and mass rebuild of packages is happening in copr. You can follow these instructions to test locally in mock if your package builds with Python 3.11:
https://copr.fedorainfracloud.org/coprs/g/python/python3.11/

Let us know here if you have any questions.

Python 3.11 is planned to be included in Fedora 37. To make that update smoother, we're building Fedora packages with all pre-releases of Python 3.11.
A build failure prevents us from testing all dependent packages (transitive [Build]Requires), so if this package is required a lot, it's important for us to get it fixed soon.
We'd appreciate help from the people who know this package best, but if you don't want to work on this now, let us know so we can try to work around it on our side.

Comment 1 Antonio T. sagitter 2022-03-28 19:52:48 UTC
I need help of author.

Comment 2 Frederic Han 2022-03-29 09:40:19 UTC
this code is generated by cython. With python3.9 don't see it in giacpy.cpp.

it seems related to this part of cython:

https://github.com/cython/cython/blob/master/Cython/Utility/Coroutine.c

It is not clear to me which part of giacpy.pyx makes cython use this.

Do you have success to build sagemath with python3.11?

Comment 3 Tomáš Hrnčiar 2022-03-29 10:17:13 UTC
Hello Frederic,

sagemath wasn't built with python3.11 so far, because there is a bunch of missing dependencies. I've tried to disable them by flipping bconds and triggered the build. The results will be available here: https://copr.fedorainfracloud.org/coprs/build/3910840 . 


(In reply to Frederic Han from comment #2)
> this code is generated by cython. With python3.9 don't see it in giacpy.cpp.
> 
> it seems related to this part of cython:
> 
> https://github.com/cython/cython/blob/master/Cython/Utility/Coroutine.c

Beware, in Fedora we are using the stable branch of Cython (0.29.x) and it might differ from the master branch.

https://github.com/cython/cython/blob/0.29.x/Cython/Utility/Coroutine.c

It's also possible that the bug is in Cython. Coroutine.c file was already modified to accommodate Python 3.11 so maybe it will need some further changes.

Comment 4 Frederic Han 2022-03-29 11:56:46 UTC
I think it may be already solved in cython's master branch.

I have done the following test:
in the broken file giacpy.cpp I have replaced a couple of lines in Pyx_Coroutine_SendEx with the code from coroutine.c in master branch of cython and it worked.

{{{
diff --git a/giacpy-auto-p311.cpp b/giacpy.cpp
index ce79a80..21a4f79 100644
--- a/giacpy-auto-p311.cpp
+++ b/giacpy.cpp
@@ -164180,21 +164180,29 @@ PyObject *__Pyx_Coroutine_SendEx(__pyx_CoroutineObject *self, PyObject *value, i
     if (exc_state->exc_value) {
         #if CYTHON_COMPILING_IN_PYPY || CYTHON_COMPILING_IN_PYSTON
         #else
-        PyTracebackObject *tb;
-        #if PY_VERSION_HEX >= 0x030B00A4
-        tb = __Pyx__ExceptionGetTraceback(exc_state->exc_value);
+        PyObject *exc_tb;
+        #if PY_VERSION_HEX >= 0x030B00a4
+        // owned reference!
+        exc_tb = PyException_GetTraceback(exc_state->exc_value);
         #else
-        tb = exc_state->exc_traceback;
+        exc_tb = exc_state->exc_traceback;
         #endif
-        if (tb) {
+        if (exc_tb) {
+            PyTracebackObject *tb = (PyTracebackObject *) exc_tb;
             PyFrameObject *f = tb->tb_frame;
+
             assert(f->f_back == NULL);
             #if PY_VERSION_HEX >= 0x030B00A1
+            // PyThreadState_GetFrame returns NULL if there isn't a current frame
+            // which is a valid state so no need to check
             f->f_back = PyThreadState_GetFrame(tstate);
             #else
             Py_XINCREF(tstate->frame);
             f->f_back = tstate->frame;
             #endif
+            #if PY_VERSION_HEX >= 0x030B00a4
+            Py_DECREF(exc_tb);
+            #endif
         }
         #endif
}}}

so I think that a cython upgrade  shoud solve this.
Best
Frederic

Comment 5 Frederic Han 2022-03-29 13:11:20 UTC
So I can confirm that the changes we need appear in this commit of Coroutine.c

Adapt to "exc_info" changes in CPython 3.11a4:
https://github.com/cython/cython/commit/776957022d062ed24edea192b719720118ee3576

I was able to modify Coroutine.c and  cythonize giacpy  in mock with python3.11  and build giacpy.

More precisely we need at least the folling changes:

{{{
diff Coroutine.c.orig Coroutine.c
724,726c724,727
<         PyTracebackObject *tb;
<         #if PY_VERSION_HEX >= 0x030B00A4
<         tb = __Pyx__ExceptionGetTraceback(exc_state->exc_value);
---
>         PyObject *exc_tb;
>         #if PY_VERSION_HEX >= 0x030B00a4
>         // owned reference!
>         exc_tb = PyException_GetTraceback(exc_state->exc_value);
728c729
<         tb = exc_state->exc_traceback;
---
>         exc_tb = exc_state->exc_traceback;
730c731,732
<         if (tb) {
---
>         if (exc_tb) {
>             PyTracebackObject *tb = (PyTracebackObject *) exc_tb;
741a744,746
>             #if PY_VERSION_HEX >= 0x030B00a4
>             Py_DECREF(exc_tb);
>             #endif
745c750
}}}

Comment 6 Miro Hrončok 2022-06-22 11:49:14 UTC
Closing this in bulk as it built with Python 3.11. If this needs to remain open for a followup, feel free to reopen, I won't close in bulk again.


Note You need to log in before you can comment on or make changes to this bug.