Bugzilla (bugzilla.redhat.com) will be under maintenance for infrastructure upgrades and will not be unavailable on July 31st between 12:30 AM - 05:30 AM UTC. We appreciate your understanding and patience. You can follow status.redhat.com for details.
Bug 1808355 - python-greenlet fails to build with Python 3.9: error: ‘_Py_DEC_REFTOTAL’ undeclared
Summary: python-greenlet fails to build with Python 3.9: error: ‘_Py_DEC_REFTOTAL’ und...
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: python-greenlet
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
Assignee: Kevin Fenzi
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: PYTHON39
TreeView+ depends on / blocked
 
Reported: 2020-02-28 10:04 UTC by Miro Hrončok
Modified: 2020-05-25 13:01 UTC (History)
10 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-05-25 13:01:18 UTC
Type: Bug


Attachments (Terms of Use)

Description Miro Hrončok 2020-02-28 10:04:44 UTC
python-greenlet fails to build with Python 3.9.0a4.

gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fPIC -I/usr/include/python3.9 -c greenlet.c -o build/temp.linux-x86_64-3.9/greenlet.o
greenlet.c: In function ‘g_initialstub’:
greenlet.c:823:4: warning: ‘PyEval_CallObjectWithKeywords’ is deprecated [-Wdeprecated-declarations]
  823 |    result = PyEval_CallObjectWithKeywords(
      |    ^~~~~~
In file included from /usr/include/python3.9/Python.h:141,
                 from greenlet.h:8,
                 from greenlet.c:5:
/usr/include/python3.9/ceval.h:17:43: note: declared here
   17 | Py_DEPRECATED(3.9) PyAPI_FUNC(PyObject *) PyEval_CallObjectWithKeywords(
      |                                           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
greenlet.c: In function ‘green_dealloc’:
greenlet.c:1046:4: error: ‘_Py_DEC_REFTOTAL’ undeclared (first use in this function); did you mean ‘_Py_DECREF’?
 1046 |    _Py_DEC_REFTOTAL;
      |    ^~~~~~~~~~~~~~~~
      |    _Py_DECREF
greenlet.c:1046:4: note: each undeclared identifier is reported only once for each function it appears in
error: command 'gcc' failed with exit status 1

For the build logs, see:
https://copr-be.cloud.fedoraproject.org/results/@python/python3.9/fedora-rawhide-x86_64/01256259-python-greenlet/

For all our attempts to build python-greenlet with Python 3.9, see:
https://copr.fedorainfracloud.org/coprs/g/python/python3.9/package/python-greenlet/

Testing and mass rebuild of packages is happening in copr. You can follow these instructions to test locally in mock if your package builds with Python 3.9:
https://copr.fedorainfracloud.org/coprs/g/python/python3.9/

Let us know here if you have any questions.

Python 3.9 will be included in Fedora 33. To make that update smoother, we're building Fedora packages with early pre-releases of Python 3.9.
A build failure prevents us from testing all dependent packages (transitive [Build]Requires), so if this package is required a lot, it's important for us to get it fixed soon.
We'd appreciate help from the people who know this package best, but if you don't want to work on this now, let us know so we can try to work around it on our side.

Comment 1 Miro Hrončok 2020-03-16 18:30:23 UTC
Setting the severity to high.

This package is part of the initial bootstrap sequence. Without it, we cannot proceed with the bootstrap in a Koji side tag.

https://fedoraproject.org/wiki/Changes/Python3.9#Important_dates_and_plan

The current plan is to follow the "ideal point when we can start rebuilding in Koji" -- that is we need to get this bug fixed approximately in 2 months.

That includes potential uncovered bugs in packages that depend on this one.

Please knowledge that you have read this message and that you can dedicate time to fix it. If you know already that you won't be able to fix it by the deadline, please let us know ASAP, so we can allocate resources to do that. Thank You.

(This comment is posted to multiple Bugzillas, please forgive me if it's not 100% accurate.)

Comment 2 Kevin Fenzi 2020-03-16 21:56:29 UTC
Looks like this one succeded?

https://copr-be.cloud.fedoraproject.org/results/%40python/python3.9/fedora-rawhide-x86_64/01210367-python-greenlet/

That was from january tho. Is that the right place to look?

Comment 3 Miro Hrončok 2020-03-16 22:04:39 UTC
https://copr.fedorainfracloud.org/coprs/g/python/python3.9/package/python-greenlet/ is indeed a right place to look.

However, the successful build from January is with 3.9.0a3. The failure started with 3.9.0a4.

Note that _underscored names are not part of the API. _Py_DEC_REFTOTAL was removed in https://github.com/python/cpython/commit/49932fec62c616ec88da52642339d83ae719e924

commit 49932fec62c616ec88da52642339d83ae719e924
Author: Victor Stinner <vstinner@python.org>
Date:   Mon Feb 3 17:55:05 2020 +0100

    bpo-39542: Simplify _Py_NewReference() (GH-18332)
    
    * Remove _Py_INC_REFTOTAL and _Py_DEC_REFTOTAL macros: modify
      directly _Py_RefTotal.
    * _Py_ForgetReference() is no longer defined if the Py_TRACE_REFS
      macro is not defined.
    * Remove _Py_NewReference() implementation from object.c:
      unify the two implementations in object.h inline function.
    * Fix Py_TRACE_REFS build: _Py_INC_TPALLOCS() macro has been removed.

 Include/object.h          | 42 +++++++++++++++---------------------------
 Modules/_testcapimodule.c |  8 +++++---
 Objects/bytesobject.c     |  6 +++++-
 Objects/dictobject.c      | 26 ++++++++++++++++++--------
 Objects/object.c          | 21 +++++----------------
 Objects/tupleobject.c     |  9 +++++++--
 Objects/unicodeobject.c   |  6 +++++-
 7 files changed, 60 insertions(+), 58 deletions(-)

Comment 4 Victor Stinner 2020-03-18 14:19:04 UTC
I proposed a fix upstream:  https://github.com/python-greenlet/greenlet/pull/161

> Note that _underscored names are not part of the API. _Py_DEC_REFTOTAL was removed in https://github.com/python/cpython/commit/49932fec62c616ec88da52642339d83ae719e924

I'm always surprised when I discover that developers rely on private APIs... But greenlet is very low-level, and so has to "hack" CPython internals ;-)

Comment 5 Kevin Fenzi 2020-03-19 23:25:28 UTC
So, shall we just carry Victor's patch for now? :) 

I'm not sure how active upstream really is...

Comment 6 Miro Hrončok 2020-03-19 23:49:20 UTC
We can give it couple weeks.

The next mass rebuild in our copr will happen after the 3.9.0a5 release, that should happen early next week. https://www.python.org/dev/peps/pep-0596/#schedule

I'll make sure we patch greenlet manually during that.

Comment 7 Victor Stinner 2020-03-20 08:42:03 UTC
> I'm not sure how active upstream really is...

The project is still active: I see multiple changes many one month ago.
https://github.com/python-greenlet/greenlet/commits/master

That's way more active than many projects ;-)


Note You need to log in before you can comment on or make changes to this bug.