Bug 2022252 - mercurial fails to build with Python 3.11: AttributeError: module 'inspect' has no attribute 'getargspec'
Summary: mercurial fails to build with Python 3.11: AttributeError: module 'inspect' h...
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: mercurial
Version: 36
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Mads Kiilerich
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: PYTHON3.11
TreeView+ depends on / blocked
 
Reported: 2021-11-11 07:58 UTC by Tomáš Hrnčiar
Modified: 2022-02-18 17:53 UTC (History)
10 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-02-18 17:53:55 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Tomáš Hrnčiar 2021-11-11 07:58:01 UTC
mercurial fails to build with Python 3.11.0a2.

mercurial/cext/pathencode.c: In function 'encodedir':
mercurial/cext/pathencode.c:179:32: error: lvalue required as decrement operand
  179 |                 Py_SIZE(newobj)--;
      |                                ^~
mercurial/cext/pathencode.c: In function 'pathencode':
mercurial/cext/pathencode.c:794:40: error: lvalue required as decrement operand
  794 |                         Py_SIZE(newobj)--;
      |                                        ^~

Since Py_SIZE() is changed to a inline static function, Py_SIZE(obj) = new_size must be replaced with Py_SET_SIZE(obj, new_size): see the Py_SET_SIZE() function (available since Python 3.9). For backward compatibility, this macro can be used:

#if PY_VERSION_HEX < 0x030900A4 && !defined(Py_SET_SIZE)
static inline void _Py_SET_SIZE(PyVarObject *ob, Py_ssize_t size)
{ ob->ob_size = size; }
#define Py_SET_SIZE(ob, size) _Py_SET_SIZE((PyVarObject*)(ob), size)
#endif

(Contributed by Victor Stinner in bpo-39573.)

https://bugs.python.org/issue39573
https://docs.python.org/3.11/whatsnew/3.11.html

For the build logs, see:
https://copr-be.cloud.fedoraproject.org/results/@python/python3.11/fedora-rawhide-x86_64/02939344-mercurial/

For all our attempts to build mercurial with Python 3.11, see:
https://copr.fedorainfracloud.org/coprs/g/python/python3.11/package/mercurial/

Testing and mass rebuild of packages is happening in copr. You can follow these instructions to test locally in mock if your package builds with Python 3.11:
https://copr.fedorainfracloud.org/coprs/g/python/python3.11/

Let us know here if you have any questions.

Python 3.11 is planned to be included in Fedora 37. To make that update smoother, we're building Fedora packages with all pre-releases of Python 3.11.
A build failure prevents us from testing all dependent packages (transitive [Build]Requires), so if this package is required a lot, it's important for us to get it fixed soon.
We'd appreciate help from the people who know this package best, but if you don't want to work on this now, let us know so we can try to work around it on our side.

Comment 1 Mads Kiilerich 2021-11-22 20:43:29 UTC
Fixed upstream on the stable branch, probably to be released in 6.0 - https://www.mercurial-scm.org/repo/hg/rev/e35807332598

Comment 2 Tomáš Hrnčiar 2022-02-04 13:05:40 UTC
Thank you for the fix. Seems like there is another issue with mercurial and Python 3.11.

AttributeError: module 'inspect' has no attribute 'getargspec'

You can see full log here:
https://download.copr.fedorainfracloud.org/results/@python/python3.11/fedora-rawhide-x86_64/03295004-mercurial/builder-live.log.gz

Comment 3 Mads Kiilerich 2022-02-05 13:28:27 UTC
(In reply to Tomáš Hrnčiar from comment #2)
> AttributeError: module 'inspect' has no attribute 'getargspec'

I a fix has been posted upstream - https://www.mercurial-scm.org/pipermail/mercurial-devel/2022-February/147743.html .


Note that building the Mercurial package with the 3.11 included in f35 still will fail due to an incorrect %python3_sitearch, caused by:

$ python3.11 -Ic "import sysconfig; print(sysconfig.get_path('platlib'))"
/usr/local/lib64/python3.11/site-packages

vs

$ python3.10 -Ic "import sysconfig; print(sysconfig.get_path('platlib'))"
/usr/lib64/python3.10/site-packages

Comment 4 Miro Hrončok 2022-02-05 18:39:43 UTC
%python3_sitearch is correct, but it's evaluated as:

$ RPM_BUILD_ROOT= python3.11 -Ic "import sysconfig; print(sysconfig.get_path('platlib'))"

Comment 5 Mads Kiilerich 2022-02-06 14:23:50 UTC
Ok, I see it is complicated. 00251-change-user-install-location.patch and the RPM_BUILD_ROOT usage in sysconfig is surprising to me. I would expect Python to default to the user default, and the RPM build environment more explicitly tweaking Python when building system packages. Dependency on "magic" environment variables makes packaging harder when we no longer can expect things to behave the same way when testing locally.

But evidently, this dependency on RPM_BUILD_ROOT changed somehow. It matters for platlib on 3.11, but not on 3.10 .

I just know that with "%define __python3 /usr/bin/python3.11" in my spec and running "fedpkg local" on f35, the build fails because %python3_sitearch is /usr/local/lib64/python3.11/site-packages. With 3.10 it is /usr/lib64/python3.10/site-packages and the package builds as usual.

AFAICS, the problem is that RPM_BUILD_ROOT is set too late. It isn't set when the spec is preprocessed and /usr/lib/rpm/macros.d/macros.python3 invokes python to set %python3_sitearch.

Comment 6 Miro Hrončok 2022-02-06 21:48:04 UTC
> Ok, I see it is complicated. 

It is very complex and complicated. I might need to summarize this somewhere. The starting point is the deprecation of distutils and https://bugs.python.org/issue43976


> But evidently, this dependency on RPM_BUILD_ROOT changed somehow. It matters for platlib on 3.11, but not on 3.10 .

Yes, it matters on 3.10 on Fedora 36 as well, but we haven't dared to change the behavior on released Fedoras.



> I just know that with "%define __python3 /usr/bin/python3.11" in my spec and running "fedpkg local" on f35, the build fails because %python3_sitearch is /usr/local/lib64/python3.11/site-packages. With 3.10 it is /usr/lib64/python3.10/site-packages and the package builds as usual...

That can be fixed. In fact, I have not expected anybody to actually do "%define __python3 /usr/bin/python3.11" in Fedora 35. I can fix that by backporting https://src.fedoraproject.org/rpms/python-rpm-macros/pull-request/110

Comment 7 Miro Hrončok 2022-02-07 10:41:05 UTC
> That can be fixed. In fact, I have not expected anybody to actually do "%define __python3 /usr/bin/python3.11" in Fedora 35. I can fix that by backporting https://src.fedoraproject.org/rpms/python-rpm-macros/pull-request/110

https://src.fedoraproject.org/rpms/python-rpm-macros/pull-request/130

Comment 8 Mads Kiilerich 2022-02-07 14:35:52 UTC
(In reply to Miro Hrončok from comment #6)
> I can fix that by backporting
> https://src.fedoraproject.org/rpms/python-rpm-macros/pull-request/110

Personally, I'm now aware and can do without it. It might be beneficial for others, but also risky. In principle, it seems like a good idea that f35 has all the dependencies of its python3.11 package. Even better if the f35 3.11 did things the same way as 3.10, without backporting new functionality to f35 after release.

Comment 9 Miro Hrončok 2022-02-07 16:05:05 UTC
Nah, it's not risky. It changes nothing for f35's python3.10. It was even already backported to RHEL 9.

Comment 10 Ben Cotton 2022-02-08 20:08:52 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 36 development cycle.
Changing version to 36.

Comment 11 Mads Kiilerich 2022-02-18 12:34:34 UTC
(In reply to Tomáš Hrnčiar from comment #2)
> Thank you for the fix. Seems like there is another issue with mercurial and
> Python 3.11.
> 
> AttributeError: module 'inspect' has no attribute 'getargspec'

This has been fixed in Mercurial 6.0.3, which now is in f36 and rawhide.

Please, can you confirm it works in your 3.11 environment?

Comment 12 Miro Hrončok 2022-02-18 17:53:55 UTC
The package in copr is set to pick up all commits to rawhide:

https://copr.fedorainfracloud.org/coprs/g/python/python3.11/package/mercurial/

Lates automated build (6.0.3-1) was successful, thank you!


Note You need to log in before you can comment on or make changes to this bug.