Bug 2299346

Summary: mercurial is broken (and FTBFS) with Python 3.13.0b4
Product: [Fedora] Fedora Reporter: Ben Beasley <code>
Component: mercurialAssignee: Mads Kiilerich <mads>
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: unspecified    
Version: rawhideCC: fedoraproject.org, mads, ndbecker2, opohorel, pstodulk
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2024-07-29 21:37:46 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2244836    

Description Ben Beasley 2024-07-22 19:09:27 UTC
$ fedpkg co mercurial
$ cd mercurial
$ fedpkg mockbuild --enablerepo=local

make -C doc
make[1]: Entering directory '/builddir/build/BUILD/mercurial-6.8-build/mercurial-6.8/doc'
/usr/bin/python3 gendoc.py "hg-ssh.8" > hg-ssh.8.txt.tmp
Traceback (most recent call last):
  File "/builddir/build/BUILD/mercurial-6.8-build/mercurial-6.8/doc/gendoc.py", line 29, in <module>
    from mercurial import (
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "/builddir/build/BUILD/mercurial-6.8-build/mercurial-6.8/hgdemandimport/demandimportpy3.py", line 52, in exec_module
    super().exec_module(module)
  File "<frozen importlib.util>", line 261, in exec_module
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "/builddir/build/BUILD/mercurial-6.8-build/mercurial-6.8/hgdemandimport/demandimportpy3.py", line 52, in exec_module
    super().exec_module(module)
  File "<frozen importlib.util>", line 271, in exec_module
AttributeError: partially initialized module 'threading' from '/usr/lib64/python3.13/threading.py' has no attribute 'RLock' (most likely due to a circular import)

In the build, this looks like a documentation-building problem, but it is much worse, as any basic use of mercurial crashes at runtime:

$ mock -r fedora-rawhide-x86 --enablerepo=local --clean
$ mock -r fedora-rawhide-x86_64 --enablerepo=local -i hg
$ mock -r fedora-rawhide-x86_64 --chroot 'hg st'

Traceback (most recent call last):
  File "/usr/bin/hg", line 57, in <module>
    from mercurial import dispatch
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "/usr/lib64/python3.13/site-packages/hgdemandimport/demandimportpy3.py", line 52, in exec_module
    super().exec_module(module)
  File "<frozen importlib.util>", line 261, in exec_module
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "/usr/lib64/python3.13/site-packages/hgdemandimport/demandimportpy3.py", line 52, in exec_module
    super().exec_module(module)
  File "<frozen importlib.util>", line 271, in exec_module
AttributeError: partially initialized module 'threading' from '/usr/lib64/python3.13/threading.py' has no attribute 'RLock' (most likely due to a circular import)

Reproducible: Always

Comment 1 Ben Beasley 2024-07-22 19:33:14 UTC
Based on Debian’s experiences backporting unreleased 3.12 fixes[1], this problem can be expected to show up in a future Python 3.12.5 release as well. (Thanks to Elliott Sales de Andrade for noticing that.)

Recent work on bug fixes for subinterpreters in the threading module[2][3] looks like it is a likely source for the new behavior.

[1] https://linux.debian.bugs.dist.narkive.com/dEAXq1Ed/bug-1076493-python3-12-latest-update-breaks-mercurial
[2] https://github.com/python/cpython/commits/3.13/Lib/threading.py
[3] https://github.com/python/cpython/commits/3.12/Lib/threading.py

Comment 2 Mads Kiilerich 2024-07-22 19:42:14 UTC
There are more details on https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1076449 .

Upstream Mercurial is investigating different solutions/workarounds.

The simple quick urgent workaround is to set the environment variable HGDEMANDIMPORT=disable .

A more likely final solution is to add '_weakrefset' and 'threading' to /usr/lib64/python3.12/site-packages/hgdemandimport/__init__.py IGNORES .

Comment 3 Mads Kiilerich 2024-07-26 17:20:12 UTC
Should be fixed with https://koji.fedoraproject.org/koji/buildinfo?buildID=2517518

Comment 4 Ben Beasley 2024-07-29 21:37:46 UTC
I confirm both that the HGDEMANDIMPORT=disable workaround was successful before a fix was available, and that https://koji.fedoraproject.org/koji/buildinfo?buildID=2517518 fixed the problem. Thanks!