Bug 2345516

Summary: python-trio fails to build with Python 3.14: test_ki_protection_doesnt_leave_cyclic_garbage: Left contains one more item: <coroutine object test_ki_protection_doesnt_leave_cyclic_garbage at 0x7fbaa2f162c0>
Product: [Fedora] Fedora Reporter: Karolina Surma <ksurma>
Component: python-trioAssignee: Ben Beasley <code>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: rawhideCC: carl, code, ksurma, mhroncok, python-packagers-sig
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: python-trio-0.30.0^1.256522a-1.fc43 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2025-06-19 11:20:05 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2322407    

Description Karolina Surma 2025-02-13 11:07:18 UTC
python-trio fails to build with Python 3.14.05.

________________ test_ki_protection_doesnt_leave_cyclic_garbage ________________

    @pytest.mark.skipif(
        sys.implementation.name != "cpython",
        reason="Only makes sense with refcounting GC",
    )
    async def test_ki_protection_doesnt_leave_cyclic_garbage() -> None:
        class MyException(Exception):
            pass
    
        async def demo() -> None:
            async def handle_error() -> None:
                try:
                    raise MyException
                except MyException as e:
                    exceptions.append(e)
    
            exceptions: list[MyException] = []
            try:
                async with _core.open_nursery() as n:
                    n.start_soon(handle_error)
                raise ExceptionGroup("errors", exceptions)
            finally:
                exceptions = []
    
        exc: Exception | None = None
        try:
            await demo()
        except ExceptionGroup as excs:
            exc = excs.exceptions[0]
    
        assert isinstance(exc, MyException)
>       assert gc.get_referrers(exc) == no_other_refs()
E       assert [<coroutine o...7fbaa2f162c0>] == []
E         
E         Left contains one more item: <coroutine object test_ki_protection_doesnt_leave_cyclic_garbage at 0x7fbaa2f162c0>
E         
E         Full diff:
E         - []
E         + [
E         +     <coroutine object test_ki_protection_doesnt_leave_cyclic_garbage at 0x7fbaa2f162c0>,
E         + ]

https://docs.python.org/3.14/whatsnew/3.14.html

For the build logs, see:
https://copr-be.cloud.fedoraproject.org/results/@python/python3.14/fedora-rawhide-x86_64/08651119-python-trio/

For all our attempts to build python-trio with Python 3.14, see:
https://copr.fedorainfracloud.org/coprs/g/python/python3.14/package/python-trio/

Testing and mass rebuild of packages is happening in copr.
You can follow these instructions to test locally in mock if your package builds with Python 3.14:
https://copr.fedorainfracloud.org/coprs/g/python/python3.14/

Let us know here if you have any questions.

Python 3.14 is planned to be included in Fedora 43.
To make that update smoother, we're building Fedora packages with all pre-releases of Python 3.14.
A build failure prevents us from testing all dependent packages (transitive [Build]Requires),
so if this package is required a lot, it's important for us to get it fixed soon.

We'd appreciate help from the people who know this package best,
but if you don't want to work on this now, let us know so we can try to work around it on our side.

Comment 1 Ben Beasley 2025-02-13 16:54:00 UTC
Reproduced in a virtualenv and reported upstream: https://github.com/python-trio/trio/issues/3209

Comment 2 Ben Beasley 2025-02-13 16:56:07 UTC
This appears to be caused by https://github.com/python/cpython/issues/125603.

Comment 3 Ben Beasley 2025-02-13 16:57:35 UTC
A proposed fix exists in https://github.com/python/cpython/pull/125640, so this may be solved upstream in CPython before 3.14.0 final.

Comment 4 Ben Beasley 2025-02-13 16:59:08 UTC
Assigning this to myself since I’m now keeping an eye on the CPython issue.

Comment 5 Ben Beasley 2025-02-13 19:15:13 UTC
Upon reflection, I think it would make sense to skip this test on Python 3.14 for the time being, since it seems clear that the issue is in CPython and that it is breaking the *test*, not the *implementation*.

https://src.fedoraproject.org/rpms/python-trio/pull-request/10

Comment 6 Karolina Surma 2025-05-21 12:39:06 UTC
And with beta1 we're having even more fun in trio: 

_core/_tests/test_run.py::test_error_in_run_loop Fatal Python error: Segmentation fault

Comment 7 Ben Beasley 2025-06-18 13:02:05 UTC
Hmm, this did get rebuilt for Python 3.14 somehow, https://koji.fedoraproject.org/koji/buildinfo?buildID=2724387, but it certainly doesn’t build now. I see the same error Karolina reported in https://bugzilla.redhat.com/show_bug.cgi?id=2345516#c6:

_core/_tests/test_run.py::test_error_in_run_loop Fatal Python error: Segmentation fault

Thread 0x00007fdbec9f66c0 [Trio thread 0] (most recent call first):
  File "/builddir/build/BUILD/python-trio-0.29.0-build/BUILDROOT/usr/lib/python3.14/site-packages/trio/_core/_thread_cache.py", line 194 in _work
  File "/usr/lib64/python3.14/threading.py", line 1023 in run
  File "/usr/lib64/python3.14/threading.py", line 1081 in _bootstrap_inner
  File "/usr/lib64/python3.14/threading.py", line 1043 in _bootstrap

Current thread 0x00007fdbf04a0bc0 [pytest] (most recent call first):
  Garbage-collecting
  File "/builddir/build/BUILD/python-trio-0.29.0-build/BUILDROOT/usr/lib/python3.14/site-packages/trio/_core/_tests/tutil.py", line 59 in gc_collect_harder
  File "/builddir/build/BUILD/python-trio-0.29.0-build/BUILDROOT/usr/lib/python3.14/site-packages/trio/_core/_tests/tutil.py", line 75 in ignore_coroutine_never_awaited_warnings
  File "/usr/lib64/python3.14/contextlib.py", line 148 in __exit__
  File "/builddir/build/BUILD/python-trio-0.29.0-build/BUILDROOT/usr/lib/python3.14/site-packages/trio/_core/_tests/test_run.py", line 932 in test_error_in_run_loop
  File "/usr/lib64/python3.14/contextlib.py", line 85 in inner
  File "/usr/lib/python3.14/site-packages/_pytest/python.py", line 159 in pytest_pyfunc_call
  File "/usr/lib/python3.14/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/usr/lib/python3.14/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/usr/lib/python3.14/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/usr/lib/python3.14/site-packages/_pytest/python.py", line 1627 in runtest
  File "/usr/lib/python3.14/site-packages/_pytest/runner.py", line 174 in pytest_runtest_call
  File "/usr/lib/python3.14/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/usr/lib/python3.14/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/usr/lib/python3.14/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/usr/lib/python3.14/site-packages/_pytest/runner.py", line 242 in <lambda>
  File "/usr/lib/python3.14/site-packages/_pytest/runner.py", line 341 in from_call
  File "/usr/lib/python3.14/site-packages/_pytest/runner.py", line 241 in call_and_report
  File "/usr/lib/python3.14/site-packages/_pytest/runner.py", line 132 in runtestprotocol
  File "/usr/lib/python3.14/site-packages/_pytest/runner.py", line 113 in pytest_runtest_protocol
  File "/usr/lib/python3.14/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/usr/lib/python3.14/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/usr/lib/python3.14/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/usr/lib/python3.14/site-packages/_pytest/main.py", line 362 in pytest_runtestloop
  File "/usr/lib/python3.14/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/usr/lib/python3.14/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/usr/lib/python3.14/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/usr/lib/python3.14/site-packages/_pytest/main.py", line 337 in _main
  File "/usr/lib/python3.14/site-packages/_pytest/main.py", line 283 in wrap_session
  File "/usr/lib/python3.14/site-packages/_pytest/main.py", line 330 in pytest_cmdline_main
  File "/usr/lib/python3.14/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/usr/lib/python3.14/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/usr/lib/python3.14/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/usr/lib/python3.14/site-packages/_pytest/config/__init__.py", line 175 in main
  File "/usr/lib/python3.14/site-packages/_pytest/config/__init__.py", line 201 in console_main
  File "/usr/bin/pytest", line 8 in <module>

Current thread's C stack trace (most recent call first):
  Binary file "/lib64/libpython3.14.so.1.0", at _Py_DumpStack+0x4c [0x7fdbeff1e7dc]
  Binary file "/lib64/libpython3.14.so.1.0", at +0x11f260 [0x7fdbeff1f260]
  Binary file "/lib64/libc.so.6", at +0x19bf0 [0x7fdbefc28bf0]
  Binary file "/lib64/libpython3.14.so.1.0", at +0x27c1a2 [0x7fdbf007c1a2]
  Binary file "/lib64/libpython3.14.so.1.0", at +0x18b3df [0x7fdbeff8b3df]
  Binary file "/lib64/libpython3.14.so.1.0", at +0x1c8ef3 [0x7fdbeffc8ef3]
  Binary file "/lib64/libpython3.14.so.1.0", at +0x1da05f [0x7fdbeffda05f]
  Binary file "/lib64/libpython3.14.so.1.0", at +0x20c0db [0x7fdbf000c0db]
  Binary file "/lib64/libpython3.14.so.1.0", at +0x1f31e9 [0x7fdbefff31e9]
  Binary file "/lib64/libpython3.14.so.1.0", at +0x2569c9 [0x7fdbf00569c9]
  Binary file "/lib64/libpython3.14.so.1.0", at +0x2c989c [0x7fdbf00c989c]
  Binary file "/lib64/libpython3.14.so.1.0", at _PyEval_EvalFrameDefault+0xbf8b [0x7fdbeff7811b]
  Binary file "/lib64/libpython3.14.so.1.0", at +0x1fdf9f [0x7fdbefffdf9f]
  Binary file "/lib64/libpython3.14.so.1.0", at _PyEval_EvalFrameDefault+0x167b [0x7fdbeff6d80b]
  Binary file "/lib64/libpython3.14.so.1.0", at +0x168b85 [0x7fdbeff68b85]
  Binary file "/lib64/libpython3.14.so.1.0", at +0x253fa0 [0x7fdbf0053fa0]
  Binary file "/lib64/libpython3.14.so.1.0", at +0x2aee76 [0x7fdbf00aee76]
  Binary file "/lib64/libpython3.14.so.1.0", at _PyObject_MakeTpCall+0x36c [0x7fdbeff5447c]
  Binary file "/lib64/libpython3.14.so.1.0", at _PyEval_EvalFrameDefault+0x60ea [0x7fdbeff7227a]
  Binary file "/lib64/libpython3.14.so.1.0", at +0x168b85 [0x7fdbeff68b85]
  Binary file "/lib64/libpython3.14.so.1.0", at +0x253fa0 [0x7fdbf0053fa0]
  Binary file "/lib64/libpython3.14.so.1.0", at +0x2aee76 [0x7fdbf00aee76]
  Binary file "/lib64/libpython3.14.so.1.0", at +0x267d3a [0x7fdbf0067d3a]
  Binary file "/lib64/libpython3.14.so.1.0", at _PyEval_EvalFrameDefault+0x5d95 [0x7fdbeff71f25]
  Binary file "/lib64/libpython3.14.so.1.0", at +0x168b85 [0x7fdbeff68b85]
  Binary file "/lib64/libpython3.14.so.1.0", at +0x253fa0 [0x7fdbf0053fa0]
  Binary file "/lib64/libpython3.14.so.1.0", at +0x2aee76 [0x7fdbf00aee76]
  Binary file "/lib64/libpython3.14.so.1.0", at _PyObject_MakeTpCall+0x36c [0x7fdbeff5447c]
  Binary file "/lib64/libpython3.14.so.1.0", at _PyEval_EvalFrameDefault+0x60ea [0x7fdbeff7227a]
  Binary file "/lib64/libpython3.14.so.1.0", at +0x168b85 [0x7fdbeff68b85]
  Binary file "/lib64/libpython3.14.so.1.0", at +0x253fa0 [0x7fdbf0053fa0]
  Binary file "/lib64/libpython3.14.so.1.0", at +0x2aee76 [0x7fdbf00aee76]
  <truncated rest of calls>

I don’t see any progress in CPython on https://github.com/python/cpython/pull/125640 and https://github.com/python/cpython/issues/125603, but it looks like trio upstream is tracking these issues in https://github.com/python-trio/trio/issues/3267. I think we may be able to handle this adequately by updating to a snapshot for now.

Comment 8 Miro Hrončok 2025-06-18 13:08:55 UTC
The sagfault is flaky. I was getting it only sometimes. That's why it was built successfully (I kept building it until it did).

Comment 9 Miro Hrončok 2025-06-18 13:09:30 UTC
See my comment at https://src.fedoraproject.org/rpms/python-trio/pull-request/12#comment-265382

Comment 10 Ben Beasley 2025-06-18 16:22:39 UTC
I haven’t seen the segfault in several builds of a current snapshot, so I proposed https://src.fedoraproject.org/rpms/python-trio/pull-request/13.

Comment 11 Fedora Update System 2025-06-19 11:15:44 UTC
FEDORA-2025-49cf75ff84 (python-trio-0.30.0^1.256522a-1.fc43) has been submitted as an update to Fedora 43.
https://bodhi.fedoraproject.org/updates/FEDORA-2025-49cf75ff84

Comment 12 Fedora Update System 2025-06-19 11:20:05 UTC
FEDORA-2025-49cf75ff84 (python-trio-0.30.0^1.256522a-1.fc43) has been pushed to the Fedora 43 stable repository.
If problem still persists, please make note of it in this bug report.