Bug 2359697 - python-nanobind fails to build with Python 3.14: AssertionError related to the refcounting
Summary: python-nanobind fails to build with Python 3.14: AssertionError related to th...
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: python-nanobind
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Konrad Kleine
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks: PYTHON3.14
TreeView+ depends on / blocked
 
Reported: 2025-04-15 08:20 UTC by Karolina Surma
Modified: 2025-04-25 13:50 UTC (History)
5 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2025-04-25 13:50:36 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Fedora Package Sources python-nanobind pull-request 7 0 None None None 2025-04-23 15:16:28 UTC

Description Karolina Surma 2025-04-15 08:20:37 UTC
python-nanobind fails to build with Python 3.14.0a7.

______________________________ test50_call_policy ______________________________

arg1 = 'string', arg2 = 'xxx', expect_ret = '<unfinished>'

    def case(arg1, arg2, expect_ret):  # type: (str, str, str | None) -> str
        if hasattr(sys, "getrefcount"):
            refs_before = (sys.getrefcount(arg1), sys.getrefcount(arg2))
    
        ret = None
        try:
>           ret = t.test_call_policy(arg1, arg2)
E           TypeError: test_call_policy(): incompatible function arguments. The following argument types are supported:
E               1. test_call_policy(arg0: str, arg1: str, /) -> str
E           
E           Invoked with types: str, str

tests/test_functions.py:681: TypeError

During handling of the above exception, another exception occurred:

    def test50_call_policy():
        def case(arg1, arg2, expect_ret):  # type: (str, str, str | None) -> str
            if hasattr(sys, "getrefcount"):
                refs_before = (sys.getrefcount(arg1), sys.getrefcount(arg2))
    
            ret = None
            try:
                ret = t.test_call_policy(arg1, arg2)
                assert ret == expect_ret
                return ret
            finally:
                if expect_ret is None:
                    assert t.call_policy_record() == []
                else:
                    (((arg1r, arg2r), recorded_ret),) = t.call_policy_record()
                    assert recorded_ret == expect_ret
                    assert ret is None or ret is recorded_ret
                    assert recorded_ret is not expect_ret
    
                    if hasattr(sys, "getrefcount"):
                        # Make sure no reference leak occurred: should be
                        # one in getrefcount args, one or two in locals,
                        # zero or one in the pending-return-value slot.
                        # We have to decompose this to avoid getting confused
                        # by transient additional references added by pytest's
                        # assertion rewriting.
                        ret_refs = sys.getrefcount(recorded_ret)
                        assert ret_refs == 2 + 2 * (ret is not None)
    
                    for (passed, recorded) in ((arg1, arg1r), (arg2, arg2r)):
                        if passed == "swapfrom":
                            assert recorded == "swapto"
                            if hasattr(sys, "getrefcount"):
                                recorded_refs = sys.getrefcount(recorded)
                                # recorded, arg1r, unnamed tuple, getrefcount arg
                                assert recorded_refs == 4
                        else:
                            assert passed is recorded
    
                    del passed, recorded, arg1r, arg2r
                    if hasattr(sys, "getrefcount"):
                        refs_after = (sys.getrefcount(arg1), sys.getrefcount(arg2))
                        assert refs_before == refs_after
    
        # precall throws exception
        with pytest.raises(RuntimeError, match="expected only strings"):
            case(12345, "0", None)
    
        # conversion of args fails
        with pytest.raises(TypeError):
>           case("string", "xxx", "<unfinished>")

tests/test_functions.py:724: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

arg1 = 'string', arg2 = 'xxx', expect_ret = '<unfinished>'

    def case(arg1, arg2, expect_ret):  # type: (str, str, str | None) -> str
        if hasattr(sys, "getrefcount"):
            refs_before = (sys.getrefcount(arg1), sys.getrefcount(arg2))
    
        ret = None
        try:
            ret = t.test_call_policy(arg1, arg2)
            assert ret == expect_ret
            return ret
        finally:
            if expect_ret is None:
                assert t.call_policy_record() == []
            else:
                (((arg1r, arg2r), recorded_ret),) = t.call_policy_record()
                assert recorded_ret == expect_ret
                assert ret is None or ret is recorded_ret
                assert recorded_ret is not expect_ret
    
                if hasattr(sys, "getrefcount"):
                    # Make sure no reference leak occurred: should be
                    # one in getrefcount args, one or two in locals,
                    # zero or one in the pending-return-value slot.
                    # We have to decompose this to avoid getting confused
                    # by transient additional references added by pytest's
                    # assertion rewriting.
                    ret_refs = sys.getrefcount(recorded_ret)
                    assert ret_refs == 2 + 2 * (ret is not None)
    
                for (passed, recorded) in ((arg1, arg1r), (arg2, arg2r)):
                    if passed == "swapfrom":
                        assert recorded == "swapto"
                        if hasattr(sys, "getrefcount"):
                            recorded_refs = sys.getrefcount(recorded)
                            # recorded, arg1r, unnamed tuple, getrefcount arg
                            assert recorded_refs == 4
                    else:
                        assert passed is recorded
    
                del passed, recorded, arg1r, arg2r
                if hasattr(sys, "getrefcount"):
                    refs_after = (sys.getrefcount(arg1), sys.getrefcount(arg2))
>                   assert refs_before == refs_after
E                   assert (3221225472, 2) == (3221225472, 3)
E                     
E                     At index 1 diff: 2 != 3
E                     Use -v to get more diff

tests/test_functions.py:716: AssertionError
___________________ test10_shared_from_this_create_in_python ___________________

clean = None

    def test10_shared_from_this_create_in_python(clean):
>       check_shared_from_this_py_owned(t.ExampleST, t.ExampleST, 42)

tests/test_holders.py:303: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

ty = <class 'test_holders_ext.ExampleST'>
factory = <class 'test_holders_ext.ExampleST'>, value = 42

    def check_shared_from_this_py_owned(ty, factory, value):
        e = ty(value)
    
        # Creating from Python does not enable shared_from_this
        assert e.value == value
        assert not e.has_shared_from_this()
        assert t.owns_cpp(e)
    
        # Passing to C++ as a shared_ptr does
        w = t.SharedWrapperST(e)
        assert e.has_shared_from_this()
        assert w.ptr is e
    
        # Execute shared_from_this on the C++ side
        w2 = t.SharedWrapperST.from_existing(e)
        assert e.use_count() == 2
        assert w.value == w2.value == e.value == value
        assert t.same_owner(w, w2)
    
        # Returning a raw pointer from C++ locates the existing instance
        assert w2.get_own() is w2.get_ref() is e
        assert t.owns_cpp(e)
    
        if hasattr(sys, "getrefcount"):
            # One reference is held by the C++ shared_ptr, one by our
            # locals dict, and one by the arg to getrefcount
            rc = sys.getrefcount(e)
>           assert rc == 3
E           assert 2 == 3

tests/test_holders.py:282: AssertionError
__________________ test11_shared_from_this_create_raw_in_cpp ___________________

clean = None

    def test11_shared_from_this_create_raw_in_cpp(clean):
        # Creating a raw pointer from C++ does not enable shared_from_this;
        # although the object is held by pointer rather than value, the logical
        # ownership transfers to Python and the behavior is equivalent to test10.
        # Once we get a shared_ptr it owns a reference to the Python object.
>       check_shared_from_this_py_owned(t.ExampleST, t.ExampleST.make, 10)

tests/test_holders.py:322: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

ty = <class 'test_holders_ext.ExampleST'>
factory = <nanobind.nb_func object at 0x7f0c2e058480>, value = 10

    def check_shared_from_this_py_owned(ty, factory, value):
        e = ty(value)
    
        # Creating from Python does not enable shared_from_this
        assert e.value == value
        assert not e.has_shared_from_this()
        assert t.owns_cpp(e)
    
        # Passing to C++ as a shared_ptr does
        w = t.SharedWrapperST(e)
        assert e.has_shared_from_this()
        assert w.ptr is e
    
        # Execute shared_from_this on the C++ side
        w2 = t.SharedWrapperST.from_existing(e)
        assert e.use_count() == 2
        assert w.value == w2.value == e.value == value
        assert t.same_owner(w, w2)
    
        # Returning a raw pointer from C++ locates the existing instance
        assert w2.get_own() is w2.get_ref() is e
        assert t.owns_cpp(e)
    
        if hasattr(sys, "getrefcount"):
            # One reference is held by the C++ shared_ptr, one by our
            # locals dict, and one by the arg to getrefcount
            rc = sys.getrefcount(e)
>           assert rc == 3
E           assert 2 == 3

tests/test_holders.py:282: AssertionError
_________________ test12_shared_from_this_create_shared_in_cpp _________________

clean = None

    def test12_shared_from_this_create_shared_in_cpp(clean):
        # Creating a shared_ptr from C++ enables shared_from_this. Now the
        # shared_ptr does not keep the Python object alive; it's directly
        # owning the ExampleST object on the C++ side.
        e = t.ExampleST.make_shared(10)
        assert e.value == 10
        assert e.has_shared_from_this()
        assert e.shared_from_this() is e  # same instance
        assert e.use_count() == 1
        assert not t.owns_cpp(e)
        if hasattr(sys, "getrefcount"):
            # One reference is held by our locals dict and one by the
            # arg to getrefcount
            rc = sys.getrefcount(e)
>           assert rc == 2
E           assert 1 == 2

tests/test_holders.py:343: AssertionError
________________ test13_shared_from_this_create_derived_in_cpp _________________

clean = None

    def test13_shared_from_this_create_derived_in_cpp(clean):
        # This tests that keep_shared_from_this_alive is inherited by
        # derived classes properly
    
        # Pass shared_ptr<T> to Python
        e = t.DerivedST.make_shared(20)
        assert type(e) is t.DerivedST
        assert e.value == 20
        assert e.has_shared_from_this()
        assert not t.owns_cpp(e)
        assert e.use_count() == 1
    
        # Pass it back to C++
        w = t.SharedWrapperST(e)
        assert e.use_count() == w.use_count() == 2
    
        del e
        collect()
>       assert t.stats() == (1, 0)
E       assert (1, 1) == (1, 0)
E         
E         At index 1 diff: 1 != 0
E         Use -v to get more diff

tests/test_holders.py:406: AssertionError
=========================== short test summary info ============================
FAILED tests/test_functions.py::test50_call_policy - assert (3221225472, 2) =...
FAILED tests/test_holders.py::test10_shared_from_this_create_in_python - asse...
FAILED tests/test_holders.py::test11_shared_from_this_create_raw_in_cpp - ass...
FAILED tests/test_holders.py::test12_shared_from_this_create_shared_in_cpp - ...
FAILED tests/test_holders.py::test13_shared_from_this_create_derived_in_cpp
================== 5 failed, 315 passed, 138 skipped in 3.77s ==================

https://docs.python.org/3.14/whatsnew/3.14.html

For the build logs, see:
https://copr-be.cloud.fedoraproject.org/results/@python/python3.14/fedora-rawhide-x86_64/08899392-python-nanobind/

For all our attempts to build python-nanobind with Python 3.14, see:
https://copr.fedorainfracloud.org/coprs/g/python/python3.14/package/python-nanobind/

Testing and mass rebuild of packages is happening in copr.
You can follow these instructions to test locally in mock if your package builds with Python 3.14:
https://copr.fedorainfracloud.org/coprs/g/python/python3.14/

Let us know here if you have any questions.

Python 3.14 is planned to be included in Fedora 43.
To make that update smoother, we're building Fedora packages with all pre-releases of Python 3.14.
A build failure prevents us from testing all dependent packages (transitive [Build]Requires),
so if this package is required a lot, it's important for us to get it fixed soon.

We'd appreciate help from the people who know this package best,
but if you don't want to work on this now, let us know so we can try to work around it on our side.

Comment 1 Tulio Magno Quites Machado Filho 2025-04-23 14:25:24 UTC
Upstream has already been testing with Python 3.14.0-alpha.6: https://github.com/wjakob/nanobind/commit/1e5b87938fc5ba66c9cdac3dcbafb7e9f2867e47

The tests with python-nanobind 2.7.0 have been successful recently.
It might be worth retrying this after this update reaches rawhide.
See: https://src.fedoraproject.org/rpms/python-nanobind/pull-request/7

Comment 2 Tulio Magno Quites Machado Filho 2025-04-23 14:26:37 UTC
I've just found this: https://copr.fedorainfracloud.org/coprs/g/python/python3.14/build/8955535/
Does that mean the build was indeed successful and we can now close this bug?
If that's correct, this was fast. :-)

Comment 3 Miro Hrončok 2025-04-23 15:16:29 UTC
> Does that mean the build was indeed successful

Yes, but consider it a "scratch build" as it was built form the PR and is not included in the copr's dnf repository.

> and we can now close this bug?

Once the PR is merged, yes.

Comment 4 Tulio Magno Quites Machado Filho 2025-04-25 13:50:36 UTC
We've had successful builds in Copr since the PR got merged, e.g. https://copr.fedorainfracloud.org/coprs/g/python/python3.14/build/8956783/

I'm closing this now.
Thanks!


Note You need to log in before you can comment on or make changes to this bug.