Bug 2219334 - python-cloudpickle: FTBFS in f39-python side tag with Python 3.12
Summary: python-cloudpickle: FTBFS in f39-python side tag with Python 3.12
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: python-cloudpickle
Version: 39
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Miro Hrončok
QA Contact: Fedora Extras Quality Assurance
URL: https://koschei.fedoraproject.org/pac...
Whiteboard:
Depends On:
Blocks: F40FTBFS PYTHON3.12 F39FTBFS
TreeView+ depends on / blocked
 
Reported: 2023-07-03 09:06 UTC by Tomáš Hrnčiar
Modified: 2024-08-09 19:33 UTC (History)
7 users (show)

Fixed In Version: python-cloudpickle-2.2.1-2.fc39 python-cloudpickle-2.2.1-4.fc40 python-cloudpickle-3.0.0-8.el10_0
Clone Of:
Environment:
Last Closed: 2023-09-07 10:52:34 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github cloudpipe cloudpickle issues 507 0 None open Python 3.12: TypeError: cannot create weak reference to 'typing.TypeVar' object 2023-07-03 14:29:13 UTC
Github python cpython issues 106403 0 None open 3.12 regression: cannot create weak reference to 'typing.TypeVar' object 2023-07-04 10:13:25 UTC

Description Tomáš Hrnčiar 2023-07-03 09:06:16 UTC
python-cloudpickle fails to build with Python 3.12.0b3 in f39-sidetag

For the build logs, see:
https://koji.fedoraproject.org/koji/buildinfo?buildID=2218638

To reproduce this locally you can do:
    fedpkg mock-config --target f39-python > ~/.config/mock/f39-python.cfg
    fedpkg srpm
    mock -r f39-python --rebuild $SRPM

Or you can submit a scratch build in side tag with:
    fedpkg build --target=f39-python --scratch --srpm

Once fixed, please submit the real build into side tag with:
    fedpkg build --target=f39-python

=================================== FAILURES ===================================
____________________ CloudPickleTest.test_generic_subclass _____________________
self = <tests.cloudpickle_test.CloudPickleTest testMethod=test_generic_subclass>
    def test_generic_subclass(self):
        T = typing.TypeVar('T')
    
        class Base(typing.Generic[T]):
            pass
    
        class DerivedAny(Base):
            pass
    
        class LeafAny(DerivedAny):
            pass
    
        class DerivedInt(Base[int]):
            pass
    
        class LeafInt(DerivedInt):
            pass
    
        class DerivedT(Base[T]):
            pass
    
        class LeafT(DerivedT[T]):
            pass
    
        klasses = [
            Base, DerivedAny, LeafAny, DerivedInt, LeafInt, DerivedT, LeafT
        ]
        for klass in klasses:
>           assert pickle_depickle(klass, protocol=self.protocol) is klass
tests/cloudpickle_test.py:2423: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/cloudpickle_test.py:81: in pickle_depickle
    return pickle.loads(cloudpickle.dumps(obj, protocol=protocol))
cloudpickle/cloudpickle_fast.py:73: in dumps
    cp.dump(obj)
cloudpickle/cloudpickle_fast.py:632: in dump
    return Pickler.dump(self, obj)
cloudpickle/cloudpickle.py:909: in _typevar_reduce
    return (_make_typevar, _decompose_typevar(obj))
cloudpickle/cloudpickle.py:899: in _decompose_typevar
    _get_or_create_tracker_id(obj),
cloudpickle/cloudpickle.py:107: in _get_or_create_tracker_id
    class_tracker_id = _DYNAMIC_CLASS_TRACKER_BY_CLASS.get(class_def)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <WeakKeyDictionary at 0xf6742fc0>, key = ~T, default = None
    def get(self, key, default=None):
>       return self.data.get(ref(key),default)
E       TypeError: cannot create weak reference to 'typing.TypeVar' object
/usr/lib/python3.12/weakref.py:452: TypeError
______________________ CloudPickleTest.test_generic_type _______________________
self = <tests.cloudpickle_test.CloudPickleTest testMethod=test_generic_type>
    def test_generic_type(self):
        T = typing.TypeVar('T')
    
        class C(typing.Generic[T]):
            pass
    
>       assert pickle_depickle(C, protocol=self.protocol) is C
tests/cloudpickle_test.py:2364: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/cloudpickle_test.py:81: in pickle_depickle
    return pickle.loads(cloudpickle.dumps(obj, protocol=protocol))
cloudpickle/cloudpickle_fast.py:73: in dumps
    cp.dump(obj)
cloudpickle/cloudpickle_fast.py:632: in dump
    return Pickler.dump(self, obj)
cloudpickle/cloudpickle.py:909: in _typevar_reduce
    return (_make_typevar, _decompose_typevar(obj))
cloudpickle/cloudpickle.py:899: in _decompose_typevar
    _get_or_create_tracker_id(obj),
cloudpickle/cloudpickle.py:107: in _get_or_create_tracker_id
    class_tracker_id = _DYNAMIC_CLASS_TRACKER_BY_CLASS.get(class_def)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <WeakKeyDictionary at 0xf6742fc0>, key = ~T, default = None
    def get(self, key, default=None):
>       return self.data.get(ref(key),default)
E       TypeError: cannot create weak reference to 'typing.TypeVar' object
/usr/lib/python3.12/weakref.py:452: TypeError
__________ CloudPickleTest.test_locally_defined_class_with_type_hints __________
self = <tests.cloudpickle_test.CloudPickleTest testMethod=test_locally_defined_class_with_type_hints>
    def test_locally_defined_class_with_type_hints(self):
        with subprocess_worker(protocol=self.protocol) as worker:
            for type_ in _all_types_to_test():
                class MyClass:
                    def method(self, arg: type_) -> type_:
                        return arg
                MyClass.__annotations__ = {'attribute': type_}
    
                def check_annotations(obj, expected_type, expected_type_str):
                    assert obj.__annotations__["attribute"] == expected_type
                    assert (
                        obj.method.__annotations__["arg"] == expected_type
                    )
                    assert (
                        obj.method.__annotations__["return"]
                        == expected_type
                    )
                    return "ok"
    
                obj = MyClass()
                assert check_annotations(obj, type_, "type_") == "ok"
>               assert (
                    worker.run(check_annotations, obj, type_, "type_") == "ok"
                )
tests/cloudpickle_test.py:2457: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/testutils.py:146: in run
    input_payload = dumps((func, args, kwargs), protocol=self.protocol)
cloudpickle/cloudpickle_fast.py:73: in dumps
    cp.dump(obj)
cloudpickle/cloudpickle_fast.py:632: in dump
    return Pickler.dump(self, obj)
cloudpickle/cloudpickle.py:909: in _typevar_reduce
    return (_make_typevar, _decompose_typevar(obj))
cloudpickle/cloudpickle.py:899: in _decompose_typevar
    _get_or_create_tracker_id(obj),
cloudpickle/cloudpickle.py:107: in _get_or_create_tracker_id
    class_tracker_id = _DYNAMIC_CLASS_TRACKER_BY_CLASS.get(class_def)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <WeakKeyDictionary at 0xf6742fc0>, key = ~T, default = None
    def get(self, key, default=None):
>       return self.data.get(ref(key),default)
E       TypeError: cannot create weak reference to 'typing.TypeVar' object
/usr/lib/python3.12/weakref.py:452: TypeError
_ CloudPickleTest.test_pickle_constructs_from_module_registered_for_pickling_by_value _
self = <tests.cloudpickle_test.CloudPickleTest testMethod=test_pickle_constructs_from_module_registered_for_pickling_by_value>
    def test_pickle_constructs_from_module_registered_for_pickling_by_value(self):  # noqa
        _prev_sys_path = sys.path.copy()
        try:
            # We simulate an interactive session that:
            # - we start from the /path/to/cloudpickle/tests directory, where a
            #   local .py file (mock_local_file) is located.
            # - uses constructs from mock_local_file in remote workers that do
            #   not have access to this file. This situation is
            #   the justification behind the
            #   (un)register_pickle_by_value(module) api that cloudpickle
            #   exposes.
            _mock_interactive_session_cwd = os.path.dirname(__file__)
    
            # First, remove sys.path entries that could point to
            # /path/to/cloudpickle/tests and be in inherited by the worker
            _maybe_remove(sys.path, '')
            _maybe_remove(sys.path, _mock_interactive_session_cwd)
    
            # Add the desired session working directory
            sys.path.insert(0, _mock_interactive_session_cwd)
    
            with subprocess_worker(protocol=self.protocol) as w:
                # Make the module unavailable in the remote worker
                w.run(
                    lambda p: sys.path.remove(p), _mock_interactive_session_cwd
                )
                # Import the actual file after starting the module since the
                # worker is started using fork on Linux, which will inherits
                # the parent sys.modules. On Python>3.6, the worker can be
                # started using spawn using mp_context in ProcessPoolExectutor.
                # TODO Once Python 3.6 reaches end of life, rely on mp_context
                # instead.
                import mock_local_folder.mod as mod
                # The constructs whose pickling mechanism is changed using
                # register_pickle_by_value are functions, classes, TypeVar and
                # modules.
                from mock_local_folder.mod import (
                    local_function, LocalT, LocalClass
                )
    
                # Make sure the module/constructs are unimportable in the
                # worker.
                with pytest.raises(ImportError):
                    w.run(lambda: __import__("mock_local_folder.mod"))
                with pytest.raises(ImportError):
                    w.run(
                        lambda: __import__("mock_local_folder.subfolder.mod")
                    )
    
                for o in [mod, local_function, LocalT, LocalClass]:
                    with pytest.raises(ImportError):
                        w.run(lambda: o)
    
                register_pickle_by_value(mod)
                # function
                assert w.run(lambda: local_function()) == local_function()
                # typevar
>               assert w.run(lambda: LocalT.__name__) == LocalT.__name__
tests/cloudpickle_test.py:2583: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/testutils.py:146: in run
    input_payload = dumps((func, args, kwargs), protocol=self.protocol)
cloudpickle/cloudpickle_fast.py:73: in dumps
    cp.dump(obj)
cloudpickle/cloudpickle_fast.py:632: in dump
    return Pickler.dump(self, obj)
cloudpickle/cloudpickle.py:911: in _typevar_reduce
    return (_make_typevar, _decompose_typevar(obj))
cloudpickle/cloudpickle.py:899: in _decompose_typevar
    _get_or_create_tracker_id(obj),
cloudpickle/cloudpickle.py:107: in _get_or_create_tracker_id
    class_tracker_id = _DYNAMIC_CLASS_TRACKER_BY_CLASS.get(class_def)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <WeakKeyDictionary at 0xf6742fc0>, key = ~LocalT, default = None
    def get(self, key, default=None):
>       return self.data.get(ref(key),default)
E       TypeError: cannot create weak reference to 'typing.TypeVar' object
/usr/lib/python3.12/weakref.py:452: TypeError
_________________ CloudPickleTest.test_pickle_dynamic_typevar __________________
self = <tests.cloudpickle_test.CloudPickleTest testMethod=test_pickle_dynamic_typevar>
    def test_pickle_dynamic_typevar(self):
        T = typing.TypeVar('T')
>       depickled_T = pickle_depickle(T, protocol=self.protocol)
tests/cloudpickle_test.py:2330: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/cloudpickle_test.py:81: in pickle_depickle
    return pickle.loads(cloudpickle.dumps(obj, protocol=protocol))
cloudpickle/cloudpickle_fast.py:73: in dumps
    cp.dump(obj)
cloudpickle/cloudpickle_fast.py:632: in dump
    return Pickler.dump(self, obj)
cloudpickle/cloudpickle.py:909: in _typevar_reduce
    return (_make_typevar, _decompose_typevar(obj))
cloudpickle/cloudpickle.py:899: in _decompose_typevar
    _get_or_create_tracker_id(obj),
cloudpickle/cloudpickle.py:107: in _get_or_create_tracker_id
    class_tracker_id = _DYNAMIC_CLASS_TRACKER_BY_CLASS.get(class_def)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <WeakKeyDictionary at 0xf6742fc0>, key = ~T, default = None
    def get(self, key, default=None):
>       return self.data.get(ref(key),default)
E       TypeError: cannot create weak reference to 'typing.TypeVar' object
/usr/lib/python3.12/weakref.py:452: TypeError
___________ CloudPickleTest.test_pickle_dynamic_typevar_memoization ____________
self = <tests.cloudpickle_test.CloudPickleTest testMethod=test_pickle_dynamic_typevar_memoization>
    def test_pickle_dynamic_typevar_memoization(self):
        T = typing.TypeVar('T')
>       depickled_T1, depickled_T2 = pickle_depickle((T, T),
                                                     protocol=self.protocol)
tests/cloudpickle_test.py:2345: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/cloudpickle_test.py:81: in pickle_depickle
    return pickle.loads(cloudpickle.dumps(obj, protocol=protocol))
cloudpickle/cloudpickle_fast.py:73: in dumps
    cp.dump(obj)
cloudpickle/cloudpickle_fast.py:632: in dump
    return Pickler.dump(self, obj)
cloudpickle/cloudpickle.py:909: in _typevar_reduce
    return (_make_typevar, _decompose_typevar(obj))
cloudpickle/cloudpickle.py:899: in _decompose_typevar
    _get_or_create_tracker_id(obj),
cloudpickle/cloudpickle.py:107: in _get_or_create_tracker_id
    class_tracker_id = _DYNAMIC_CLASS_TRACKER_BY_CLASS.get(class_def)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <WeakKeyDictionary at 0xf6742fc0>, key = ~T, default = None
    def get(self, key, default=None):
>       return self.data.get(ref(key),default)
E       TypeError: cannot create weak reference to 'typing.TypeVar' object
/usr/lib/python3.12/weakref.py:452: TypeError
_____________ CloudPickleTest.test_pickle_dynamic_typevar_tracking _____________
self = <tests.cloudpickle_test.CloudPickleTest testMethod=test_pickle_dynamic_typevar_tracking>
    def test_pickle_dynamic_typevar_tracking(self):
        T = typing.TypeVar("T")
>       T2 = subprocess_pickle_echo(T, protocol=self.protocol)
tests/cloudpickle_test.py:2340: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/testutils.py:89: in subprocess_pickle_echo
    out = subprocess_pickle_string(input_data,
tests/testutils.py:63: in subprocess_pickle_string
    pickle_string = dumps(input_data, protocol=protocol)
cloudpickle/cloudpickle_fast.py:73: in dumps
    cp.dump(obj)
cloudpickle/cloudpickle_fast.py:632: in dump
    return Pickler.dump(self, obj)
cloudpickle/cloudpickle.py:909: in _typevar_reduce
    return (_make_typevar, _decompose_typevar(obj))
cloudpickle/cloudpickle.py:899: in _decompose_typevar
    _get_or_create_tracker_id(obj),
cloudpickle/cloudpickle.py:107: in _get_or_create_tracker_id
    class_tracker_id = _DYNAMIC_CLASS_TRACKER_BY_CLASS.get(class_def)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <WeakKeyDictionary at 0xf6742fc0>, key = ~T, default = None
    def get(self, key, default=None):
>       return self.data.get(ref(key),default)
E       TypeError: cannot create weak reference to 'typing.TypeVar' object
/usr/lib/python3.12/weakref.py:452: TypeError
________________ Protocol2CloudPickleTest.test_generic_subclass ________________
self = <tests.cloudpickle_test.Protocol2CloudPickleTest testMethod=test_generic_subclass>
    def test_generic_subclass(self):
        T = typing.TypeVar('T')
    
        class Base(typing.Generic[T]):
            pass
    
        class DerivedAny(Base):
            pass
    
        class LeafAny(DerivedAny):
            pass
    
        class DerivedInt(Base[int]):
            pass
    
        class LeafInt(DerivedInt):
            pass
    
        class DerivedT(Base[T]):
            pass
    
        class LeafT(DerivedT[T]):
            pass
    
        klasses = [
            Base, DerivedAny, LeafAny, DerivedInt, LeafInt, DerivedT, LeafT
        ]
        for klass in klasses:
>           assert pickle_depickle(klass, protocol=self.protocol) is klass
tests/cloudpickle_test.py:2423: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/cloudpickle_test.py:81: in pickle_depickle
    return pickle.loads(cloudpickle.dumps(obj, protocol=protocol))
cloudpickle/cloudpickle_fast.py:73: in dumps
    cp.dump(obj)
cloudpickle/cloudpickle_fast.py:632: in dump
    return Pickler.dump(self, obj)
cloudpickle/cloudpickle.py:909: in _typevar_reduce
    return (_make_typevar, _decompose_typevar(obj))
cloudpickle/cloudpickle.py:899: in _decompose_typevar
    _get_or_create_tracker_id(obj),
cloudpickle/cloudpickle.py:107: in _get_or_create_tracker_id
    class_tracker_id = _DYNAMIC_CLASS_TRACKER_BY_CLASS.get(class_def)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <WeakKeyDictionary at 0xf6742fc0>, key = ~T, default = None
    def get(self, key, default=None):
>       return self.data.get(ref(key),default)
E       TypeError: cannot create weak reference to 'typing.TypeVar' object
/usr/lib/python3.12/weakref.py:452: TypeError
__________________ Protocol2CloudPickleTest.test_generic_type __________________
self = <tests.cloudpickle_test.Protocol2CloudPickleTest testMethod=test_generic_type>
    def test_generic_type(self):
        T = typing.TypeVar('T')
    
        class C(typing.Generic[T]):
            pass
    
>       assert pickle_depickle(C, protocol=self.protocol) is C
tests/cloudpickle_test.py:2364: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/cloudpickle_test.py:81: in pickle_depickle
    return pickle.loads(cloudpickle.dumps(obj, protocol=protocol))
cloudpickle/cloudpickle_fast.py:73: in dumps
    cp.dump(obj)
cloudpickle/cloudpickle_fast.py:632: in dump
    return Pickler.dump(self, obj)
cloudpickle/cloudpickle.py:909: in _typevar_reduce
    return (_make_typevar, _decompose_typevar(obj))
cloudpickle/cloudpickle.py:899: in _decompose_typevar
    _get_or_create_tracker_id(obj),
cloudpickle/cloudpickle.py:107: in _get_or_create_tracker_id
    class_tracker_id = _DYNAMIC_CLASS_TRACKER_BY_CLASS.get(class_def)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <WeakKeyDictionary at 0xf6742fc0>, key = ~T, default = None
    def get(self, key, default=None):
>       return self.data.get(ref(key),default)
E       TypeError: cannot create weak reference to 'typing.TypeVar' object
/usr/lib/python3.12/weakref.py:452: TypeError
_____ Protocol2CloudPickleTest.test_locally_defined_class_with_type_hints ______
self = <tests.cloudpickle_test.Protocol2CloudPickleTest testMethod=test_locally_defined_class_with_type_hints>
    def test_locally_defined_class_with_type_hints(self):
        with subprocess_worker(protocol=self.protocol) as worker:
            for type_ in _all_types_to_test():
                class MyClass:
                    def method(self, arg: type_) -> type_:
                        return arg
                MyClass.__annotations__ = {'attribute': type_}
    
                def check_annotations(obj, expected_type, expected_type_str):
                    assert obj.__annotations__["attribute"] == expected_type
                    assert (
                        obj.method.__annotations__["arg"] == expected_type
                    )
                    assert (
                        obj.method.__annotations__["return"]
                        == expected_type
                    )
                    return "ok"
    
                obj = MyClass()
                assert check_annotations(obj, type_, "type_") == "ok"
>               assert (
                    worker.run(check_annotations, obj, type_, "type_") == "ok"
                )
tests/cloudpickle_test.py:2457: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/testutils.py:146: in run
    input_payload = dumps((func, args, kwargs), protocol=self.protocol)
cloudpickle/cloudpickle_fast.py:73: in dumps
    cp.dump(obj)
cloudpickle/cloudpickle_fast.py:632: in dump
    return Pickler.dump(self, obj)
cloudpickle/cloudpickle.py:909: in _typevar_reduce
    return (_make_typevar, _decompose_typevar(obj))
cloudpickle/cloudpickle.py:899: in _decompose_typevar
    _get_or_create_tracker_id(obj),
cloudpickle/cloudpickle.py:107: in _get_or_create_tracker_id
    class_tracker_id = _DYNAMIC_CLASS_TRACKER_BY_CLASS.get(class_def)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <WeakKeyDictionary at 0xf6742fc0>, key = ~T, default = None
    def get(self, key, default=None):
>       return self.data.get(ref(key),default)
E       TypeError: cannot create weak reference to 'typing.TypeVar' object
/usr/lib/python3.12/weakref.py:452: TypeError
_ Protocol2CloudPickleTest.test_pickle_constructs_from_module_registered_for_pickling_by_value _
self = <tests.cloudpickle_test.Protocol2CloudPickleTest testMethod=test_pickle_constructs_from_module_registered_for_pickling_by_value>
    def test_pickle_constructs_from_module_registered_for_pickling_by_value(self):  # noqa
        _prev_sys_path = sys.path.copy()
        try:
            # We simulate an interactive session that:
            # - we start from the /path/to/cloudpickle/tests directory, where a
            #   local .py file (mock_local_file) is located.
            # - uses constructs from mock_local_file in remote workers that do
            #   not have access to this file. This situation is
            #   the justification behind the
            #   (un)register_pickle_by_value(module) api that cloudpickle
            #   exposes.
            _mock_interactive_session_cwd = os.path.dirname(__file__)
    
            # First, remove sys.path entries that could point to
            # /path/to/cloudpickle/tests and be in inherited by the worker
            _maybe_remove(sys.path, '')
            _maybe_remove(sys.path, _mock_interactive_session_cwd)
    
            # Add the desired session working directory
            sys.path.insert(0, _mock_interactive_session_cwd)
    
            with subprocess_worker(protocol=self.protocol) as w:
                # Make the module unavailable in the remote worker
                w.run(
                    lambda p: sys.path.remove(p), _mock_interactive_session_cwd
                )
                # Import the actual file after starting the module since the
                # worker is started using fork on Linux, which will inherits
                # the parent sys.modules. On Python>3.6, the worker can be
                # started using spawn using mp_context in ProcessPoolExectutor.
                # TODO Once Python 3.6 reaches end of life, rely on mp_context
                # instead.
                import mock_local_folder.mod as mod
                # The constructs whose pickling mechanism is changed using
                # register_pickle_by_value are functions, classes, TypeVar and
                # modules.
                from mock_local_folder.mod import (
                    local_function, LocalT, LocalClass
                )
    
                # Make sure the module/constructs are unimportable in the
                # worker.
                with pytest.raises(ImportError):
                    w.run(lambda: __import__("mock_local_folder.mod"))
                with pytest.raises(ImportError):
                    w.run(
                        lambda: __import__("mock_local_folder.subfolder.mod")
                    )
    
                for o in [mod, local_function, LocalT, LocalClass]:
                    with pytest.raises(ImportError):
                        w.run(lambda: o)
    
                register_pickle_by_value(mod)
                # function
                assert w.run(lambda: local_function()) == local_function()
                # typevar
>               assert w.run(lambda: LocalT.__name__) == LocalT.__name__
tests/cloudpickle_test.py:2583: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/testutils.py:146: in run
    input_payload = dumps((func, args, kwargs), protocol=self.protocol)
cloudpickle/cloudpickle_fast.py:73: in dumps
    cp.dump(obj)
cloudpickle/cloudpickle_fast.py:632: in dump
    return Pickler.dump(self, obj)
cloudpickle/cloudpickle.py:911: in _typevar_reduce
    return (_make_typevar, _decompose_typevar(obj))
cloudpickle/cloudpickle.py:899: in _decompose_typevar
    _get_or_create_tracker_id(obj),
cloudpickle/cloudpickle.py:107: in _get_or_create_tracker_id
    class_tracker_id = _DYNAMIC_CLASS_TRACKER_BY_CLASS.get(class_def)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <WeakKeyDictionary at 0xf6742fc0>, key = ~LocalT, default = None
    def get(self, key, default=None):
>       return self.data.get(ref(key),default)
E       TypeError: cannot create weak reference to 'typing.TypeVar' object
/usr/lib/python3.12/weakref.py:452: TypeError
_____________ Protocol2CloudPickleTest.test_pickle_dynamic_typevar _____________
self = <tests.cloudpickle_test.Protocol2CloudPickleTest testMethod=test_pickle_dynamic_typevar>
    def test_pickle_dynamic_typevar(self):
        T = typing.TypeVar('T')
>       depickled_T = pickle_depickle(T, protocol=self.protocol)
tests/cloudpickle_test.py:2330: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/cloudpickle_test.py:81: in pickle_depickle
    return pickle.loads(cloudpickle.dumps(obj, protocol=protocol))
cloudpickle/cloudpickle_fast.py:73: in dumps
    cp.dump(obj)
cloudpickle/cloudpickle_fast.py:632: in dump
    return Pickler.dump(self, obj)
cloudpickle/cloudpickle.py:909: in _typevar_reduce
    return (_make_typevar, _decompose_typevar(obj))
cloudpickle/cloudpickle.py:899: in _decompose_typevar
    _get_or_create_tracker_id(obj),
cloudpickle/cloudpickle.py:107: in _get_or_create_tracker_id
    class_tracker_id = _DYNAMIC_CLASS_TRACKER_BY_CLASS.get(class_def)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <WeakKeyDictionary at 0xf6742fc0>, key = ~T, default = None
    def get(self, key, default=None):
>       return self.data.get(ref(key),default)
E       TypeError: cannot create weak reference to 'typing.TypeVar' object
/usr/lib/python3.12/weakref.py:452: TypeError
_______ Protocol2CloudPickleTest.test_pickle_dynamic_typevar_memoization _______
self = <tests.cloudpickle_test.Protocol2CloudPickleTest testMethod=test_pickle_dynamic_typevar_memoization>
    def test_pickle_dynamic_typevar_memoization(self):
        T = typing.TypeVar('T')
>       depickled_T1, depickled_T2 = pickle_depickle((T, T),
                                                     protocol=self.protocol)
tests/cloudpickle_test.py:2345: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/cloudpickle_test.py:81: in pickle_depickle
    return pickle.loads(cloudpickle.dumps(obj, protocol=protocol))
cloudpickle/cloudpickle_fast.py:73: in dumps
    cp.dump(obj)
cloudpickle/cloudpickle_fast.py:632: in dump
    return Pickler.dump(self, obj)
cloudpickle/cloudpickle.py:909: in _typevar_reduce
    return (_make_typevar, _decompose_typevar(obj))
cloudpickle/cloudpickle.py:899: in _decompose_typevar
    _get_or_create_tracker_id(obj),
cloudpickle/cloudpickle.py:107: in _get_or_create_tracker_id
    class_tracker_id = _DYNAMIC_CLASS_TRACKER_BY_CLASS.get(class_def)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <WeakKeyDictionary at 0xf6742fc0>, key = ~T, default = None
    def get(self, key, default=None):
>       return self.data.get(ref(key),default)
E       TypeError: cannot create weak reference to 'typing.TypeVar' object
/usr/lib/python3.12/weakref.py:452: TypeError
________ Protocol2CloudPickleTest.test_pickle_dynamic_typevar_tracking _________
self = <tests.cloudpickle_test.Protocol2CloudPickleTest testMethod=test_pickle_dynamic_typevar_tracking>
    def test_pickle_dynamic_typevar_tracking(self):
        T = typing.TypeVar("T")
>       T2 = subprocess_pickle_echo(T, protocol=self.protocol)
tests/cloudpickle_test.py:2340: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/testutils.py:89: in subprocess_pickle_echo
    out = subprocess_pickle_string(input_data,
tests/testutils.py:63: in subprocess_pickle_string
    pickle_string = dumps(input_data, protocol=protocol)
cloudpickle/cloudpickle_fast.py:73: in dumps
    cp.dump(obj)
cloudpickle/cloudpickle_fast.py:632: in dump
    return Pickler.dump(self, obj)
cloudpickle/cloudpickle.py:909: in _typevar_reduce
    return (_make_typevar, _decompose_typevar(obj))
cloudpickle/cloudpickle.py:899: in _decompose_typevar
    _get_or_create_tracker_id(obj),
cloudpickle/cloudpickle.py:107: in _get_or_create_tracker_id
    class_tracker_id = _DYNAMIC_CLASS_TRACKER_BY_CLASS.get(class_def)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <WeakKeyDictionary at 0xf6742fc0>, key = ~T, default = None
    def get(self, key, default=None):
>       return self.data.get(ref(key),default)
E       TypeError: cannot create weak reference to 'typing.TypeVar' object
/usr/lib/python3.12/weakref.py:452: TypeError
...
=========================== short test summary info ============================
FAILED tests/cloudpickle_test.py::CloudPickleTest::test_generic_subclass - Ty...
FAILED tests/cloudpickle_test.py::CloudPickleTest::test_generic_type - TypeEr...
FAILED tests/cloudpickle_test.py::CloudPickleTest::test_locally_defined_class_with_type_hints
FAILED tests/cloudpickle_test.py::CloudPickleTest::test_pickle_constructs_from_module_registered_for_pickling_by_value
FAILED tests/cloudpickle_test.py::CloudPickleTest::test_pickle_dynamic_typevar
FAILED tests/cloudpickle_test.py::CloudPickleTest::test_pickle_dynamic_typevar_memoization
FAILED tests/cloudpickle_test.py::CloudPickleTest::test_pickle_dynamic_typevar_tracking
FAILED tests/cloudpickle_test.py::Protocol2CloudPickleTest::test_generic_subclass
FAILED tests/cloudpickle_test.py::Protocol2CloudPickleTest::test_generic_type
FAILED tests/cloudpickle_test.py::Protocol2CloudPickleTest::test_locally_defined_class_with_type_hints
FAILED tests/cloudpickle_test.py::Protocol2CloudPickleTest::test_pickle_constructs_from_module_registered_for_pickling_by_value
FAILED tests/cloudpickle_test.py::Protocol2CloudPickleTest::test_pickle_dynamic_typevar
FAILED tests/cloudpickle_test.py::Protocol2CloudPickleTest::test_pickle_dynamic_typevar_memoization
FAILED tests/cloudpickle_test.py::Protocol2CloudPickleTest::test_pickle_dynamic_typevar_tracking
==== 14 failed, 226 passed, 15 skipped, 5 deselected, 12 warnings in 21.86s ====

Comment 1 Ben Beasley 2023-07-03 14:29:14 UTC
Submitted upstream: https://github.com/cloudpipe/cloudpickle/issues/507

Comment 2 Ben Beasley 2023-07-03 14:45:18 UTC
PR to work around distutils removal (the easy part): https://github.com/cloudpipe/cloudpickle/pull/508

Comment 3 Miro Hrončok 2023-07-03 18:10:32 UTC
This is currently one of the biggest blockers for the Python 3.12 rebuild.

Also, Lumír is on vacation.

Comment 4 Ben Beasley 2023-07-03 18:28:55 UTC
(In reply to Miro Hrončok from comment #3)
> This is currently one of the biggest blockers for the Python 3.12 rebuild.
> 
> Also, Lumír is on vacation.

Any idea what should be done about “TypeError: cannot create weak reference to 'typing.TypeVar' object”?

Comment 5 Miro Hrončok 2023-07-03 18:30:03 UTC
Not yet. Will discus with Petr Viktorin ASAP.

Comment 6 Miro Hrončok 2023-07-04 10:13:26 UTC
> Any idea what should be done about “TypeError: cannot create weak reference to 'typing.TypeVar' object”?

Apparently, a CPython regression: https://github.com/python/cpython/issues/106403

Will see if we skip the tests, if joblib and others build.

Comment 7 Miro Hrončok 2023-07-04 11:35:24 UTC
Tests skipped. Leaving MODIFIED, we will want to unskip them later.

Comment 8 Fedora Release Engineering 2023-08-16 07:15:21 UTC
This bug appears to have been reported against 'rawhide' during the Fedora Linux 39 development cycle.
Changing version to 39.

Comment 9 Fedora Update System 2023-09-07 10:51:13 UTC
FEDORA-2023-6f2952f81d has been submitted as an update to Fedora 40. https://bodhi.fedoraproject.org/updates/FEDORA-2023-6f2952f81d

Comment 10 Fedora Update System 2023-09-07 10:52:34 UTC
FEDORA-2023-6f2952f81d has been pushed to the Fedora 40 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 11 Fedora Update System 2024-08-09 19:30:20 UTC
FEDORA-EPEL-2024-c4b2008a49 (python-cloudpickle-3.0.0-8.el10_0) has been submitted as an update to Fedora EPEL 10.0.
https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2024-c4b2008a49

Comment 12 Fedora Update System 2024-08-09 19:33:13 UTC
FEDORA-EPEL-2024-c4b2008a49 (python-cloudpickle-3.0.0-8.el10_0) has been pushed to the Fedora EPEL 10.0 stable repository.
If problem still persists, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.