Bug 2219334

Summary: python-cloudpickle: FTBFS in f39-python side tag with Python 3.12
Product: [Fedora] Fedora Reporter: Tomáš Hrnčiar <thrnciar>
Component: python-cloudpickleAssignee: Miro Hrončok <mhroncok>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 39CC: code, cstratak, epel-packagers-sig, jonathan, lbalhar, mhroncok, python-packagers-sig
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
URL: https://koschei.fedoraproject.org/package/python-cloudpickle
Whiteboard:
Fixed In Version: python-cloudpickle-2.2.1-2.fc39 python-cloudpickle-2.2.1-4.fc40 python-cloudpickle-3.0.0-8.el10_0 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-09-07 10:52:34 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2135404, 2168842, 2231791    

Description Tomáš Hrnčiar 2023-07-03 09:06:16 UTC 
python-cloudpickle fails to build with Python 3.12.0b3 in f39-sidetag

For the build logs, see:
https://koji.fedoraproject.org/koji/buildinfo?buildID=2218638

To reproduce this locally you can do:
    fedpkg mock-config --target f39-python > ~/.config/mock/f39-python.cfg
    fedpkg srpm
    mock -r f39-python --rebuild $SRPM

Or you can submit a scratch build in side tag with:
    fedpkg build --target=f39-python --scratch --srpm

Once fixed, please submit the real build into side tag with:
    fedpkg build --target=f39-python

=================================== FAILURES ===================================
____________________ CloudPickleTest.test_generic_subclass _____________________
self = <tests.cloudpickle_test.CloudPickleTest testMethod=test_generic_subclass>
    def test_generic_subclass(self):
        T = typing.TypeVar('T')
    
        class Base(typing.Generic[T]):
            pass
    
        class DerivedAny(Base):
            pass
    
        class LeafAny(DerivedAny):
            pass
    
        class DerivedInt(Base[int]):
            pass
    
        class LeafInt(DerivedInt):
            pass
    
        class DerivedT(Base[T]):
            pass
    
        class LeafT(DerivedT[T]):
            pass
    
        klasses = [
            Base, DerivedAny, LeafAny, DerivedInt, LeafInt, DerivedT, LeafT
        ]
        for klass in klasses:
>           assert pickle_depickle(klass, protocol=self.protocol) is klass
tests/cloudpickle_test.py:2423: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/cloudpickle_test.py:81: in pickle_depickle
    return pickle.loads(cloudpickle.dumps(obj, protocol=protocol))
cloudpickle/cloudpickle_fast.py:73: in dumps
    cp.dump(obj)
cloudpickle/cloudpickle_fast.py:632: in dump
    return Pickler.dump(self, obj)
cloudpickle/cloudpickle.py:909: in _typevar_reduce
    return (_make_typevar, _decompose_typevar(obj))
cloudpickle/cloudpickle.py:899: in _decompose_typevar
    _get_or_create_tracker_id(obj),
cloudpickle/cloudpickle.py:107: in _get_or_create_tracker_id
    class_tracker_id = _DYNAMIC_CLASS_TRACKER_BY_CLASS.get(class_def)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <WeakKeyDictionary at 0xf6742fc0>, key = ~T, default = None
    def get(self, key, default=None):
>       return self.data.get(ref(key),default)
E       TypeError: cannot create weak reference to 'typing.TypeVar' object
/usr/lib/python3.12/weakref.py:452: TypeError
______________________ CloudPickleTest.test_generic_type _______________________
self = <tests.cloudpickle_test.CloudPickleTest testMethod=test_generic_type>
    def test_generic_type(self):
        T = typing.TypeVar('T')
    
        class C(typing.Generic[T]):
            pass
    
>       assert pickle_depickle(C, protocol=self.protocol) is C
tests/cloudpickle_test.py:2364: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/cloudpickle_test.py:81: in pickle_depickle
    return pickle.loads(cloudpickle.dumps(obj, protocol=protocol))
cloudpickle/cloudpickle_fast.py:73: in dumps
    cp.dump(obj)
cloudpickle/cloudpickle_fast.py:632: in dump
    return Pickler.dump(self, obj)
cloudpickle/cloudpickle.py:909: in _typevar_reduce
    return (_make_typevar, _decompose_typevar(obj))
cloudpickle/cloudpickle.py:899: in _decompose_typevar
    _get_or_create_tracker_id(obj),
cloudpickle/cloudpickle.py:107: in _get_or_create_tracker_id
    class_tracker_id = _DYNAMIC_CLASS_TRACKER_BY_CLASS.get(class_def)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <WeakKeyDictionary at 0xf6742fc0>, key = ~T, default = None
    def get(self, key, default=None):
>       return self.data.get(ref(key),default)
E       TypeError: cannot create weak reference to 'typing.TypeVar' object
/usr/lib/python3.12/weakref.py:452: TypeError
__________ CloudPickleTest.test_locally_defined_class_with_type_hints __________
self = <tests.cloudpickle_test.CloudPickleTest testMethod=test_locally_defined_class_with_type_hints>
    def test_locally_defined_class_with_type_hints(self):
        with subprocess_worker(protocol=self.protocol) as worker:
            for type_ in _all_types_to_test():
                class MyClass:
                    def method(self, arg: type_) -> type_:
                        return arg
                MyClass.__annotations__ = {'attribute': type_}
    
                def check_annotations(obj, expected_type, expected_type_str):
                    assert obj.__annotations__["attribute"] == expected_type
                    assert (
                        obj.method.__annotations__["arg"] == expected_type
                    )
                    assert (
                        obj.method.__annotations__["return"]
                        == expected_type
                    )
                    return "ok"
    
                obj = MyClass()
                assert check_annotations(obj, type_, "type_") == "ok"
>               assert (
                    worker.run(check_annotations, obj, type_, "type_") == "ok"
                )
tests/cloudpickle_test.py:2457: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/testutils.py:146: in run
    input_payload = dumps((func, args, kwargs), protocol=self.protocol)
cloudpickle/cloudpickle_fast.py:73: in dumps
    cp.dump(obj)
cloudpickle/cloudpickle_fast.py:632: in dump
    return Pickler.dump(self, obj)
cloudpickle/cloudpickle.py:909: in _typevar_reduce
    return (_make_typevar, _decompose_typevar(obj))
cloudpickle/cloudpickle.py:899: in _decompose_typevar
    _get_or_create_tracker_id(obj),
cloudpickle/cloudpickle.py:107: in _get_or_create_tracker_id
    class_tracker_id = _DYNAMIC_CLASS_TRACKER_BY_CLASS.get(class_def)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <WeakKeyDictionary at 0xf6742fc0>, key = ~T, default = None
    def get(self, key, default=None):
>       return self.data.get(ref(key),default)
E       TypeError: cannot create weak reference to 'typing.TypeVar' object
/usr/lib/python3.12/weakref.py:452: TypeError
_ CloudPickleTest.test_pickle_constructs_from_module_registered_for_pickling_by_value _
self = <tests.cloudpickle_test.CloudPickleTest testMethod=test_pickle_constructs_from_module_registered_for_pickling_by_value>
    def test_pickle_constructs_from_module_registered_for_pickling_by_value(self):  # noqa
        _prev_sys_path = sys.path.copy()
        try:
            # We simulate an interactive session that:
            # - we start from the /path/to/cloudpickle/tests directory, where a
            #   local .py file (mock_local_file) is located.
            # - uses constructs from mock_local_file in remote workers that do
            #   not have access to this file. This situation is
            #   the justification behind the
            #   (un)register_pickle_by_value(module) api that cloudpickle
            #   exposes.
            _mock_interactive_session_cwd = os.path.dirname(__file__)
    
            # First, remove sys.path entries that could point to
            # /path/to/cloudpickle/tests and be in inherited by the worker
            _maybe_remove(sys.path, '')
            _maybe_remove(sys.path, _mock_interactive_session_cwd)
    
            # Add the desired session working directory
            sys.path.insert(0, _mock_interactive_session_cwd)
    
            with subprocess_worker(protocol=self.protocol) as w:
                # Make the module unavailable in the remote worker
                w.run(
                    lambda p: sys.path.remove(p), _mock_interactive_session_cwd
                )
                # Import the actual file after starting the module since the
                # worker is started using fork on Linux, which will inherits
                # the parent sys.modules. On Python>3.6, the worker can be
                # started using spawn using mp_context in ProcessPoolExectutor.
                # TODO Once Python 3.6 reaches end of life, rely on mp_context
                # instead.
                import mock_local_folder.mod as mod
                # The constructs whose pickling mechanism is changed using
                # register_pickle_by_value are functions, classes, TypeVar and
                # modules.
                from mock_local_folder.mod import (
                    local_function, LocalT, LocalClass
                )
    
                # Make sure the module/constructs are unimportable in the
                # worker.
                with pytest.raises(ImportError):
                    w.run(lambda: __import__("mock_local_folder.mod"))
                with pytest.raises(ImportError):
                    w.run(
                        lambda: __import__("mock_local_folder.subfolder.mod")
                    )
    
                for o in [mod, local_function, LocalT, LocalClass]:
                    with pytest.raises(ImportError):
                        w.run(lambda: o)
    
                register_pickle_by_value(mod)
                # function
                assert w.run(lambda: local_function()) == local_function()
                # typevar
>               assert w.run(lambda: LocalT.__name__) == LocalT.__name__
tests/cloudpickle_test.py:2583: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/testutils.py:146: in run
    input_payload = dumps((func, args, kwargs), protocol=self.protocol)
cloudpickle/cloudpickle_fast.py:73: in dumps
    cp.dump(obj)
cloudpickle/cloudpickle_fast.py:632: in dump
    return Pickler.dump(self, obj)
cloudpickle/cloudpickle.py:911: in _typevar_reduce
    return (_make_typevar, _decompose_typevar(obj))
cloudpickle/cloudpickle.py:899: in _decompose_typevar
    _get_or_create_tracker_id(obj),
cloudpickle/cloudpickle.py:107: in _get_or_create_tracker_id
    class_tracker_id = _DYNAMIC_CLASS_TRACKER_BY_CLASS.get(class_def)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <WeakKeyDictionary at 0xf6742fc0>, key = ~LocalT, default = None
    def get(self, key, default=None):
>       return self.data.get(ref(key),default)
E       TypeError: cannot create weak reference to 'typing.TypeVar' object
/usr/lib/python3.12/weakref.py:452: TypeError
_________________ CloudPickleTest.test_pickle_dynamic_typevar __________________
self = <tests.cloudpickle_test.CloudPickleTest testMethod=test_pickle_dynamic_typevar>
    def test_pickle_dynamic_typevar(self):
        T = typing.TypeVar('T')
>       depickled_T = pickle_depickle(T, protocol=self.protocol)
tests/cloudpickle_test.py:2330: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/cloudpickle_test.py:81: in pickle_depickle
    return pickle.loads(cloudpickle.dumps(obj, protocol=protocol))
cloudpickle/cloudpickle_fast.py:73: in dumps
    cp.dump(obj)
cloudpickle/cloudpickle_fast.py:632: in dump
    return Pickler.dump(self, obj)
cloudpickle/cloudpickle.py:909: in _typevar_reduce
    return (_make_typevar, _decompose_typevar(obj))
cloudpickle/cloudpickle.py:899: in _decompose_typevar
    _get_or_create_tracker_id(obj),
cloudpickle/cloudpickle.py:107: in _get_or_create_tracker_id
    class_tracker_id = _DYNAMIC_CLASS_TRACKER_BY_CLASS.get(class_def)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <WeakKeyDictionary at 0xf6742fc0>, key = ~T, default = None
    def get(self, key, default=None):
>       return self.data.get(ref(key),default)
E       TypeError: cannot create weak reference to 'typing.TypeVar' object
/usr/lib/python3.12/weakref.py:452: TypeError
___________ CloudPickleTest.test_pickle_dynamic_typevar_memoization ____________
self = <tests.cloudpickle_test.CloudPickleTest testMethod=test_pickle_dynamic_typevar_memoization>
    def test_pickle_dynamic_typevar_memoization(self):
        T = typing.TypeVar('T')
>       depickled_T1, depickled_T2 = pickle_depickle((T, T),
                                                     protocol=self.protocol)
tests/cloudpickle_test.py:2345: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/cloudpickle_test.py:81: in pickle_depickle
    return pickle.loads(cloudpickle.dumps(obj, protocol=protocol))
cloudpickle/cloudpickle_fast.py:73: in dumps
    cp.dump(obj)
cloudpickle/cloudpickle_fast.py:632: in dump
    return Pickler.dump(self, obj)
cloudpickle/cloudpickle.py:909: in _typevar_reduce
    return (_make_typevar, _decompose_typevar(obj))
cloudpickle/cloudpickle.py:899: in _decompose_typevar
    _get_or_create_tracker_id(obj),
cloudpickle/cloudpickle.py:107: in _get_or_create_tracker_id
    class_tracker_id = _DYNAMIC_CLASS_TRACKER_BY_CLASS.get(class_def)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <WeakKeyDictionary at 0xf6742fc0>, key = ~T, default = None
    def get(self, key, default=None):
>       return self.data.get(ref(key),default)
E       TypeError: cannot create weak reference to 'typing.TypeVar' object
/usr/lib/python3.12/weakref.py:452: TypeError
_____________ CloudPickleTest.test_pickle_dynamic_typevar_tracking _____________
self = <tests.cloudpickle_test.CloudPickleTest testMethod=test_pickle_dynamic_typevar_tracking>
    def test_pickle_dynamic_typevar_tracking(self):
        T = typing.TypeVar("T")
>       T2 = subprocess_pickle_echo(T, protocol=self.protocol)
tests/cloudpickle_test.py:2340: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/testutils.py:89: in subprocess_pickle_echo
    out = subprocess_pickle_string(input_data,
tests/testutils.py:63: in subprocess_pickle_string
    pickle_string = dumps(input_data, protocol=protocol)
cloudpickle/cloudpickle_fast.py:73: in dumps
    cp.dump(obj)
cloudpickle/cloudpickle_fast.py:632: in dump
    return Pickler.dump(self, obj)
cloudpickle/cloudpickle.py:909: in _typevar_reduce
    return (_make_typevar, _decompose_typevar(obj))
cloudpickle/cloudpickle.py:899: in _decompose_typevar
    _get_or_create_tracker_id(obj),
cloudpickle/cloudpickle.py:107: in _get_or_create_tracker_id
    class_tracker_id = _DYNAMIC_CLASS_TRACKER_BY_CLASS.get(class_def)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <WeakKeyDictionary at 0xf6742fc0>, key = ~T, default = None
    def get(self, key, default=None):
>       return self.data.get(ref(key),default)
E       TypeError: cannot create weak reference to 'typing.TypeVar' object
/usr/lib/python3.12/weakref.py:452: TypeError
________________ Protocol2CloudPickleTest.test_generic_subclass ________________
self = <tests.cloudpickle_test.Protocol2CloudPickleTest testMethod=test_generic_subclass>
    def test_generic_subclass(self):
        T = typing.TypeVar('T')
    
        class Base(typing.Generic[T]):
            pass
    
        class DerivedAny(Base):
            pass
    
        class LeafAny(DerivedAny):
            pass
    
        class DerivedInt(Base[int]):
            pass
    
        class LeafInt(DerivedInt):
            pass
    
        class DerivedT(Base[T]):
            pass
    
        class LeafT(DerivedT[T]):
            pass
    
        klasses = [
            Base, DerivedAny, LeafAny, DerivedInt, LeafInt, DerivedT, LeafT
        ]
        for klass in klasses:
>           assert pickle_depickle(klass, protocol=self.protocol) is klass
tests/cloudpickle_test.py:2423: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/cloudpickle_test.py:81: in pickle_depickle
    return pickle.loads(cloudpickle.dumps(obj, protocol=protocol))
cloudpickle/cloudpickle_fast.py:73: in dumps
    cp.dump(obj)
cloudpickle/cloudpickle_fast.py:632: in dump
    return Pickler.dump(self, obj)
cloudpickle/cloudpickle.py:909: in _typevar_reduce
    return (_make_typevar, _decompose_typevar(obj))
cloudpickle/cloudpickle.py:899: in _decompose_typevar
    _get_or_create_tracker_id(obj),
cloudpickle/cloudpickle.py:107: in _get_or_create_tracker_id
    class_tracker_id = _DYNAMIC_CLASS_TRACKER_BY_CLASS.get(class_def)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <WeakKeyDictionary at 0xf6742fc0>, key = ~T, default = None
    def get(self, key, default=None):
>       return self.data.get(ref(key),default)
E       TypeError: cannot create weak reference to 'typing.TypeVar' object
/usr/lib/python3.12/weakref.py:452: TypeError
__________________ Protocol2CloudPickleTest.test_generic_type __________________
self = <tests.cloudpickle_test.Protocol2CloudPickleTest testMethod=test_generic_type>
    def test_generic_type(self):
        T = typing.TypeVar('T')
    
        class C(typing.Generic[T]):
            pass
    
>       assert pickle_depickle(C, protocol=self.protocol) is C
tests/cloudpickle_test.py:2364: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/cloudpickle_test.py:81: in pickle_depickle
    return pickle.loads(cloudpickle.dumps(obj, protocol=protocol))
cloudpickle/cloudpickle_fast.py:73: in dumps
    cp.dump(obj)
cloudpickle/cloudpickle_fast.py:632: in dump
    return Pickler.dump(self, obj)
cloudpickle/cloudpickle.py:909: in _typevar_reduce
    return (_make_typevar, _decompose_typevar(obj))
cloudpickle/cloudpickle.py:899: in _decompose_typevar
    _get_or_create_tracker_id(obj),
cloudpickle/cloudpickle.py:107: in _get_or_create_tracker_id
    class_tracker_id = _DYNAMIC_CLASS_TRACKER_BY_CLASS.get(class_def)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <WeakKeyDictionary at 0xf6742fc0>, key = ~T, default = None
    def get(self, key, default=None):
>       return self.data.get(ref(key),default)
E       TypeError: cannot create weak reference to 'typing.TypeVar' object
/usr/lib/python3.12/weakref.py:452: TypeError
_____ Protocol2CloudPickleTest.test_locally_defined_class_with_type_hints ______
self = <tests.cloudpickle_test.Protocol2CloudPickleTest testMethod=test_locally_defined_class_with_type_hints>
    def test_locally_defined_class_with_type_hints(self):
        with subprocess_worker(protocol=self.protocol) as worker:
            for type_ in _all_types_to_test():
                class MyClass:
                    def method(self, arg: type_) -> type_:
                        return arg
                MyClass.__annotations__ = {'attribute': type_}
    
                def check_annotations(obj, expected_type, expected_type_str):
                    assert obj.__annotations__["attribute"] == expected_type
                    assert (
                        obj.method.__annotations__["arg"] == expected_type
                    )
                    assert (
                        obj.method.__annotations__["return"]
                        == expected_type
                    )
                    return "ok"
    
                obj = MyClass()
                assert check_annotations(obj, type_, "type_") == "ok"
>               assert (
                    worker.run(check_annotations, obj, type_, "type_") == "ok"
                )
tests/cloudpickle_test.py:2457: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/testutils.py:146: in run
    input_payload = dumps((func, args, kwargs), protocol=self.protocol)
cloudpickle/cloudpickle_fast.py:73: in dumps
    cp.dump(obj)
cloudpickle/cloudpickle_fast.py:632: in dump
    return Pickler.dump(self, obj)
cloudpickle/cloudpickle.py:909: in _typevar_reduce
    return (_make_typevar, _decompose_typevar(obj))
cloudpickle/cloudpickle.py:899: in _decompose_typevar
    _get_or_create_tracker_id(obj),
cloudpickle/cloudpickle.py:107: in _get_or_create_tracker_id
    class_tracker_id = _DYNAMIC_CLASS_TRACKER_BY_CLASS.get(class_def)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <WeakKeyDictionary at 0xf6742fc0>, key = ~T, default = None
    def get(self, key, default=None):
>       return self.data.get(ref(key),default)
E       TypeError: cannot create weak reference to 'typing.TypeVar' object
/usr/lib/python3.12/weakref.py:452: TypeError
_ Protocol2CloudPickleTest.test_pickle_constructs_from_module_registered_for_pickling_by_value _
self = <tests.cloudpickle_test.Protocol2CloudPickleTest testMethod=test_pickle_constructs_from_module_registered_for_pickling_by_value>
    def test_pickle_constructs_from_module_registered_for_pickling_by_value(self):  # noqa
        _prev_sys_path = sys.path.copy()
        try:
            # We simulate an interactive session that:
            # - we start from the /path/to/cloudpickle/tests directory, where a
            #   local .py file (mock_local_file) is located.
            # - uses constructs from mock_local_file in remote workers that do
            #   not have access to this file. This situation is
            #   the justification behind the
            #   (un)register_pickle_by_value(module) api that cloudpickle
            #   exposes.
            _mock_interactive_session_cwd = os.path.dirname(__file__)
    
            # First, remove sys.path entries that could point to
            # /path/to/cloudpickle/tests and be in inherited by the worker
            _maybe_remove(sys.path, '')
            _maybe_remove(sys.path, _mock_interactive_session_cwd)
    
            # Add the desired session working directory
            sys.path.insert(0, _mock_interactive_session_cwd)
    
            with subprocess_worker(protocol=self.protocol) as w:
                # Make the module unavailable in the remote worker
                w.run(
                    lambda p: sys.path.remove(p), _mock_interactive_session_cwd
                )
                # Import the actual file after starting the module since the
                # worker is started using fork on Linux, which will inherits
                # the parent sys.modules. On Python>3.6, the worker can be
                # started using spawn using mp_context in ProcessPoolExectutor.
                # TODO Once Python 3.6 reaches end of life, rely on mp_context
                # instead.
                import mock_local_folder.mod as mod
                # The constructs whose pickling mechanism is changed using
                # register_pickle_by_value are functions, classes, TypeVar and
                # modules.
                from mock_local_folder.mod import (
                    local_function, LocalT, LocalClass
                )
    
                # Make sure the module/constructs are unimportable in the
                # worker.
                with pytest.raises(ImportError):
                    w.run(lambda: __import__("mock_local_folder.mod"))
                with pytest.raises(ImportError):
                    w.run(
                        lambda: __import__("mock_local_folder.subfolder.mod")
                    )
    
                for o in [mod, local_function, LocalT, LocalClass]:
                    with pytest.raises(ImportError):
                        w.run(lambda: o)
    
                register_pickle_by_value(mod)
                # function
                assert w.run(lambda: local_function()) == local_function()
                # typevar
>               assert w.run(lambda: LocalT.__name__) == LocalT.__name__
tests/cloudpickle_test.py:2583: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/testutils.py:146: in run
    input_payload = dumps((func, args, kwargs), protocol=self.protocol)
cloudpickle/cloudpickle_fast.py:73: in dumps
    cp.dump(obj)
cloudpickle/cloudpickle_fast.py:632: in dump
    return Pickler.dump(self, obj)
cloudpickle/cloudpickle.py:911: in _typevar_reduce
    return (_make_typevar, _decompose_typevar(obj))
cloudpickle/cloudpickle.py:899: in _decompose_typevar
    _get_or_create_tracker_id(obj),
cloudpickle/cloudpickle.py:107: in _get_or_create_tracker_id
    class_tracker_id = _DYNAMIC_CLASS_TRACKER_BY_CLASS.get(class_def)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <WeakKeyDictionary at 0xf6742fc0>, key = ~LocalT, default = None
    def get(self, key, default=None):
>       return self.data.get(ref(key),default)
E       TypeError: cannot create weak reference to 'typing.TypeVar' object
/usr/lib/python3.12/weakref.py:452: TypeError
_____________ Protocol2CloudPickleTest.test_pickle_dynamic_typevar _____________
self = <tests.cloudpickle_test.Protocol2CloudPickleTest testMethod=test_pickle_dynamic_typevar>
    def test_pickle_dynamic_typevar(self):
        T = typing.TypeVar('T')
>       depickled_T = pickle_depickle(T, protocol=self.protocol)
tests/cloudpickle_test.py:2330: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/cloudpickle_test.py:81: in pickle_depickle
    return pickle.loads(cloudpickle.dumps(obj, protocol=protocol))
cloudpickle/cloudpickle_fast.py:73: in dumps
    cp.dump(obj)
cloudpickle/cloudpickle_fast.py:632: in dump
    return Pickler.dump(self, obj)
cloudpickle/cloudpickle.py:909: in _typevar_reduce
    return (_make_typevar, _decompose_typevar(obj))
cloudpickle/cloudpickle.py:899: in _decompose_typevar
    _get_or_create_tracker_id(obj),
cloudpickle/cloudpickle.py:107: in _get_or_create_tracker_id
    class_tracker_id = _DYNAMIC_CLASS_TRACKER_BY_CLASS.get(class_def)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <WeakKeyDictionary at 0xf6742fc0>, key = ~T, default = None
    def get(self, key, default=None):
>       return self.data.get(ref(key),default)
E       TypeError: cannot create weak reference to 'typing.TypeVar' object
/usr/lib/python3.12/weakref.py:452: TypeError
_______ Protocol2CloudPickleTest.test_pickle_dynamic_typevar_memoization _______
self = <tests.cloudpickle_test.Protocol2CloudPickleTest testMethod=test_pickle_dynamic_typevar_memoization>
    def test_pickle_dynamic_typevar_memoization(self):
        T = typing.TypeVar('T')
>       depickled_T1, depickled_T2 = pickle_depickle((T, T),
                                                     protocol=self.protocol)
tests/cloudpickle_test.py:2345: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/cloudpickle_test.py:81: in pickle_depickle
    return pickle.loads(cloudpickle.dumps(obj, protocol=protocol))
cloudpickle/cloudpickle_fast.py:73: in dumps
    cp.dump(obj)
cloudpickle/cloudpickle_fast.py:632: in dump
    return Pickler.dump(self, obj)
cloudpickle/cloudpickle.py:909: in _typevar_reduce
    return (_make_typevar, _decompose_typevar(obj))
cloudpickle/cloudpickle.py:899: in _decompose_typevar
    _get_or_create_tracker_id(obj),
cloudpickle/cloudpickle.py:107: in _get_or_create_tracker_id
    class_tracker_id = _DYNAMIC_CLASS_TRACKER_BY_CLASS.get(class_def)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <WeakKeyDictionary at 0xf6742fc0>, key = ~T, default = None
    def get(self, key, default=None):
>       return self.data.get(ref(key),default)
E       TypeError: cannot create weak reference to 'typing.TypeVar' object
/usr/lib/python3.12/weakref.py:452: TypeError
________ Protocol2CloudPickleTest.test_pickle_dynamic_typevar_tracking _________
self = <tests.cloudpickle_test.Protocol2CloudPickleTest testMethod=test_pickle_dynamic_typevar_tracking>
    def test_pickle_dynamic_typevar_tracking(self):
        T = typing.TypeVar("T")
>       T2 = subprocess_pickle_echo(T, protocol=self.protocol)
tests/cloudpickle_test.py:2340: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/testutils.py:89: in subprocess_pickle_echo
    out = subprocess_pickle_string(input_data,
tests/testutils.py:63: in subprocess_pickle_string
    pickle_string = dumps(input_data, protocol=protocol)
cloudpickle/cloudpickle_fast.py:73: in dumps
    cp.dump(obj)
cloudpickle/cloudpickle_fast.py:632: in dump
    return Pickler.dump(self, obj)
cloudpickle/cloudpickle.py:909: in _typevar_reduce
    return (_make_typevar, _decompose_typevar(obj))
cloudpickle/cloudpickle.py:899: in _decompose_typevar
    _get_or_create_tracker_id(obj),
cloudpickle/cloudpickle.py:107: in _get_or_create_tracker_id
    class_tracker_id = _DYNAMIC_CLASS_TRACKER_BY_CLASS.get(class_def)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <WeakKeyDictionary at 0xf6742fc0>, key = ~T, default = None
    def get(self, key, default=None):
>       return self.data.get(ref(key),default)
E       TypeError: cannot create weak reference to 'typing.TypeVar' object
/usr/lib/python3.12/weakref.py:452: TypeError
...
=========================== short test summary info ============================
FAILED tests/cloudpickle_test.py::CloudPickleTest::test_generic_subclass - Ty...
FAILED tests/cloudpickle_test.py::CloudPickleTest::test_generic_type - TypeEr...
FAILED tests/cloudpickle_test.py::CloudPickleTest::test_locally_defined_class_with_type_hints
FAILED tests/cloudpickle_test.py::CloudPickleTest::test_pickle_constructs_from_module_registered_for_pickling_by_value
FAILED tests/cloudpickle_test.py::CloudPickleTest::test_pickle_dynamic_typevar
FAILED tests/cloudpickle_test.py::CloudPickleTest::test_pickle_dynamic_typevar_memoization
FAILED tests/cloudpickle_test.py::CloudPickleTest::test_pickle_dynamic_typevar_tracking
FAILED tests/cloudpickle_test.py::Protocol2CloudPickleTest::test_generic_subclass
FAILED tests/cloudpickle_test.py::Protocol2CloudPickleTest::test_generic_type
FAILED tests/cloudpickle_test.py::Protocol2CloudPickleTest::test_locally_defined_class_with_type_hints
FAILED tests/cloudpickle_test.py::Protocol2CloudPickleTest::test_pickle_constructs_from_module_registered_for_pickling_by_value
FAILED tests/cloudpickle_test.py::Protocol2CloudPickleTest::test_pickle_dynamic_typevar
FAILED tests/cloudpickle_test.py::Protocol2CloudPickleTest::test_pickle_dynamic_typevar_memoization
FAILED tests/cloudpickle_test.py::Protocol2CloudPickleTest::test_pickle_dynamic_typevar_tracking
==== 14 failed, 226 passed, 15 skipped, 5 deselected, 12 warnings in 21.86s ====
Comment 1 Ben Beasley 2023-07-03 14:29:14 UTC 
Submitted upstream: https://github.com/cloudpipe/cloudpickle/issues/507
Comment 2 Ben Beasley 2023-07-03 14:45:18 UTC 
PR to work around distutils removal (the easy part): https://github.com/cloudpipe/cloudpickle/pull/508
Comment 3 Miro Hrončok 2023-07-03 18:10:32 UTC 
This is currently one of the biggest blockers for the Python 3.12 rebuild.

Also, Lumír is on vacation.
Comment 4 Ben Beasley 2023-07-03 18:28:55 UTC 
(In reply to Miro Hrončok from comment #3)
> This is currently one of the biggest blockers for the Python 3.12 rebuild.
> 
> Also, Lumír is on vacation.

Any idea what should be done about “TypeError: cannot create weak reference to 'typing.TypeVar' object”?
Comment 5 Miro Hrončok 2023-07-03 18:30:03 UTC 
Not yet. Will discus with Petr Viktorin ASAP.
Comment 6 Miro Hrončok 2023-07-04 10:13:26 UTC 
> Any idea what should be done about “TypeError: cannot create weak reference to 'typing.TypeVar' object”?

Apparently, a CPython regression: https://github.com/python/cpython/issues/106403

Will see if we skip the tests, if joblib and others build.
Comment 7 Miro Hrončok 2023-07-04 11:35:24 UTC 
Tests skipped. Leaving MODIFIED, we will want to unskip them later.
Comment 8 Fedora Release Engineering 2023-08-16 07:15:21 UTC 
This bug appears to have been reported against 'rawhide' during the Fedora Linux 39 development cycle.
Changing version to 39.
Comment 9 Fedora Update System 2023-09-07 10:51:13 UTC 
FEDORA-2023-6f2952f81d has been submitted as an update to Fedora 40. https://bodhi.fedoraproject.org/updates/FEDORA-2023-6f2952f81d
Comment 10 Fedora Update System 2023-09-07 10:52:34 UTC 
FEDORA-2023-6f2952f81d has been pushed to the Fedora 40 stable repository.
If problem still persists, please make note of it in this bug report.
Comment 11 Fedora Update System 2024-08-09 19:30:20 UTC 
FEDORA-EPEL-2024-c4b2008a49 (python-cloudpickle-3.0.0-8.el10_0) has been submitted as an update to Fedora EPEL 10.0.
https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2024-c4b2008a49
Comment 12 Fedora Update System 2024-08-09 19:33:13 UTC 
FEDORA-EPEL-2024-c4b2008a49 (python-cloudpickle-3.0.0-8.el10_0) has been pushed to the Fedora EPEL 10.0 stable repository.
If problem still persists, please make note of it in this bug report.