Bug 2093921 - python-cloudpickle fails to build with Python 3.11: NameError: name 'g129' is not defined
Summary: python-cloudpickle fails to build with Python 3.11: NameError: name 'g129' is...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: python-cloudpickle
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Lumír Balhar
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 2098867 (view as bug list)
Depends On:
Blocks: PYTHON3.11 F37FTBFS F37FailsToInstall
TreeView+ depends on / blocked
 
Reported: 2022-06-06 11:44 UTC by Tomáš Hrnčiar
Modified: 2022-06-20 12:21 UTC (History)
3 users (show)

Fixed In Version: python-cloudpickle-2.1.0-3.fc37
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-06-20 12:21:43 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Tomáš Hrnčiar 2022-06-06 11:44:22 UTC
python-cloudpickle fails to build with Python 3.11.0b3.


=================================== FAILURES ===================================
______________________ CloudPickleTest.test_extended_arg _______________________

self = <tests.cloudpickle_test.CloudPickleTest testMethod=test_extended_arg>

    @pytest.mark.skipif(
        (3, 11, 0, 'beta') <= sys.version_info < (3, 11, 0, 'beta', 2),
        reason="https://github.com/python/cpython/issues/92932"
    )
    def test_extended_arg(self):
        # Functions with more than 65535 global vars prefix some global
        # variable references with the EXTENDED_ARG opcode.
        nvars = 65537 + 258
        names = ['g%d' % i for i in range(1, nvars)]
        r = random.Random(42)
        d = {name: r.randrange(100) for name in names}
        # def f(x):
        #     x = g1, g2, ...
        #     return zlib.crc32(bytes(bytearray(x)))
        code = """
        import zlib
    
        def f():
            x = {tup}
            return zlib.crc32(bytes(bytearray(x)))
        """.format(tup=', '.join(names))
        exec(textwrap.dedent(code), d, d)
        f = d['f']
        res = f()
        data = cloudpickle.dumps([f, f], protocol=self.protocol)
        d = f = None
        f2, f3 = pickle.loads(data)
        self.assertTrue(f2 is f3)
>       self.assertEqual(f2(), res)

tests/cloudpickle_test.py:1009: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

>   ???
E   NameError: name 'g129' is not defined

<string>:5: NameError
__________________ Protocol2CloudPickleTest.test_extended_arg __________________

self = <tests.cloudpickle_test.Protocol2CloudPickleTest testMethod=test_extended_arg>

    @pytest.mark.skipif(
        (3, 11, 0, 'beta') <= sys.version_info < (3, 11, 0, 'beta', 2),
        reason="https://github.com/python/cpython/issues/92932"
    )
    def test_extended_arg(self):
        # Functions with more than 65535 global vars prefix some global
        # variable references with the EXTENDED_ARG opcode.
        nvars = 65537 + 258
        names = ['g%d' % i for i in range(1, nvars)]
        r = random.Random(42)
        d = {name: r.randrange(100) for name in names}
        # def f(x):
        #     x = g1, g2, ...
        #     return zlib.crc32(bytes(bytearray(x)))
        code = """
        import zlib
    
        def f():
            x = {tup}
            return zlib.crc32(bytes(bytearray(x)))
        """.format(tup=', '.join(names))
        exec(textwrap.dedent(code), d, d)
        f = d['f']
        res = f()
        data = cloudpickle.dumps([f, f], protocol=self.protocol)
        d = f = None
        f2, f3 = pickle.loads(data)
        self.assertTrue(f2 is f3)
>       self.assertEqual(f2(), res)

tests/cloudpickle_test.py:1009: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

>   ???
E   NameError: name 'g129' is not defined

<string>:5: NameError
=============================== warnings summary ===============================
tests/cloudpickle_test.py::CloudPickleTest::test_tornado_coroutine
tests/cloudpickle_test.py::CloudPickleTest::test_tornado_coroutine
tests/cloudpickle_test.py::Protocol2CloudPickleTest::test_tornado_coroutine
  /usr/lib64/python3.11/site-packages/tornado/ioloop.py:263: DeprecationWarning: There is no current event loop
    loop = asyncio.get_event_loop()

tests/cloudpickle_test.py::CloudPickleTest::test_tornado_coroutine
  /usr/lib64/python3.11/site-packages/tornado/platform/asyncio.py:279: DeprecationWarning: There is no current event loop
    super().initialize(asyncio.get_event_loop(), **kwargs)

tests/cloudpickle_test.py::CloudPickleTest::test_tornado_coroutine
tests/cloudpickle_test.py::Protocol2CloudPickleTest::test_tornado_coroutine
  /usr/lib64/python3.11/site-packages/tornado/platform/asyncio.py:193: DeprecationWarning: There is no current event loop
    old_loop = asyncio.get_event_loop()

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED tests/cloudpickle_test.py::CloudPickleTest::test_extended_arg - NameEr...
FAILED tests/cloudpickle_test.py::Protocol2CloudPickleTest::test_extended_arg
===== 2 failed, 238 passed, 13 skipped, 3 deselected, 6 warnings in 9.59s ======

https://docs.python.org/3.11/whatsnew/3.11.html

For the build logs, see:
https://copr-be.cloud.fedoraproject.org/results/@python/python3.11/fedora-rawhide-x86_64/04495191-python-cloudpickle/

For all our attempts to build python-cloudpickle with Python 3.11, see:
https://copr.fedorainfracloud.org/coprs/g/python/python3.11/package/python-cloudpickle/

Testing and mass rebuild of packages is happening in copr. You can follow these instructions to test locally in mock if your package builds with Python 3.11:
https://copr.fedorainfracloud.org/coprs/g/python/python3.11/

Let us know here if you have any questions.

Python 3.11 is planned to be included in Fedora 37. To make that update smoother, we're building Fedora packages with all pre-releases of Python 3.11.
A build failure prevents us from testing all dependent packages (transitive [Build]Requires), so if this package is required a lot, it's important for us to get it fixed soon.
We'd appreciate help from the people who know this package best, but if you don't want to work on this now, let us know so we can try to work around it on our side.

Comment 1 Lumír Balhar 2022-06-07 10:40:15 UTC
It seems that this is a known problem. The test is skipped for 3.11 beta [0] and the fix is ready in Python [1] but it's not included in the latest release so the skip condition has to be adjusted slightly.

[0] https://github.com/cloudpipe/cloudpickle/blob/8bbea3e140767f51dd935a3c8f21c9a8e8702b7c/tests/cloudpickle_test.py#L981-L984
[1] https://github.com/python/cpython/issues/92932

Comment 2 Lumír Balhar 2022-06-07 10:42:45 UTC
Proposed simple fix: https://github.com/cloudpipe/cloudpickle/pull/474

Comment 3 Lumír Balhar 2022-06-07 11:21:51 UTC
The fix is so simple that I've managed to do it wrong, here is a better one: https://github.com/cloudpipe/cloudpickle/pull/475

Comment 4 Miro Hrončok 2022-06-20 10:17:42 UTC
*** Bug 2098867 has been marked as a duplicate of this bug. ***

Comment 5 Fedora Update System 2022-06-20 12:18:46 UTC
FEDORA-2022-ba041525f0 has been submitted as an update to Fedora 37. https://bodhi.fedoraproject.org/updates/FEDORA-2022-ba041525f0

Comment 6 Fedora Update System 2022-06-20 12:21:43 UTC
FEDORA-2022-ba041525f0 has been pushed to the Fedora 37 stable repository.
If problem still persists, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.