Bug 1959006
Summary: | python-lz4 fails to build with Python 3.10: MemoryError in tests | ||
---|---|---|---|
Product: | [Fedora] Fedora | Reporter: | Miro Hrončok <mhroncok> |
Component: | python-lz4 | Assignee: | Miro Hrončok <mhroncok> |
Status: | CLOSED RAWHIDE | QA Contact: | Fedora Extras Quality Assurance <extras-qa> |
Severity: | unspecified | Docs Contact: | |
Priority: | high | ||
Version: | rawhide | CC: | jonathan.underwood, josdekloe, mhroncok, orion, thrnciar, zbyszek |
Target Milestone: | --- | ||
Target Release: | --- | ||
Hardware: | Unspecified | ||
OS: | Unspecified | ||
Whiteboard: | |||
Fixed In Version: | Doc Type: | If docs needed, set a value | |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | 2021-06-12 23:27:47 UTC | Type: | Bug |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: | |||
Bug Depends On: | |||
Bug Blocks: | 1890881, 1927309, 1927313, 1927646, 1959011, 1968973, 1968984 |
Description
Miro Hrončok
2021-05-10 14:49:36 UTC
This is a mass-posted update. Sorry if it is not 100% accurate to this bugzilla. The Python 3.10 rebuild is in progress in a Koji side tag. If you manage to fix the problem, please commit the fix in the rawhide branch, but don't build the package in regular rawhide. You can either build the package in the side tag, with: $ fedpkg build --target=f35-python Or you can the build and we will eventually build it for you. Note that the rebuild is still in progress, so not all (build) dependencies of this package might be available right away. Thanks. See also https://fedoraproject.org/wiki/Changes/Python3.10 If you have general questions about the rebuild, please use this mailing list thread: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/G47SGOYIQLRDTWGOSLSWERZSSHXDEDH5/ The f35-python side tag has been merged to Rawhide. From now on, build as you would normally build. *** Bug 1969020 has been marked as a duplicate of this bug. *** This is blocking a whole chain of packages, hence raising priority. might be related to this upstream issue? https://github.com/python-lz4/python-lz4/issues/219 Likely. Let's see: https://src.fedoraproject.org/rpms/python-lz4/pull-request/2 ppc64le still has MemoryError :/ =================================== FAILURES =================================== __________________ test_invalid_config_d_4[store_comp_size2] ___________________ store_comp_size = {'store_comp_size': 4} def test_invalid_config_d_4(store_comp_size): d_kwargs = {} d_kwargs['strategy'] = "double_buffer" d_kwargs['buffer_size'] = 1 << (8 * store_comp_size['store_comp_size']) d_kwargs.update(store_comp_size) if store_comp_size['store_comp_size'] >= 4: if os.environ.get('TRAVIS') is not None: pytest.skip('Skipping test on Travis due to insufficient memory') if sys.maxsize < 0xffffffff: pytest.skip('Py_ssize_t too small for this test') if psutil.virtual_memory().available < 4 * d_kwargs['buffer_size']: # The internal LZ4 context will request at least 3 times buffer_size # as memory (2 buffer_size for the double-buffer, and 1.x buffer_size # for the output buffer), so round up to 4 buffer_size. pytest.skip('Insufficient system memory for this test') # Make sure the page size is larger than what the input bound will be, # but still fit in 4 bytes d_kwargs['buffer_size'] -= 1 # No failure expected during instanciation/initialization > lz4.stream.LZ4StreamDecompressor(**d_kwargs) d_kwargs = {'buffer_size': 4294967295, 'store_comp_size': 4, 'strategy': 'double_buffer'} store_comp_size = {'store_comp_size': 4} tests/stream/test_stream_1.py:199: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <lz4.stream.LZ4StreamDecompressor object at 0x7ffe344a7bb0> strategy = 'double_buffer', buffer_size = 4294967295, return_bytearray = 0 store_comp_size = 4, dictionary = '' def __init__(self, strategy, buffer_size, return_bytearray=False, store_comp_size=4, dictionary=""): """ Instantiates and initializes a LZ4 stream decompression context. Args: strategy (str): Buffer management strategy. Can be: ``double_buffer``. buffer_size (int): Size of one buffer of the double-buffer used internally for stream decompression in the case of ``double_buffer`` strategy. Keyword Args: return_bytearray (bool): If ``False`` (the default) then the function will return a ``bytes`` object. If ``True``, then the function will return a ``bytearray`` object. store_comp_size (int): Specify the size in bytes of the following compressed block. Can be: ``1``, ``2`` or ``4`` (default: ``4``). dictionary (str, bytes or buffer-compatible object): If specified, perform decompression using this initial dictionary. Raises: Exceptions occuring during the context initialization. OverflowError: raised if the ``dictionary`` parameter is too large for the LZ4 context. ValueError: raised if some parameters are invalid. MemoryError: raised if some internal resources cannot be allocated. RuntimeError: raised if some internal resources cannot be initialized. """ return_bytearray = 1 if return_bytearray else 0 > self._context = _create_context(strategy, "decompress", buffer_size, return_bytearray=return_bytearray, store_comp_size=store_comp_size, dictionary=dictionary) E MemoryError: Could not allocate output buffer buffer_size = 4294967295 dictionary = '' return_bytearray = 0 self = <lz4.stream.LZ4StreamDecompressor object at 0x7ffe344a7bb0> store_comp_size = 4 strategy = 'double_buffer' lz4/stream/__init__.py:45: MemoryError I think we are OK to skip this test. It is also skipped on Travis, etc. I see some arches have tests disabled entirely. Running a fully tested scratch build to see what to skip on what arches. I've got the same MemoryError on aarch64 and even on x86_64. Hence not ppc64le specific. Skipping it appears to suceed the tests on all Little Endian architectures. Trying to run at least some tests on s390x now, but they were already skipped entirely there so if I won't succeed, I'll keep it that way. |