Bug 1904379
| Summary: | pillow aborts on Python 3.10: free(): invalid pointer | ||
|---|---|---|---|
| Product: | [Fedora] Fedora | Reporter: | Miro Hrončok <mhroncok> |
| Component: | python-pillow | Assignee: | Sandro Mani <manisandro> |
| Status: | CLOSED ERRATA | QA Contact: | Fedora Extras Quality Assurance <extras-qa> |
| Severity: | high | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | rawhide | CC: | eclipseo, hugovk+redhatbugzilla, loganjerry, manisandro, miminar, thrnciar, trpost |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | python-pillow-8.1.0-2.fc34 python-pillow-8.1.0-2.eln108 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2021-01-09 10:32:33 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | |||
| Bug Blocks: | 1890881, 1896390, 1898977, 1899498, 1907320, 1914338 | ||
|
Description
Miro Hrončok
2020-12-04 09:13:12 UTC
Sandro, could you please have a look? Valgrind says: ==117== Invalid free() / delete / delete[] / realloc() ==117== at 0x483D9F1: free (vg_replace_malloc.c:538) ==117== by 0x493CF01: _PyMem_RawFree (obmalloc.c:127) ==117== by 0x493CF01: PyMem_RawFree (obmalloc.c:595) ==117== by 0x493CF01: _PyObject_Free (obmalloc.c:1922) ==117== by 0x493CF01: _PyObject_Free (obmalloc.c:1913) ==117== by 0x4956999: _Py_Dealloc (object.c:2209) ==117== by 0x4956999: _Py_DECREF (object.h:448) ==117== by 0x4956999: _Py_XDECREF (object.h:514) ==117== by 0x4956999: tupledealloc (tupleobject.c:276) ==117== by 0x495AFF8: _Py_XDECREF (object.h:514) ==117== by 0x495AFF8: code_dealloc (codeobject.c:658) ==117== by 0x4957228: _PyDealloc (object.c:2209) ==117== by 0x4957228: _Py_DECREF (object.h:448) ==117== by 0x4957228: func_clear (funcobject.c:593) ==117== by 0x495708D: func_dealloc (funcobject.c:614) ==117== by 0x4947966: _Py_Dealloc (object.c:2209) ==117== by 0x4947966: _Py_DECREF (object.h:448) ==117== by 0x4947966: _Py_XDECREF (object.h:514) ==117== by 0x4947966: free_keys_object (dictobject.c:628) ==117== by 0x49EE0C9: type_clear (typeobject.c:3836) ==117== by 0x4941864: delete_garbage (gcmodule.c:1018) ==117== by 0x4941864: gc_collect_main (gcmodule.c:1301) ==117== by 0x49EFD9B: _PyGC_CollectNoFail.isra.0 (gcmodule.c:2090) ==117== by 0x49F2E74: interpreter_clear (pystate.c:323) ==117== by 0x49F19DA: _Py_InterpreterState_Clear (pystate.c:355) ==117== by 0x49F19DA: finalize_interp_clear (pylifecycle.c:1600) ==117== Address 0x4b67f40 is 0 bytes inside data symbol "_Py_TrueStruct" So the reference count of _Py_TrueStruct is reaching 0. The crash can be triggered by bypassing the python portions of pillow and going straight for the shared objects:
$ python3 -c 'from PIL import _imaging'
free(): invalid pointer
Aborted (core dumped)
$ python3 -c 'from PIL import _webp'
free(): invalid pointer
Aborted (core dumped)
Also, regarding the comment about it working in the interactive prompt, that's only true if you press Ctrl-D to exit. If you run quit() instead:
$ python3
Python 3.10.0a4 (default, Jan 4 2021, 00:00:00)
[GCC 11.0.0 20201223 (Red Hat 11.0.0-0)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from PIL import _imaging
>>> quit()
free(): invalid pointer
Aborted (core dumped)
What is strange is that running the test suite with pytest during the build doesn't trigger any bug.
Full Pytest summary:
+ PYTHONPATH=/builddir/python-pillow/Pillow-8.1.0:/builddir/build/BUILDROOT/python-pillow-8.1.0-2.fc34.x86_64/usr/lib64/python3.10/site-packages
+ /usr/bin/python3 -m pytest
============================= test session starts ==============================
platform linux -- Python 3.10.0a4, pytest-6.0.2, py-1.10.0, pluggy-0.13.1
rootdir: /builddir/python-pillow/Pillow-8.1.0, configfile: setup.cfg, testpaths: Tests
collected 1734 items / 1 skipped / 1733 selected
Tests/test_000_sanity.py . [ 0%]
Tests/test_binary.py ... [ 0%]
Tests/test_bmp_reference.py ... [ 0%]
Tests/test_box_blur.py .............. [ 1%]
Tests/test_color_lut.py .......................... [ 2%]
Tests/test_core_resources.py .............. [ 3%]
Tests/test_decompression_bomb.py ......... [ 4%]
Tests/test_features.py .............. [ 4%]
Tests/test_file_apng.py .................... [ 5%]
Tests/test_file_blp.py ... [ 6%]
Tests/test_file_bmp.py ........... [ 6%]
Tests/test_file_bufrstub.py .... [ 7%]
Tests/test_file_container.py .......... [ 7%]
Tests/test_file_cur.py .. [ 7%]
Tests/test_file_dcx.py ......... [ 8%]
Tests/test_file_dds.py .............. [ 9%]
Tests/test_file_eps.py ................ [ 9%]
Tests/test_file_fitsstub.py .... [ 10%]
Tests/test_file_fli.py .......... [ 10%]
Tests/test_file_fpx.py .. [ 10%]
Tests/test_file_ftex.py .. [ 11%]
Tests/test_file_gbr.py ... [ 11%]
Tests/test_file_gd.py ... [ 11%]
Tests/test_file_gif.py ..............ss................................. [ 14%]
.. [ 14%]
Tests/test_file_gimpgradient.py .......... [ 14%]
Tests/test_file_gimppalette.py .. [ 14%]
Tests/test_file_gribstub.py .... [ 15%]
Tests/test_file_hdf5stub.py .... [ 15%]
Tests/test_file_icns.py .sss..... [ 15%]
Tests/test_file_ico.py ......... [ 16%]
Tests/test_file_im.py ............ [ 17%]
Tests/test_file_iptc.py ..... [ 17%]
Tests/test_file_jpeg.py .....................................ss......... [ 20%]
............s [ 20%]
Tests/test_file_jpeg2k.py ......................... [ 22%]
Tests/test_file_libtiff.py ............................................. [ 25%]
............ [ 25%]
Tests/test_file_libtiff_small.py ... [ 25%]
Tests/test_file_mcidas.py .. [ 26%]
Tests/test_file_mic.py ...... [ 26%]
Tests/test_file_mpo.py ................. [ 27%]
Tests/test_file_msp.py ....ss. [ 27%]
Tests/test_file_palm.py .X.. [ 27%]
Tests/test_file_pcd.py . [ 28%]
Tests/test_file_pcx.py ............ [ 28%]
Tests/test_file_pdf.py ............. [ 29%]
Tests/test_file_pixar.py .. [ 29%]
Tests/test_file_png.py ............................................... [ 32%]
Tests/test_file_ppm.py ....... [ 32%]
Tests/test_file_psd.py ............. [ 33%]
Tests/test_file_sgi.py .......... [ 34%]
Tests/test_file_spider.py ................ [ 34%]
Tests/test_file_sun.py ..s [ 35%]
Tests/test_file_tar.py .... [ 35%]
Tests/test_file_tga.py ......... [ 35%]
Tests/test_file_tiff.py ................................................ [ 38%]
.....ss [ 39%]
Tests/test_file_tiff_metadata.py ................. [ 40%]
Tests/test_file_wal.py . [ 40%]
Tests/test_file_webp.py ............ [ 40%]
Tests/test_file_webp_alpha.py .... [ 41%]
Tests/test_file_webp_animated.py ...... [ 41%]
Tests/test_file_webp_lossless.py . [ 41%]
Tests/test_file_webp_metadata.py ....... [ 41%]
Tests/test_file_wmf.py ..... [ 42%]
Tests/test_file_xbm.py ..... [ 42%]
Tests/test_file_xpm.py ... [ 42%]
Tests/test_file_xvthumb.py ... [ 42%]
Tests/test_font_bdf.py .. [ 42%]
Tests/test_font_leaks.py .. [ 42%]
Tests/test_font_pcf.py ..... [ 43%]
Tests/test_font_pcf_charsets.py ......... [ 43%]
Tests/test_format_hsv.py .... [ 44%]
Tests/test_format_lab.py ... [ 44%]
Tests/test_image.py .................................................... [ 47%]
............. [ 47%]
Tests/test_image_access.py ..............................s [ 49%]
Tests/test_image_array.py .. [ 49%]
Tests/test_image_convert.py ................ [ 50%]
Tests/test_image_copy.py .. [ 50%]
Tests/test_image_crop.py ...... [ 51%]
Tests/test_image_draft.py ... [ 51%]
Tests/test_image_entropy.py . [ 51%]
Tests/test_image_filter.py ......... [ 51%]
Tests/test_image_frombytes.py . [ 52%]
Tests/test_image_fromqimage.py ..... [ 52%]
Tests/test_image_getbands.py . [ 52%]
Tests/test_image_getbbox.py .. [ 52%]
Tests/test_image_getcolors.py .. [ 52%]
Tests/test_image_getdata.py .. [ 52%]
Tests/test_image_getextrema.py .. [ 52%]
Tests/test_image_getim.py . [ 52%]
Tests/test_image_getpalette.py . [ 52%]
Tests/test_image_getprojection.py . [ 52%]
Tests/test_image_histogram.py . [ 53%]
Tests/test_image_load.py ..... [ 53%]
Tests/test_image_mode.py .. [ 53%]
Tests/test_image_paste.py ........... [ 54%]
Tests/test_image_point.py .... [ 54%]
Tests/test_image_putalpha.py ... [ 54%]
Tests/test_image_putdata.py ....... [ 54%]
Tests/test_image_putpalette.py ... [ 55%]
Tests/test_image_quantize.py ....... [ 55%]
Tests/test_image_reduce.py ............ [ 56%]
Tests/test_image_resample.py .................xx................. [ 58%]
Tests/test_image_resize.py ............... [ 59%]
Tests/test_image_rotate.py .............. [ 59%]
Tests/test_image_split.py ... [ 60%]
Tests/test_image_thumbnail.py ........ [ 60%]
Tests/test_image_tobitmap.py . [ 60%]
Tests/test_image_tobytes.py . [ 60%]
Tests/test_image_transform.py ................................... [ 62%]
Tests/test_image_transpose.py ........ [ 63%]
Tests/test_imagechops.py .............................. [ 64%]
Tests/test_imagecms.py ............................ [ 66%]
Tests/test_imagecolor.py .... [ 66%]
Tests/test_imagedraw.py ................................................ [ 69%]
............s............................ [ 71%]
Tests/test_imagedraw2.py ................ [ 72%]
Tests/test_imageenhance.py ... [ 72%]
Tests/test_imagefile.py ................. [ 73%]
Tests/test_imagefont.py ................................................ [ 76%]
........................................................................ [ 80%]
................................ [ 82%]
Tests/test_imagefontctl.py ......................................ss..... [ 85%]
................................... [ 87%]
Tests/test_imagegrab.py sss..ss [ 87%]
Tests/test_imagemath.py ...................... [ 88%]
Tests/test_imagemorph.py .................. [ 90%]
Tests/test_imageops.py ............. [ 90%]
Tests/test_imageops_usm.py ..... [ 91%]
Tests/test_imagepalette.py ........... [ 91%]
Tests/test_imagepath.py ........... [ 92%]
Tests/test_imageqt.py .. [ 92%]
Tests/test_imagesequence.py ........ [ 92%]
Tests/test_imageshow.py ....s.. [ 93%]
Tests/test_imagestat.py ... [ 93%]
Tests/test_imagetk.py ssss [ 93%]
Tests/test_imagewin.py ...sssss [ 94%]
Tests/test_lib_image.py . [ 94%]
Tests/test_lib_pack.py .................................... [ 96%]
Tests/test_locale.py s [ 96%]
Tests/test_main.py . [ 96%]
Tests/test_map.py .. [ 96%]
Tests/test_mode_i16.py ... [ 96%]
Tests/test_numpy.py ............. [ 97%]
Tests/test_pdfparser.py .... [ 97%]
Tests/test_pickle.py ............... [ 98%]
Tests/test_psdraw.py .. [ 98%]
Tests/test_qt_image_toqimage.py . [ 98%]
Tests/test_sgi_crash.py .... [ 98%]
Tests/test_shell_injection.py ssss [ 99%]
Tests/test_tiff_crashes.py ss. [ 99%]
Tests/test_tiff_ifdrational.py .... [ 99%]
Tests/test_util.py ...... [ 99%]
Tests/test_webp_leaks.py . [100%]
=============================== warnings summary ===============================
Tests/test_numpy.py::test_numpy_to_image
/builddir/python-pillow/Pillow-8.1.0/Tests/test_numpy.py:34: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. Use `bool` by itself, which is identical in behavior, to silence this warning. If you specifically wanted the numpy scalar type, use `np.bool_` here.
assert_image(to_image(numpy.bool, 1, 1), "1", TEST_IMAGE_SIZE)
Tests/test_numpy.py::test_numpy_to_image
/builddir/python-pillow/Pillow-8.1.0/Tests/test_numpy.py:68: DeprecationWarning: `np.float` is a deprecated alias for the builtin `float`. Use `float` by itself, which is identical in behavior, to silence this warning. If you specifically wanted the numpy scalar type, use `np.float_` here.
assert_image(to_image(numpy.float), "F", TEST_IMAGE_SIZE)
Tests/test_numpy.py::test_roundtrip_eye
/builddir/python-pillow/Pillow-8.1.0/Tests/test_numpy.py:194: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. Use `bool` by itself, which is identical in behavior, to silence this warning. If you specifically wanted the numpy scalar type, use `np.bool_` here.
numpy.bool,
Tests/test_numpy.py::test_roundtrip_eye
/builddir/python-pillow/Pillow-8.1.0/Tests/test_numpy.py:202: DeprecationWarning: `np.float` is a deprecated alias for the builtin `float`. Use `float` by itself, which is identical in behavior, to silence this warning. If you specifically wanted the numpy scalar type, use `np.float_` here.
numpy.float,
Tests/test_numpy.py::test_bool
/builddir/python-pillow/Pillow-8.1.0/Tests/test_numpy.py:221: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. Use `bool` by itself, which is identical in behavior, to silence this warning. If you specifically wanted the numpy scalar type, use `np.bool_` here.
a = numpy.zeros((10, 2), dtype=numpy.bool)
-- Docs: https://docs.pytest.org/en/stable/warnings.html
=========================== short test summary info ============================
SKIPPED [1] Tests/test_pyroma.py:5: Pyroma not installed
SKIPPED [1] Tests/test_file_gif.py:222: Netpbm not available
SKIPPED [1] Tests/test_file_gif.py:233: Netpbm not available
SKIPPED [1] Tests/test_file_icns.py:29: Requires macOS
SKIPPED [1] Tests/test_file_icns.py:42: Requires macOS
SKIPPED [1] Tests/test_file_icns.py:59: Requires macOS
SKIPPED [1] Tests/test_file_jpeg.py:572: djpeg not available
SKIPPED [1] Tests/test_file_jpeg.py:578: cjpeg not available
SKIPPED [1] Tests/test_file_jpeg.py:804: Windows only
SKIPPED [1] Tests/test_file_msp.py:59: Extra image files not installed
SKIPPED [1] Tests/test_file_msp.py:73: Even More Extra image files not installed
SKIPPED [1] Tests/test_file_sun.py:36: Extra image files not installed
SKIPPED [1] Tests/test_file_tiff.py:608: Extra image files not installed
SKIPPED [1] Tests/test_file_tiff.py:621: Windows only
SKIPPED [1] Tests/test_image_access.py:364: Failing on AppVeyor / GitHub Actions when run from subprocess, not from shell
SKIPPED [1] Tests/test_imagedraw.py:867: failing
SKIPPED [2] Tests/test_imagefontctl.py:252: fails with this font
SKIPPED [1] Tests/test_imagegrab.py:13: requires Windows or macOS
SKIPPED [1] Tests/test_imagegrab.py:37: X connection failed: error 5
SKIPPED [1] Tests/test_imagegrab.py:39: tests missing XCB
SKIPPED [1] Tests/test_imagegrab.py:77: Windows only
SKIPPED [1] Tests/test_imagegrab.py:87: Windows only
SKIPPED [1] Tests/test_imageshow.py:44: Only run on CIs; hangs on Windows CIs
SKIPPED [4] Tests/test_imagetk.py:30: TCL Error: no display name and no $DISPLAY environment variable
SKIPPED [1] Tests/test_imagewin.py:37: Windows only
SKIPPED [1] Tests/test_imagewin.py:47: Windows only
SKIPPED [1] Tests/test_imagewin.py:58: Windows only
SKIPPED [1] Tests/test_imagewin.py:72: Windows only
SKIPPED [1] Tests/test_imagewin.py:87: Windows only
SKIPPED [1] Tests/test_locale.py:31: Polish locale not available
SKIPPED [1] Tests/test_shell_injection.py:25: djpeg not available
SKIPPED [1] Tests/test_shell_injection.py:34: cjpeg not available
SKIPPED [1] Tests/test_shell_injection.py:39: Netpbm not available
SKIPPED [1] Tests/test_shell_injection.py:45: Netpbm not available
SKIPPED [2] Tests/test_tiff_crashes.py:37: test image not found
XFAIL Tests/test_image_resample.py::TestCoreResampleAlphaCorrect::test_levels_rgba
Current implementation isn't precise enough
XFAIL Tests/test_image_resample.py::TestCoreResampleAlphaCorrect::test_levels_la
Current implementation isn't precise enough
XPASS Tests/test_file_palm.py::test_p_mode Palm P image is wrong
===== 1692 passed, 40 skipped, 2 xfailed, 1 xpassed, 5 warnings in 56.26s ======
So I went downstream https://github.com/python-pillow/Pillow/issues/5193 And they fixed the issue in https://github.com/python-pillow/Pillow/pull/5194 I have tested it with the Python 3.10 repo: $ python3 -c 'from PIL import _imaging' $ python3 -c 'from PIL import _webp' $ python3 -c 'from PIL import Image' All are working. I have then tested to build my dependent package Bubblemail and the process went through without any issue. I am sorry I committed the fix directly to dist-git, I thought I was in my fork: https://src.fedoraproject.org/rpms/python-pillow/c/7cd707c485f814af7dda741bc9a275269d3fc765?branch=master The SPEC should be cleaned up though (globbing all of sitearch is now forbidden for example), and run the test suite from upstream with Pytest. FEDORA-2021-f77bb74f01 has been pushed to the Fedora 34 stable repository. If problem still persists, please make note of it in this bug report. Thanks for taking care of this! Awesome! Thanks all. FEDORA-2021-a43527895b has been pushed to the Fedora ELN stable repository. If problem still persists, please make note of it in this bug report. *** Bug 1896390 has been marked as a duplicate of this bug. *** *** Bug 1899498 has been marked as a duplicate of this bug. *** *** Bug 1907320 has been marked as a duplicate of this bug. *** *** Bug 1914338 has been marked as a duplicate of this bug. *** *** Bug 1898977 has been marked as a duplicate of this bug. *** |