Bug 1904379
Summary: | pillow aborts on Python 3.10: free(): invalid pointer | ||
---|---|---|---|
Product: | [Fedora] Fedora | Reporter: | Miro Hrončok <mhroncok> |
Component: | python-pillow | Assignee: | Sandro Mani <manisandro> |
Status: | CLOSED ERRATA | QA Contact: | Fedora Extras Quality Assurance <extras-qa> |
Severity: | high | Docs Contact: | |
Priority: | unspecified | ||
Version: | rawhide | CC: | hugovk+redhatbugzilla, loganjerry, manisandro, miminar, thrnciar, trpost, zebob.m |
Target Milestone: | --- | ||
Target Release: | --- | ||
Hardware: | Unspecified | ||
OS: | Unspecified | ||
Whiteboard: | |||
Fixed In Version: | python-pillow-8.1.0-2.fc34 python-pillow-8.1.0-2.eln108 | Doc Type: | If docs needed, set a value |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | 2021-01-09 10:32:33 UTC | Type: | Bug |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: | |||
Bug Depends On: | |||
Bug Blocks: | 1890881, 1896390, 1898977, 1899498, 1907320, 1914338 |
Description
Miro Hrončok
2020-12-04 09:13:12 UTC
Sandro, could you please have a look? Valgrind says: ==117== Invalid free() / delete / delete[] / realloc() ==117== at 0x483D9F1: free (vg_replace_malloc.c:538) ==117== by 0x493CF01: _PyMem_RawFree (obmalloc.c:127) ==117== by 0x493CF01: PyMem_RawFree (obmalloc.c:595) ==117== by 0x493CF01: _PyObject_Free (obmalloc.c:1922) ==117== by 0x493CF01: _PyObject_Free (obmalloc.c:1913) ==117== by 0x4956999: _Py_Dealloc (object.c:2209) ==117== by 0x4956999: _Py_DECREF (object.h:448) ==117== by 0x4956999: _Py_XDECREF (object.h:514) ==117== by 0x4956999: tupledealloc (tupleobject.c:276) ==117== by 0x495AFF8: _Py_XDECREF (object.h:514) ==117== by 0x495AFF8: code_dealloc (codeobject.c:658) ==117== by 0x4957228: _PyDealloc (object.c:2209) ==117== by 0x4957228: _Py_DECREF (object.h:448) ==117== by 0x4957228: func_clear (funcobject.c:593) ==117== by 0x495708D: func_dealloc (funcobject.c:614) ==117== by 0x4947966: _Py_Dealloc (object.c:2209) ==117== by 0x4947966: _Py_DECREF (object.h:448) ==117== by 0x4947966: _Py_XDECREF (object.h:514) ==117== by 0x4947966: free_keys_object (dictobject.c:628) ==117== by 0x49EE0C9: type_clear (typeobject.c:3836) ==117== by 0x4941864: delete_garbage (gcmodule.c:1018) ==117== by 0x4941864: gc_collect_main (gcmodule.c:1301) ==117== by 0x49EFD9B: _PyGC_CollectNoFail.isra.0 (gcmodule.c:2090) ==117== by 0x49F2E74: interpreter_clear (pystate.c:323) ==117== by 0x49F19DA: _Py_InterpreterState_Clear (pystate.c:355) ==117== by 0x49F19DA: finalize_interp_clear (pylifecycle.c:1600) ==117== Address 0x4b67f40 is 0 bytes inside data symbol "_Py_TrueStruct" So the reference count of _Py_TrueStruct is reaching 0. The crash can be triggered by bypassing the python portions of pillow and going straight for the shared objects:
$ python3 -c 'from PIL import _imaging'
free(): invalid pointer
Aborted (core dumped)
$ python3 -c 'from PIL import _webp'
free(): invalid pointer
Aborted (core dumped)
Also, regarding the comment about it working in the interactive prompt, that's only true if you press Ctrl-D to exit. If you run quit() instead:
$ python3
Python 3.10.0a4 (default, Jan 4 2021, 00:00:00)
[GCC 11.0.0 20201223 (Red Hat 11.0.0-0)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from PIL import _imaging
>>> quit()
free(): invalid pointer
Aborted (core dumped)
What is strange is that running the test suite with pytest during the build doesn't trigger any bug. Full Pytest summary: + PYTHONPATH=/builddir/python-pillow/Pillow-8.1.0:/builddir/build/BUILDROOT/python-pillow-8.1.0-2.fc34.x86_64/usr/lib64/python3.10/site-packages + /usr/bin/python3 -m pytest ============================= test session starts ============================== platform linux -- Python 3.10.0a4, pytest-6.0.2, py-1.10.0, pluggy-0.13.1 rootdir: /builddir/python-pillow/Pillow-8.1.0, configfile: setup.cfg, testpaths: Tests collected 1734 items / 1 skipped / 1733 selected Tests/test_000_sanity.py . [ 0%] Tests/test_binary.py ... [ 0%] Tests/test_bmp_reference.py ... [ 0%] Tests/test_box_blur.py .............. [ 1%] Tests/test_color_lut.py .......................... [ 2%] Tests/test_core_resources.py .............. [ 3%] Tests/test_decompression_bomb.py ......... [ 4%] Tests/test_features.py .............. [ 4%] Tests/test_file_apng.py .................... [ 5%] Tests/test_file_blp.py ... [ 6%] Tests/test_file_bmp.py ........... [ 6%] Tests/test_file_bufrstub.py .... [ 7%] Tests/test_file_container.py .......... [ 7%] Tests/test_file_cur.py .. [ 7%] Tests/test_file_dcx.py ......... [ 8%] Tests/test_file_dds.py .............. [ 9%] Tests/test_file_eps.py ................ [ 9%] Tests/test_file_fitsstub.py .... [ 10%] Tests/test_file_fli.py .......... [ 10%] Tests/test_file_fpx.py .. [ 10%] Tests/test_file_ftex.py .. [ 11%] Tests/test_file_gbr.py ... [ 11%] Tests/test_file_gd.py ... [ 11%] Tests/test_file_gif.py ..............ss................................. [ 14%] .. [ 14%] Tests/test_file_gimpgradient.py .......... [ 14%] Tests/test_file_gimppalette.py .. [ 14%] Tests/test_file_gribstub.py .... [ 15%] Tests/test_file_hdf5stub.py .... [ 15%] Tests/test_file_icns.py .sss..... [ 15%] Tests/test_file_ico.py ......... [ 16%] Tests/test_file_im.py ............ [ 17%] Tests/test_file_iptc.py ..... [ 17%] Tests/test_file_jpeg.py .....................................ss......... [ 20%] ............s [ 20%] Tests/test_file_jpeg2k.py ......................... [ 22%] Tests/test_file_libtiff.py ............................................. [ 25%] ............ [ 25%] Tests/test_file_libtiff_small.py ... [ 25%] Tests/test_file_mcidas.py .. [ 26%] Tests/test_file_mic.py ...... [ 26%] Tests/test_file_mpo.py ................. [ 27%] Tests/test_file_msp.py ....ss. [ 27%] Tests/test_file_palm.py .X.. [ 27%] Tests/test_file_pcd.py . [ 28%] Tests/test_file_pcx.py ............ [ 28%] Tests/test_file_pdf.py ............. [ 29%] Tests/test_file_pixar.py .. [ 29%] Tests/test_file_png.py ............................................... [ 32%] Tests/test_file_ppm.py ....... [ 32%] Tests/test_file_psd.py ............. [ 33%] Tests/test_file_sgi.py .......... [ 34%] Tests/test_file_spider.py ................ [ 34%] Tests/test_file_sun.py ..s [ 35%] Tests/test_file_tar.py .... [ 35%] Tests/test_file_tga.py ......... [ 35%] Tests/test_file_tiff.py ................................................ [ 38%] .....ss [ 39%] Tests/test_file_tiff_metadata.py ................. [ 40%] Tests/test_file_wal.py . [ 40%] Tests/test_file_webp.py ............ [ 40%] Tests/test_file_webp_alpha.py .... [ 41%] Tests/test_file_webp_animated.py ...... [ 41%] Tests/test_file_webp_lossless.py . [ 41%] Tests/test_file_webp_metadata.py ....... [ 41%] Tests/test_file_wmf.py ..... [ 42%] Tests/test_file_xbm.py ..... [ 42%] Tests/test_file_xpm.py ... [ 42%] Tests/test_file_xvthumb.py ... [ 42%] Tests/test_font_bdf.py .. [ 42%] Tests/test_font_leaks.py .. [ 42%] Tests/test_font_pcf.py ..... [ 43%] Tests/test_font_pcf_charsets.py ......... [ 43%] Tests/test_format_hsv.py .... [ 44%] Tests/test_format_lab.py ... [ 44%] Tests/test_image.py .................................................... [ 47%] ............. [ 47%] Tests/test_image_access.py ..............................s [ 49%] Tests/test_image_array.py .. [ 49%] Tests/test_image_convert.py ................ [ 50%] Tests/test_image_copy.py .. [ 50%] Tests/test_image_crop.py ...... [ 51%] Tests/test_image_draft.py ... [ 51%] Tests/test_image_entropy.py . [ 51%] Tests/test_image_filter.py ......... [ 51%] Tests/test_image_frombytes.py . [ 52%] Tests/test_image_fromqimage.py ..... [ 52%] Tests/test_image_getbands.py . [ 52%] Tests/test_image_getbbox.py .. [ 52%] Tests/test_image_getcolors.py .. [ 52%] Tests/test_image_getdata.py .. [ 52%] Tests/test_image_getextrema.py .. [ 52%] Tests/test_image_getim.py . [ 52%] Tests/test_image_getpalette.py . [ 52%] Tests/test_image_getprojection.py . [ 52%] Tests/test_image_histogram.py . [ 53%] Tests/test_image_load.py ..... [ 53%] Tests/test_image_mode.py .. [ 53%] Tests/test_image_paste.py ........... [ 54%] Tests/test_image_point.py .... [ 54%] Tests/test_image_putalpha.py ... [ 54%] Tests/test_image_putdata.py ....... [ 54%] Tests/test_image_putpalette.py ... [ 55%] Tests/test_image_quantize.py ....... [ 55%] Tests/test_image_reduce.py ............ [ 56%] Tests/test_image_resample.py .................xx................. [ 58%] Tests/test_image_resize.py ............... [ 59%] Tests/test_image_rotate.py .............. [ 59%] Tests/test_image_split.py ... [ 60%] Tests/test_image_thumbnail.py ........ [ 60%] Tests/test_image_tobitmap.py . [ 60%] Tests/test_image_tobytes.py . [ 60%] Tests/test_image_transform.py ................................... [ 62%] Tests/test_image_transpose.py ........ [ 63%] Tests/test_imagechops.py .............................. [ 64%] Tests/test_imagecms.py ............................ [ 66%] Tests/test_imagecolor.py .... [ 66%] Tests/test_imagedraw.py ................................................ [ 69%] ............s............................ [ 71%] Tests/test_imagedraw2.py ................ [ 72%] Tests/test_imageenhance.py ... [ 72%] Tests/test_imagefile.py ................. [ 73%] Tests/test_imagefont.py ................................................ [ 76%] ........................................................................ [ 80%] ................................ [ 82%] Tests/test_imagefontctl.py ......................................ss..... [ 85%] ................................... [ 87%] Tests/test_imagegrab.py sss..ss [ 87%] Tests/test_imagemath.py ...................... [ 88%] Tests/test_imagemorph.py .................. [ 90%] Tests/test_imageops.py ............. [ 90%] Tests/test_imageops_usm.py ..... [ 91%] Tests/test_imagepalette.py ........... [ 91%] Tests/test_imagepath.py ........... [ 92%] Tests/test_imageqt.py .. [ 92%] Tests/test_imagesequence.py ........ [ 92%] Tests/test_imageshow.py ....s.. [ 93%] Tests/test_imagestat.py ... [ 93%] Tests/test_imagetk.py ssss [ 93%] Tests/test_imagewin.py ...sssss [ 94%] Tests/test_lib_image.py . [ 94%] Tests/test_lib_pack.py .................................... [ 96%] Tests/test_locale.py s [ 96%] Tests/test_main.py . [ 96%] Tests/test_map.py .. [ 96%] Tests/test_mode_i16.py ... [ 96%] Tests/test_numpy.py ............. [ 97%] Tests/test_pdfparser.py .... [ 97%] Tests/test_pickle.py ............... [ 98%] Tests/test_psdraw.py .. [ 98%] Tests/test_qt_image_toqimage.py . [ 98%] Tests/test_sgi_crash.py .... [ 98%] Tests/test_shell_injection.py ssss [ 99%] Tests/test_tiff_crashes.py ss. [ 99%] Tests/test_tiff_ifdrational.py .... [ 99%] Tests/test_util.py ...... [ 99%] Tests/test_webp_leaks.py . [100%] =============================== warnings summary =============================== Tests/test_numpy.py::test_numpy_to_image /builddir/python-pillow/Pillow-8.1.0/Tests/test_numpy.py:34: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. Use `bool` by itself, which is identical in behavior, to silence this warning. If you specifically wanted the numpy scalar type, use `np.bool_` here. assert_image(to_image(numpy.bool, 1, 1), "1", TEST_IMAGE_SIZE) Tests/test_numpy.py::test_numpy_to_image /builddir/python-pillow/Pillow-8.1.0/Tests/test_numpy.py:68: DeprecationWarning: `np.float` is a deprecated alias for the builtin `float`. Use `float` by itself, which is identical in behavior, to silence this warning. If you specifically wanted the numpy scalar type, use `np.float_` here. assert_image(to_image(numpy.float), "F", TEST_IMAGE_SIZE) Tests/test_numpy.py::test_roundtrip_eye /builddir/python-pillow/Pillow-8.1.0/Tests/test_numpy.py:194: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. Use `bool` by itself, which is identical in behavior, to silence this warning. If you specifically wanted the numpy scalar type, use `np.bool_` here. numpy.bool, Tests/test_numpy.py::test_roundtrip_eye /builddir/python-pillow/Pillow-8.1.0/Tests/test_numpy.py:202: DeprecationWarning: `np.float` is a deprecated alias for the builtin `float`. Use `float` by itself, which is identical in behavior, to silence this warning. If you specifically wanted the numpy scalar type, use `np.float_` here. numpy.float, Tests/test_numpy.py::test_bool /builddir/python-pillow/Pillow-8.1.0/Tests/test_numpy.py:221: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. Use `bool` by itself, which is identical in behavior, to silence this warning. If you specifically wanted the numpy scalar type, use `np.bool_` here. a = numpy.zeros((10, 2), dtype=numpy.bool) -- Docs: https://docs.pytest.org/en/stable/warnings.html =========================== short test summary info ============================ SKIPPED [1] Tests/test_pyroma.py:5: Pyroma not installed SKIPPED [1] Tests/test_file_gif.py:222: Netpbm not available SKIPPED [1] Tests/test_file_gif.py:233: Netpbm not available SKIPPED [1] Tests/test_file_icns.py:29: Requires macOS SKIPPED [1] Tests/test_file_icns.py:42: Requires macOS SKIPPED [1] Tests/test_file_icns.py:59: Requires macOS SKIPPED [1] Tests/test_file_jpeg.py:572: djpeg not available SKIPPED [1] Tests/test_file_jpeg.py:578: cjpeg not available SKIPPED [1] Tests/test_file_jpeg.py:804: Windows only SKIPPED [1] Tests/test_file_msp.py:59: Extra image files not installed SKIPPED [1] Tests/test_file_msp.py:73: Even More Extra image files not installed SKIPPED [1] Tests/test_file_sun.py:36: Extra image files not installed SKIPPED [1] Tests/test_file_tiff.py:608: Extra image files not installed SKIPPED [1] Tests/test_file_tiff.py:621: Windows only SKIPPED [1] Tests/test_image_access.py:364: Failing on AppVeyor / GitHub Actions when run from subprocess, not from shell SKIPPED [1] Tests/test_imagedraw.py:867: failing SKIPPED [2] Tests/test_imagefontctl.py:252: fails with this font SKIPPED [1] Tests/test_imagegrab.py:13: requires Windows or macOS SKIPPED [1] Tests/test_imagegrab.py:37: X connection failed: error 5 SKIPPED [1] Tests/test_imagegrab.py:39: tests missing XCB SKIPPED [1] Tests/test_imagegrab.py:77: Windows only SKIPPED [1] Tests/test_imagegrab.py:87: Windows only SKIPPED [1] Tests/test_imageshow.py:44: Only run on CIs; hangs on Windows CIs SKIPPED [4] Tests/test_imagetk.py:30: TCL Error: no display name and no $DISPLAY environment variable SKIPPED [1] Tests/test_imagewin.py:37: Windows only SKIPPED [1] Tests/test_imagewin.py:47: Windows only SKIPPED [1] Tests/test_imagewin.py:58: Windows only SKIPPED [1] Tests/test_imagewin.py:72: Windows only SKIPPED [1] Tests/test_imagewin.py:87: Windows only SKIPPED [1] Tests/test_locale.py:31: Polish locale not available SKIPPED [1] Tests/test_shell_injection.py:25: djpeg not available SKIPPED [1] Tests/test_shell_injection.py:34: cjpeg not available SKIPPED [1] Tests/test_shell_injection.py:39: Netpbm not available SKIPPED [1] Tests/test_shell_injection.py:45: Netpbm not available SKIPPED [2] Tests/test_tiff_crashes.py:37: test image not found XFAIL Tests/test_image_resample.py::TestCoreResampleAlphaCorrect::test_levels_rgba Current implementation isn't precise enough XFAIL Tests/test_image_resample.py::TestCoreResampleAlphaCorrect::test_levels_la Current implementation isn't precise enough XPASS Tests/test_file_palm.py::test_p_mode Palm P image is wrong ===== 1692 passed, 40 skipped, 2 xfailed, 1 xpassed, 5 warnings in 56.26s ====== So I went downstream https://github.com/python-pillow/Pillow/issues/5193 And they fixed the issue in https://github.com/python-pillow/Pillow/pull/5194 I have tested it with the Python 3.10 repo: $ python3 -c 'from PIL import _imaging' $ python3 -c 'from PIL import _webp' $ python3 -c 'from PIL import Image' All are working. I have then tested to build my dependent package Bubblemail and the process went through without any issue. I am sorry I committed the fix directly to dist-git, I thought I was in my fork: https://src.fedoraproject.org/rpms/python-pillow/c/7cd707c485f814af7dda741bc9a275269d3fc765?branch=master The SPEC should be cleaned up though (globbing all of sitearch is now forbidden for example), and run the test suite from upstream with Pytest. FEDORA-2021-f77bb74f01 has been pushed to the Fedora 34 stable repository. If problem still persists, please make note of it in this bug report. Thanks for taking care of this! Awesome! Thanks all. FEDORA-2021-a43527895b has been pushed to the Fedora ELN stable repository. If problem still persists, please make note of it in this bug report. *** Bug 1896390 has been marked as a duplicate of this bug. *** *** Bug 1899498 has been marked as a duplicate of this bug. *** *** Bug 1907320 has been marked as a duplicate of this bug. *** *** Bug 1914338 has been marked as a duplicate of this bug. *** *** Bug 1898977 has been marked as a duplicate of this bug. *** |