+++ This bug was initially created as a clone of Bug #1665459 +++ Description of problem: The documentation of python xmlsec module https://github.com/mehcode/python-xmlsec/blob/master/doc/source/examples.rst#sign shows that it should work just fine with lxml. However, when xmlsec module is imported in python code (not even used), attempt to parse second xml file with lxml's etree.parse results in segfault. Version-Release number of selected component (if applicable): python3-xmlsec-1.3.3-3.fc29.x86_64 python3-lxml-4.2.5-1.fc29.x86_64 How reproducible: Deterministic. Steps to Reproduce: 1. cat > sf.xml <data/> Ctrl-D here 2. cat > sf.py from lxml import etree import xmlsec xml1 = etree.parse("sf.xml") print(xml1) xml2 = etree.parse("sf.xml") print(xml2) Ctrl-D here 3. python3 sf.py Actual results: <lxml.etree._ElementTree object at 0x7f39227536c8> Traceback (most recent call last): File "sf.py", line 5, in <module> xml2 = etree.parse("sf.xml") File "src/lxml/etree.pyx", line 3426, in lxml.etree.parse File "src/lxml/parser.pxi", line 1840, in lxml.etree._parseDocument File "src/lxml/parser.pxi", line 1866, in lxml.etree._parseDocumentFromURL File "src/lxml/parser.pxi", line 1770, in lxml.etree._parseDocFromFile File "src/lxml/parser.pxi", line 1163, in lxml.etree._BaseParser._parseDocFromFile File "src/lxml/parser.pxi", line 601, in lxml.etree._ParserContext._handleParseResultDoc File "src/lxml/parser.pxi", line 711, in lxml.etree._handleParseResult File "src/lxml/parser.pxi", line 651, in lxml.etree._raiseParseError Segmentation fault (core dumped) Expected results: <lxml.etree._ElementTree object at 0x7f246c117c08> <lxml.etree._ElementTree object at 0x7f246c117c88> Additional info: When the "import xmlsec" line is commented out, I get the expected result. This used to work on Fedora 27 with python3-lxml-4.1.1-1.fc27.x86_64 python3-xmlsec-1.3.3-1.fc27.x86_64 It started to fail on Fedora 28 with python3-lxml-4.2.5-1.fc28.x86_64 python3-xmlsec-1.3.3-1.fc28.x86_64 and fails up to rawhide. --- Additional comment from Jan Pazdziora on 2019-01-11 14:30:30 CET --- Backtrace is: Core was generated by `python3 sf.py'. Program terminated with signal SIGSEGV, Segmentation fault. #0 find_maxchar_surrogates (num_surrogates=<synthetic pointer>, maxchar=<synthetic pointer>, end=0xfffffffffffffffd <error: Cannot access memory at address 0xfffffffffffffffd>, begin=0x1 <error: Cannot access memory at address 0x1>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/unicodeobject.c:1660 1660 ch = *iter; Missing separate debuginfos, use: dnf debuginfo-install libgpg-error-1.31-2.fc29.x86_64 libtool-ltdl-2.4.6-25.fc29.x86_64 libxml2-2.9.8-4.fc29.x86_64 libxslt-1.1.32-3.fc29.x86_64 openssl-libs-1.1.1-0.pre9.3.fc29.x86_64 python3-xmlsec-1.3.3-3.fc29.x86_64 xmlsec1-openssl-1.2.25-5.fc29.x86_64 xz-libs-5.2.4-3.fc29.x86_64 zlib-1.2.11-10.fc29.x86_64 (gdb) bt #0 find_maxchar_surrogates (num_surrogates=<synthetic pointer>, maxchar=<synthetic pointer>, end=0xfffffffffffffffd <error: Cannot access memory at address 0xfffffffffffffffd>, begin=0x1 <error: Cannot access memory at address 0x1>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/unicodeobject.c:1660 #1 _PyUnicode_Ready (unicode=b'sg.xml') at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/unicodeobject.c:1699 #2 0x00007f7f28965651 in unicode_fromformat_write_str (precision=-1, width=-1, str=<optimized out>, writer=0x7ffd3b6ecab0) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/unicodeobject.c:2909 #3 unicode_fromformat_arg (vargs=0x7ffd3b6ecb10, f=0x7f7f28b34964 "U\", line %d\n", writer=0x7ffd3b6ecab0) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/unicodeobject.c:2801 #4 PyUnicode_FromFormatV (format=<optimized out>, vargs=<optimized out>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/unicodeobject.c:2908 #5 0x00007f7f289ffd09 in PyUnicode_FromFormat ( format=format@entry=0x7f7f28b3495b " File \"%U\", line %d\n") at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/unicodeobject.c:2960 #6 0x00007f7f28b176f1 in print_exception (value=None, f=<_io.TextIOWrapper at remote 0x7f7f28248708>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Python/pythonrun.c:753 #7 print_exception_recursive (f=<_io.TextIOWrapper at remote 0x7f7f28248708>, value=<XMLSyntaxError(error_log=<lxml.etree._ListErrorLog at remote 0x7f7f279380e8>, code=1) at remote 0x7f7f27999d38>, seen=<optimized out>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Python/pythonrun.c:901 #8 0x00007f7f28b19eeb in PyErr_Display (exception=<optimized out>, value=<XMLSyntaxError(error_log=<lxml.etree._ListErrorLog at remote 0x7f7f279380e8>, code=1) at remote 0x7f7f27999d38>, tb=<optimized out>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Python/pythonrun.c:935 #9 0x00007f7f28b19f6c in sys_excepthook ( self=self@entry=<module at remote 0x7f7f2828ef98>, args=args@entry=(<type at remote 0x56025b2e1208>, <XMLSyntaxError(error_log=<lxml.etree._ListErrorLog at remote 0x7f7f279380e8>, code=1) at remote 0x7f7f27999d38>, <traceback at remote 0x7f7f27935688>)) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Python/sysmodule.c:321 #10 0x00007f7f28a24ef3 in _PyMethodDef_RawFastCallDict ( method=<optimized out>, self=<optimized out>, args=<optimized out>, nargs=<optimized out>, kwargs=<optimized out>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/call.c:518 #11 0x00007f7f28a25150 in _PyCFunction_FastCallDict ( func=<built-in method excepthook of module object at remote 0x7f7f2828ef98>, args=<optimized out>, nargs=<optimized out>, kwargs=<optimized out>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/call.c:582 #12 0x00007f7f28b2d73e in PyErr_PrintEx (set_sys_last_vars=1) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Python/pythonrun.c:679 #13 PyErr_PrintEx (set_sys_last_vars=<optimized out>, set_sys_last_vars=<optimized out>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Python/pythonrun.c:641 #14 0x00007f7f289cd3a3 in PyRun_SimpleFileExFlags (fp=<optimized out>, filename=<optimized out>, closeit=<optimized out>, flags=0x7ffd3b6ece90) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Python/pythonrun.c:435 #15 0x00007f7f28b2f7f1 in pymain_run_file (p_cf=0x7ffd3b6ece90, filename=0x56025b210df0 L"sf.py", fp=0x56025b272ef0) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Modules/main.c:427 #16 pymain_run_filename (cf=0x7ffd3b6ece90, pymain=0x7ffd3b6ecfa0) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Modules/main.c:1627 #17 pymain_run_python (pymain=0x7ffd3b6ecfa0) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Modules/main.c:2876 #18 pymain_main (pymain=<optimized out>, pymain=<optimized out>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Modules/main.c:3037 #19 0x00007f7f28b2fa2c in _Py_UnixMain (argc=<optimized out>, argv=<optimized out>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Modules/main.c:3072 #20 0x00007f7f2859d413 in __libc_start_main (main=0x56025a8fc050 <main>, argc=2, argv=0x7ffd3b6ed0f8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffd3b6ed0e8) at ../csu/libc-start.c:308 #21 0x000056025a8fc08e in _start () --- Additional comment from Jan Pazdziora on 2019-01-11 15:25:27 CET --- Building python3-xmlsec from https://github.com/mehcode/python-xmlsec master (1.3.6) does not seem to improve the situation. --- Additional comment from Jan Pazdziora on 2019-01-11 16:04:48 CET --- Here by full, from different coredump though: (gdb) bt full #0 find_maxchar_surrogates (num_surrogates=<synthetic pointer>, maxchar=<synthetic pointer>, end=0xfffffffffffffffd <error: Cannot access memory at address 0xfffffffffffffffd>, begin=0x1 <error: Cannot access memory at address 0x1>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/unicodeobject.c:1660 iter = 0x1 <error: Cannot access memory at address 0x1> ch = <optimized out> iter = <optimized out> ch = <optimized out> #1 _PyUnicode_Ready (unicode=b'sf.xml') at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/unicodeobject.c:1699 end = 0xfffffffffffffffd <error: Cannot access memory at address 0xfffffffffffffffd> maxchar = 0 num_surrogates = 0 #2 0x00007fbac0b08651 in unicode_fromformat_write_str (precision=-1, width=-1, str=<optimized out>, writer=0x7ffd1669c160) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/unicodeobject.c:2909 length = <optimized out> fill = <optimized out> arglen = <optimized out> maxchar = <optimized out> length = <optimized out> fill = <optimized out> arglen = <optimized out> maxchar = <optimized out> #3 unicode_fromformat_arg (vargs=0x7ffd1669c1c0, f=0x7fbac0cd7964 "U\", line %d\n", writer=0x7ffd1669c160) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/unicodeobject.c:2801 obj = b'sf.xml' len = <optimized out> zeropad = <optimized out> longflag = <optimized out> longlongflag = <optimized out> size_tflag = <optimized out> p = <optimized out> width = <optimized out> precision = -1 fill = <optimized out> p = <optimized out> len = <optimized out> zeropad = <optimized out> width = <optimized out> precision = <optimized out> longflag = <optimized out> longlongflag = <optimized out> size_tflag = <optimized out> fill = <optimized out> ordinal = <optimized out> buffer = <optimized out> arglen = <optimized out> fillchar = <optimized out> number = <optimized out> s = <optimized out> obj = <optimized out> obj = <optimized out> str = <optimized out> obj = <optimized out> str = <optimized out> _py_decref_tmp = <optimized out> _py_decref_tmp = <optimized out> obj = <optimized out> repr = <optimized out> _py_decref_tmp = <optimized out> _py_decref_tmp = <optimized out> obj = <optimized out> ascii = <optimized out> _py_decref_tmp = <optimized out> _py_decref_tmp = <optimized out> #4 PyUnicode_FromFormatV (format=<optimized out>, vargs=<optimized out>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/unicodeobject.c:2908 vargs2 = {{gp_offset = 16, fp_offset = 48, overflow_arg_area = 0x7ffd1669c300, reg_save_area = 0x7ffd1669c240}} f = <optimized out> writer = { buffer = ' File "\x01\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00w\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00ÿÿÿÿÿÿÿÿ\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x10\x00\x00\x00\x00\x00\x00ÿ\x0f\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', data = 0x7fbabfbbeab0, kind = PyUnicode_1BYTE_KIND, maxchar = 127, size = 121, pos = 8, min_length = 121, min_char = 127, overallocate = 1 '\001', readonly = 0 '\000'} #5 0x00007fbac0ba2d09 in PyUnicode_FromFormat (format=format@entry=0x7fbac0cd795b " File \"%U\", line %d\n") at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/unicodeobject.c:2960 ret = <optimized out> vargs = {{gp_offset = 8, fp_offset = 48, overflow_arg_area = 0x7ffd1669c300, reg_save_area = 0x7ffd1669c240}} #6 0x00007fbac0cba6f1 in print_exception (value=None, f=<_io.TextIOWrapper at remote 0x7fbac03ec708>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Python/pythonrun.c:753 line = <optimized out> message = None filename = b'sf.xml' text = 0x0 lineno = 0 offset = -1 err = <optimized out> type = <type at remote 0x560bb3d57208> tb = <traceback at remote 0x7fbabfae0608> err = <optimized out> type = <optimized out> tb = <optimized out> PyId_print_file_and_line = <optimized out> message = <optimized out> filename = <optimized out> text = <optimized out> lineno = <optimized out> offset = <optimized out> line = <optimized out> _py_decref_tmp = <optimized out> _py_decref_tmp = <optimized out> _py_decref_tmp = <optimized out> _py_decref_tmp = <optimized out> --Type <RET> for more, q to quit, c to continue without paging--c moduleName = <optimized out> className = <optimized out> PyId___module__ = <optimized out> dot = <optimized out> _py_xdecref_tmp = <optimized out> _py_decref_tmp = <optimized out> _py_decref_tmp = <optimized out> s = <optimized out> _py_xdecref_tmp = <optimized out> _py_decref_tmp = <optimized out> _py_xdecref_tmp = <optimized out> _py_decref_tmp = <optimized out> _py_decref_tmp = <optimized out> #7 print_exception_recursive (f=<_io.TextIOWrapper at remote 0x7fbac03ec708>, value=<XMLSyntaxError(error_log=<lxml.etree._ListErrorLog at remote 0x7fbabfae11d8>, code=1) at remote 0x7fbabfb43d38>, seen=<optimized out>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Python/pythonrun.c:901 err = 0 res = <optimized out> cause = <optimized out> context = <optimized out> #8 0x00007fbac0cbceeb in PyErr_Display (exception=<optimized out>, value=<XMLSyntaxError(error_log=<lxml.etree._ListErrorLog at remote 0x7fbabfae11d8>, code=1) at remote 0x7fbabfb43d38>, tb=<optimized out>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Python/pythonrun.c:935 seen = {140440056905016} f = <_io.TextIOWrapper at remote 0x7fbac03ec708> #9 0x00007fbac0cbcf6c in sys_excepthook (self=self@entry=<module at remote 0x7fbac0432f98>, args=args@entry=(<type at remote 0x560bb3d57208>, <XMLSyntaxError(error_log=<lxml.etree._ListErrorLog at remote 0x7fbabfae11d8>, code=1) at remote 0x7fbabfb43d38>, <traceback at remote 0x7fbabfae0608>)) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Python/sysmodule.c:321 exc = <type at remote 0x560bb3d57208> value = <XMLSyntaxError(error_log=<lxml.etree._ListErrorLog at remote 0x7fbabfae11d8>, code=1) at remote 0x7fbabfb43d38> tb = <traceback at remote 0x7fbabfae0608> #10 0x00007fbac0bc7ef3 in _PyMethodDef_RawFastCallDict (method=<optimized out>, self=<optimized out>, args=<optimized out>, nargs=<optimized out>, kwargs=<optimized out>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/call.c:518 argstuple = (<type at remote 0x560bb3d57208>, <XMLSyntaxError(error_log=<lxml.etree._ListErrorLog at remote 0x7fbabfae11d8>, code=1) at remote 0x7fbabfb43d38>, <traceback at remote 0x7fbabfae0608>) meth = <optimized out> flags = 1 result = 0x0 #11 0x00007fbac0bc8150 in _PyCFunction_FastCallDict (func=<built-in method excepthook of module object at remote 0x7fbac0432f98>, args=<optimized out>, nargs=<optimized out>, kwargs=<optimized out>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/call.c:582 result = <optimized out> #12 0x00007fbac0cd073e in PyErr_PrintEx (set_sys_last_vars=1) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Python/pythonrun.c:679 stack = {<type at remote 0x560bb3d57208>, <XMLSyntaxError(error_log=<lxml.etree._ListErrorLog at remote 0x7fbabfae11d8>, code=1) at remote 0x7fbabfb43d38>, <traceback at remote 0x7fbabfae0608>} result = <optimized out> exception = <type at remote 0x560bb3d57208> v = <XMLSyntaxError(error_log=<lxml.etree._ListErrorLog at remote 0x7fbabfae11d8>, code=1) at remote 0x7fbabfb43d38> tb = <traceback at remote 0x7fbabfae0608> hook = <optimized out> exception = <optimized out> v = <optimized out> tb = <optimized out> hook = <optimized out> stack = <optimized out> result = <optimized out> exception2 = <optimized out> v2 = <optimized out> tb2 = <optimized out> _py_decref_tmp = <optimized out> _py_decref_tmp = <optimized out> _py_xdecref_tmp = <optimized out> _py_decref_tmp = <optimized out> _py_xdecref_tmp = <optimized out> _py_decref_tmp = <optimized out> _py_xdecref_tmp = <optimized out> _py_decref_tmp = <optimized out> _py_xdecref_tmp = <optimized out> _py_decref_tmp = <optimized out> _py_xdecref_tmp = <optimized out> _py_decref_tmp = <optimized out> #13 PyErr_PrintEx (set_sys_last_vars=<optimized out>, set_sys_last_vars=<optimized out>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Python/pythonrun.c:641 exception = <optimized out> v = <optimized out> tb = <optimized out> hook = <optimized out> _py_decref_tmp = <optimized out> _py_decref_tmp = <optimized out> _py_decref_tmp = <optimized out> _py_decref_tmp = <optimized out> _py_xdecref_tmp = <optimized out> _py_decref_tmp = <optimized out> _py_xdecref_tmp = <optimized out> _py_xdecref_tmp = <optimized out> _py_xdecref_tmp = <optimized out> _py_xdecref_tmp = <optimized out> _py_decref_tmp = <optimized out> _py_decref_tmp = <optimized out> exception2 = <optimized out> v2 = <optimized out> tb2 = <optimized out> stack = <optimized out> result = <optimized out> exception = <optimized out> v = <optimized out> tb = <optimized out> hook = <optimized out> _py_xdecref_tmp = <optimized out> _py_xdecref_tmp = <optimized out> _py_xdecref_tmp = <optimized out> _py_xdecref_tmp = <optimized out> _py_decref_tmp = <optimized out> _py_decref_tmp = <optimized out> exception2 = <optimized out> v2 = <optimized out> tb2 = <optimized out> stack = <optimized out> result = <optimized out> _py_xdecref_tmp = <optimized out> _py_decref_tmp = <optimized out> _py_xdecref_tmp = <optimized out> _py_decref_tmp = <optimized out> _py_xdecref_tmp = <optimized out> _py_decref_tmp = <optimized out> #14 0x00007fbac0b703a3 in PyRun_SimpleFileExFlags (fp=<optimized out>, filename=<optimized out>, closeit=<optimized out>, flags=0x7ffd1669c540) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Python/pythonrun.c:435 m = 0x0 d = {'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <SourceFileLoader(name='__main__', path='sf.py') at remote 0x7fbac03385c0>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7fbac0438c28>, '__file__': 'sf.py', '__cached__': None, 'etree': <module at remote 0x7fbac02b1688>, 'xmlsec': <module at remote 0x7fbabfad4228>, 'xml1': <lxml.etree._ElementTree at remote 0x7fbabfae0388>} v = <optimized out> ext = <optimized out> set_file_name = 1 ret = -1 len = <optimized out> _py_decref_tmp = <optimized out> #15 0x00007fbac0cd27f1 in pymain_run_file (p_cf=0x7ffd1669c540, filename=0x560bb3c86df0 L"sf.py", fp=0x560bb3ce8ef0) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Modules/main.c:427 unicode = <optimized out> bytes = b'sf.py' filename_str = <optimized out> run = <optimized out> unicode = <optimized out> bytes = <optimized out> filename_str = <optimized out> run = <optimized out> _py_decref_tmp = <optimized out> _py_xdecref_tmp = <optimized out> _py_decref_tmp = <optimized out> #16 pymain_run_filename (cf=0x7ffd1669c540, pymain=0x7ffd1669c650) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Modules/main.c:1627 fp = 0x560bb3ce8ef0 fp = <optimized out> #17 pymain_run_python (pymain=0x7ffd1669c650) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Modules/main.c:2876 cf = {cf_flags = 0} cf = <optimized out> #18 pymain_main (pymain=<optimized out>, pymain=<optimized out>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Modules/main.c:3037 res = <optimized out> res = <optimized out> #19 0x00007fbac0cd2a2c in _Py_UnixMain (argc=<optimized out>, argv=<optimized out>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Modules/main.c:3072 pymain = {argc = 2, use_bytes_argv = 1, bytes_argv = 0x7ffd1669c7a8, wchar_argv = 0x0, status = 0, err = {prefix = 0x0, msg = 0x0, user_err = 0}, stdin_is_interactive = 1, skip_first_line = 0, filename = 0x560bb3c86df0 L"sf.py", command = 0x0, module = 0x0, main_importer_path = 0x0} #20 0x00007fbac0740413 in __libc_start_main (main=0x560bb3778050 <main>, argc=2, argv=0x7ffd1669c7a8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffd1669c798) at ../csu/libc-start.c:308 self = <optimized out> result = <optimized out> unwind_buf = {cancel_jmp_buf = {{jmp_buf = {0, -2611185483236398270, 94608255582304, 140724979484576, 0, 0, -8633824211270784190, -8601362210542343358}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x7ffd1669c7c0, 0x7fbac0e42150}, data = {prev = 0x0, cleanup = 0x0, canceltype = 376031168}}} not_first_call = <optimized out> #21 0x0000560bb377808e in _start () --- Additional comment from Jan Pazdziora on 2019-01-11 16:15:10 CET --- When I change the second etree.parse call to xml2 = etree.parse("sf1.xml") to try to open nonexistent file, I still get the segfault, even if I'd expect something like <lxml.etree._ElementTree object at 0x7f29e4e65e08> Traceback (most recent call last): File "sf.py", line 5, in <module> xml2 = etree.parse("sf1.xml") File "src/lxml/etree.pyx", line 3426, in lxml.etree.parse File "src/lxml/parser.pxi", line 1840, in lxml.etree._parseDocument File "src/lxml/parser.pxi", line 1866, in lxml.etree._parseDocumentFromURL File "src/lxml/parser.pxi", line 1770, in lxml.etree._parseDocFromFile File "src/lxml/parser.pxi", line 1163, in lxml.etree._BaseParser._parseDocFromFile File "src/lxml/parser.pxi", line 601, in lxml.etree._ParserContext._handleParseResultDoc File "src/lxml/parser.pxi", line 711, in lxml.etree._handleParseResult File "src/lxml/parser.pxi", line 638, in lxml.etree._raiseParseError OSError: Error reading file 'sf1.xml': failed to load external entity "sf1.xml" So one issue is why that second etree.parse on perfectly valid XML (as described in comment 0) fails and causes the exception. But second issue is that when xmlsec is imported, python itself is not able to display that OSError: Error reading file 'sf1.xml': failed to load external entity "sf1.xml" message at all.
Let's make this bugzilla the one for the problem of Objects/unicodeobject.c with code int _PyUnicode_Ready(PyObject *unicode) { wchar_t *end; Py_UCS4 maxchar = 0; Py_ssize_t num_surrogates; #if SIZEOF_WCHAR_T == 2 Py_ssize_t length_wo_surrogates; #endif /* _PyUnicode_Ready() is only intended for old-style API usage where strings were created using _PyObject_New() and where no canonical representation (the str field) has been set yet aka strings which are not yet ready. */ assert(_PyUnicode_CHECK(unicode)); assert(_PyUnicode_KIND(unicode) == PyUnicode_WCHAR_KIND); assert(_PyUnicode_WSTR(unicode) != NULL); assert(_PyUnicode_DATA_ANY(unicode) == NULL); assert(_PyUnicode_UTF8(unicode) == NULL); /* Actually, it should neither be interned nor be anything else: */ assert(_PyUnicode_STATE(unicode).interned == SSTATE_NOT_INTERNED); end = _PyUnicode_WSTR(unicode) + _PyUnicode_WSTR_LENGTH(unicode); if (find_maxchar_surrogates(_PyUnicode_WSTR(unicode), end, &maxchar, &num_surrogates) == -1) return -1; turning that unicode parameter into something with end = 0xfffffffffffffffd on which find_maxchar_surrogates fails. #1 _PyUnicode_Ready (unicode=b'sf.xml') at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/unicodeobject.c:1699 end = 0xfffffffffffffffd <error: Cannot access memory at address 0xfffffffffffffffd> maxchar = 0 num_surrogates = 0 The original bug 1665459 would then be for the original issue -- why can't lxml load things twice.
#1 _PyUnicode_Ready (unicode=b'sg.xml') Somehow, a bytes object, rather than a (Unicode) string, is used as the filename in a backtrace. That's an entirely different datatype at the C level. I don't think Python would be setting b'sg.xml' as the filename of running code, but I don't know enough about lxml or xmlsec to tell why/where they would do it.
This is currently assigned to python-maint, and we have it in low priority. If you (anyone) can get to it faster than we can, please take the bug.
I reported this bug upstream and I proposed a fix: * https://bugs.python.org/issue37467 * https://github.com/python/cpython/pull/14504
I consider that this low priority bug can wait for the next Python 3.7.4 release (scheduled for next days).
FEDORA-2019-9bfb4a3e4b has been submitted as an update to Fedora 30. https://bodhi.fedoraproject.org/updates/FEDORA-2019-9bfb4a3e4b
FEDORA-2019-60a1defcd1 has been submitted as an update to Fedora 29. https://bodhi.fedoraproject.org/updates/FEDORA-2019-60a1defcd1
python3-3.7.4-1.fc30, python3-docs-3.7.4-1.fc30 has been pushed to the Fedora 30 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-9bfb4a3e4b
python3-3.7.4-1.fc29, python3-docs-3.7.4-1.fc29 has been pushed to the Fedora 29 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-60a1defcd1
$ I don't see the issue fixed with python3-3.7.4-1.fc30.x86_64, I'm still getting python3 sf.py <lxml.etree._ElementTree object at 0x7f80cc91d640> Traceback (most recent call last): File "sf.py", line 5, in <module> xml2 = etree.parse("sf1.xml") File "src/lxml/etree.pyx", line 3426, in lxml.etree.parse File "src/lxml/parser.pxi", line 1840, in lxml.etree._parseDocument File "src/lxml/parser.pxi", line 1866, in lxml.etree._parseDocumentFromURL File "src/lxml/parser.pxi", line 1770, in lxml.etree._parseDocFromFile File "src/lxml/parser.pxi", line 1163, in lxml.etree._BaseParser._parseDocFromFile File "src/lxml/parser.pxi", line 601, in lxml.etree._ParserContext._handleParseResultDoc File "src/lxml/parser.pxi", line 711, in lxml.etree._handleParseResult File "src/lxml/parser.pxi", line 651, in lxml.etree._raiseParseError Segmentation fault (core dumped)
Sadly, Python 3.7 release manager decided to not pick my fix into 3.7.4 final release :-( https://github.com/python/cpython/commit/8cbffc4d96d1da0fbc38da6f34f2da30c5ffd601 The fix will be part of next 3.7.5 release scheduled for the end of next September ("3.7.5 final: 2019-09-30 (expected)" says the PEP 537).
3.7.5 has been updated. Could you check if it fixes the problem?
I tested the python37 in Rawhide (Python 3.7.5): the script doesn't crash anymore. sudo dnf install -y xmlsec1 xmlsec1-devel libtool-ltdl-devel python3.7 -m venv env env/bin/python -m pip install lxml xmlsec env/bin/python sf.py Output: func=xmlSecCryptoDLLibraryCreate:file=dl.c:line=135:obj=unknown:subj=lt_dlopenext:error=7:io function failed:name="libxmlsec1-openssl"; errno=2 func=xmlSecCryptoDLGetLibraryFunctions:file=dl.c:line=442:obj=unknown:subj=xmlSecCryptoDLLibraryCreate:error=1:xmlsec library function failed:crypto=openssl func=xmlSecCryptoDLLoadLibrary:file=dl.c:line=400:obj=unknown:subj=xmlSecCryptoDLGetLibraryFunctions:error=1:xmlsec library function failed: Traceback (most recent call last): File "sf.py", line 3, in <module> import xmlsec SystemError: null argument to internal routine
FEDORA-2019-232f092db0 has been submitted as an update to Fedora 31. https://bodhi.fedoraproject.org/updates/FEDORA-2019-232f092db0
FEDORA-2019-aba3cca74a has been submitted as an update to Fedora 30. https://bodhi.fedoraproject.org/updates/FEDORA-2019-aba3cca74a
FEDORA-2019-986622833f has been submitted as an update to Fedora 29. https://bodhi.fedoraproject.org/updates/FEDORA-2019-986622833f
This message is a reminder that Fedora 29 is nearing its end of life. Fedora will stop maintaining and issuing updates for Fedora 29 on 2019-11-26. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a Fedora 'version' of '29'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 29 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete.
Confirming the fix in Fedora 29 (container) as well, with python3-3.7.5-1.fc29.x86_64.
python3-3.7.5-1.fc30 has been pushed to the Fedora 30 stable repository. If problems still persist, please make note of it in this bug report.
python3-3.7.5-1.fc29 has been pushed to the Fedora 29 stable repository. If problems still persist, please make note of it in this bug report.
python3-3.7.5-1.fc31 has been pushed to the Fedora 31 stable repository. If problems still persist, please make note of it in this bug report.