Description of problem: The documentation of python xmlsec module https://github.com/mehcode/python-xmlsec/blob/master/doc/source/examples.rst#sign shows that it should work just fine with lxml. However, when xmlsec module is imported in python code (not even used), attempt to parse second xml file with lxml's etree.parse results in segfault. Version-Release number of selected component (if applicable): python3-xmlsec-1.3.3-3.fc29.x86_64 python3-lxml-4.2.5-1.fc29.x86_64 How reproducible: Deterministic. Steps to Reproduce: 1. cat > sf.xml <data/> Ctrl-D here 2. cat > sf.py from lxml import etree import xmlsec xml1 = etree.parse("sf.xml") print(xml1) xml2 = etree.parse("sf.xml") print(xml2) Ctrl-D here 3. python3 sf.py Actual results: <lxml.etree._ElementTree object at 0x7f39227536c8> Traceback (most recent call last): File "sf.py", line 5, in <module> xml2 = etree.parse("sf.xml") File "src/lxml/etree.pyx", line 3426, in lxml.etree.parse File "src/lxml/parser.pxi", line 1840, in lxml.etree._parseDocument File "src/lxml/parser.pxi", line 1866, in lxml.etree._parseDocumentFromURL File "src/lxml/parser.pxi", line 1770, in lxml.etree._parseDocFromFile File "src/lxml/parser.pxi", line 1163, in lxml.etree._BaseParser._parseDocFromFile File "src/lxml/parser.pxi", line 601, in lxml.etree._ParserContext._handleParseResultDoc File "src/lxml/parser.pxi", line 711, in lxml.etree._handleParseResult File "src/lxml/parser.pxi", line 651, in lxml.etree._raiseParseError Segmentation fault (core dumped) Expected results: <lxml.etree._ElementTree object at 0x7f246c117c08> <lxml.etree._ElementTree object at 0x7f246c117c88> Additional info: When the "import xmlsec" line is commented out, I get the expected result. This used to work on Fedora 27 with python3-lxml-4.1.1-1.fc27.x86_64 python3-xmlsec-1.3.3-1.fc27.x86_64 It started to fail on Fedora 28 with python3-lxml-4.2.5-1.fc28.x86_64 python3-xmlsec-1.3.3-1.fc28.x86_64 and fails up to rawhide.
Backtrace is: Core was generated by `python3 sf.py'. Program terminated with signal SIGSEGV, Segmentation fault. #0 find_maxchar_surrogates (num_surrogates=<synthetic pointer>, maxchar=<synthetic pointer>, end=0xfffffffffffffffd <error: Cannot access memory at address 0xfffffffffffffffd>, begin=0x1 <error: Cannot access memory at address 0x1>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/unicodeobject.c:1660 1660 ch = *iter; Missing separate debuginfos, use: dnf debuginfo-install libgpg-error-1.31-2.fc29.x86_64 libtool-ltdl-2.4.6-25.fc29.x86_64 libxml2-2.9.8-4.fc29.x86_64 libxslt-1.1.32-3.fc29.x86_64 openssl-libs-1.1.1-0.pre9.3.fc29.x86_64 python3-xmlsec-1.3.3-3.fc29.x86_64 xmlsec1-openssl-1.2.25-5.fc29.x86_64 xz-libs-5.2.4-3.fc29.x86_64 zlib-1.2.11-10.fc29.x86_64 (gdb) bt #0 find_maxchar_surrogates (num_surrogates=<synthetic pointer>, maxchar=<synthetic pointer>, end=0xfffffffffffffffd <error: Cannot access memory at address 0xfffffffffffffffd>, begin=0x1 <error: Cannot access memory at address 0x1>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/unicodeobject.c:1660 #1 _PyUnicode_Ready (unicode=b'sg.xml') at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/unicodeobject.c:1699 #2 0x00007f7f28965651 in unicode_fromformat_write_str (precision=-1, width=-1, str=<optimized out>, writer=0x7ffd3b6ecab0) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/unicodeobject.c:2909 #3 unicode_fromformat_arg (vargs=0x7ffd3b6ecb10, f=0x7f7f28b34964 "U\", line %d\n", writer=0x7ffd3b6ecab0) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/unicodeobject.c:2801 #4 PyUnicode_FromFormatV (format=<optimized out>, vargs=<optimized out>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/unicodeobject.c:2908 #5 0x00007f7f289ffd09 in PyUnicode_FromFormat ( format=format@entry=0x7f7f28b3495b " File \"%U\", line %d\n") at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/unicodeobject.c:2960 #6 0x00007f7f28b176f1 in print_exception (value=None, f=<_io.TextIOWrapper at remote 0x7f7f28248708>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Python/pythonrun.c:753 #7 print_exception_recursive (f=<_io.TextIOWrapper at remote 0x7f7f28248708>, value=<XMLSyntaxError(error_log=<lxml.etree._ListErrorLog at remote 0x7f7f279380e8>, code=1) at remote 0x7f7f27999d38>, seen=<optimized out>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Python/pythonrun.c:901 #8 0x00007f7f28b19eeb in PyErr_Display (exception=<optimized out>, value=<XMLSyntaxError(error_log=<lxml.etree._ListErrorLog at remote 0x7f7f279380e8>, code=1) at remote 0x7f7f27999d38>, tb=<optimized out>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Python/pythonrun.c:935 #9 0x00007f7f28b19f6c in sys_excepthook ( self=self@entry=<module at remote 0x7f7f2828ef98>, args=args@entry=(<type at remote 0x56025b2e1208>, <XMLSyntaxError(error_log=<lxml.etree._ListErrorLog at remote 0x7f7f279380e8>, code=1) at remote 0x7f7f27999d38>, <traceback at remote 0x7f7f27935688>)) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Python/sysmodule.c:321 #10 0x00007f7f28a24ef3 in _PyMethodDef_RawFastCallDict ( method=<optimized out>, self=<optimized out>, args=<optimized out>, nargs=<optimized out>, kwargs=<optimized out>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/call.c:518 #11 0x00007f7f28a25150 in _PyCFunction_FastCallDict ( func=<built-in method excepthook of module object at remote 0x7f7f2828ef98>, args=<optimized out>, nargs=<optimized out>, kwargs=<optimized out>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/call.c:582 #12 0x00007f7f28b2d73e in PyErr_PrintEx (set_sys_last_vars=1) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Python/pythonrun.c:679 #13 PyErr_PrintEx (set_sys_last_vars=<optimized out>, set_sys_last_vars=<optimized out>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Python/pythonrun.c:641 #14 0x00007f7f289cd3a3 in PyRun_SimpleFileExFlags (fp=<optimized out>, filename=<optimized out>, closeit=<optimized out>, flags=0x7ffd3b6ece90) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Python/pythonrun.c:435 #15 0x00007f7f28b2f7f1 in pymain_run_file (p_cf=0x7ffd3b6ece90, filename=0x56025b210df0 L"sf.py", fp=0x56025b272ef0) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Modules/main.c:427 #16 pymain_run_filename (cf=0x7ffd3b6ece90, pymain=0x7ffd3b6ecfa0) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Modules/main.c:1627 #17 pymain_run_python (pymain=0x7ffd3b6ecfa0) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Modules/main.c:2876 #18 pymain_main (pymain=<optimized out>, pymain=<optimized out>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Modules/main.c:3037 #19 0x00007f7f28b2fa2c in _Py_UnixMain (argc=<optimized out>, argv=<optimized out>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Modules/main.c:3072 #20 0x00007f7f2859d413 in __libc_start_main (main=0x56025a8fc050 <main>, argc=2, argv=0x7ffd3b6ed0f8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffd3b6ed0e8) at ../csu/libc-start.c:308 #21 0x000056025a8fc08e in _start ()
Building python3-xmlsec from https://github.com/mehcode/python-xmlsec master (1.3.6) does not seem to improve the situation.
Here by full, from different coredump though: (gdb) bt full #0 find_maxchar_surrogates (num_surrogates=<synthetic pointer>, maxchar=<synthetic pointer>, end=0xfffffffffffffffd <error: Cannot access memory at address 0xfffffffffffffffd>, begin=0x1 <error: Cannot access memory at address 0x1>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/unicodeobject.c:1660 iter = 0x1 <error: Cannot access memory at address 0x1> ch = <optimized out> iter = <optimized out> ch = <optimized out> #1 _PyUnicode_Ready (unicode=b'sf.xml') at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/unicodeobject.c:1699 end = 0xfffffffffffffffd <error: Cannot access memory at address 0xfffffffffffffffd> maxchar = 0 num_surrogates = 0 #2 0x00007fbac0b08651 in unicode_fromformat_write_str (precision=-1, width=-1, str=<optimized out>, writer=0x7ffd1669c160) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/unicodeobject.c:2909 length = <optimized out> fill = <optimized out> arglen = <optimized out> maxchar = <optimized out> length = <optimized out> fill = <optimized out> arglen = <optimized out> maxchar = <optimized out> #3 unicode_fromformat_arg (vargs=0x7ffd1669c1c0, f=0x7fbac0cd7964 "U\", line %d\n", writer=0x7ffd1669c160) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/unicodeobject.c:2801 obj = b'sf.xml' len = <optimized out> zeropad = <optimized out> longflag = <optimized out> longlongflag = <optimized out> size_tflag = <optimized out> p = <optimized out> width = <optimized out> precision = -1 fill = <optimized out> p = <optimized out> len = <optimized out> zeropad = <optimized out> width = <optimized out> precision = <optimized out> longflag = <optimized out> longlongflag = <optimized out> size_tflag = <optimized out> fill = <optimized out> ordinal = <optimized out> buffer = <optimized out> arglen = <optimized out> fillchar = <optimized out> number = <optimized out> s = <optimized out> obj = <optimized out> obj = <optimized out> str = <optimized out> obj = <optimized out> str = <optimized out> _py_decref_tmp = <optimized out> _py_decref_tmp = <optimized out> obj = <optimized out> repr = <optimized out> _py_decref_tmp = <optimized out> _py_decref_tmp = <optimized out> obj = <optimized out> ascii = <optimized out> _py_decref_tmp = <optimized out> _py_decref_tmp = <optimized out> #4 PyUnicode_FromFormatV (format=<optimized out>, vargs=<optimized out>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/unicodeobject.c:2908 vargs2 = {{gp_offset = 16, fp_offset = 48, overflow_arg_area = 0x7ffd1669c300, reg_save_area = 0x7ffd1669c240}} f = <optimized out> writer = { buffer = ' File "\x01\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00w\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00ÿÿÿÿÿÿÿÿ\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x10\x00\x00\x00\x00\x00\x00ÿ\x0f\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', data = 0x7fbabfbbeab0, kind = PyUnicode_1BYTE_KIND, maxchar = 127, size = 121, pos = 8, min_length = 121, min_char = 127, overallocate = 1 '\001', readonly = 0 '\000'} #5 0x00007fbac0ba2d09 in PyUnicode_FromFormat (format=format@entry=0x7fbac0cd795b " File \"%U\", line %d\n") at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/unicodeobject.c:2960 ret = <optimized out> vargs = {{gp_offset = 8, fp_offset = 48, overflow_arg_area = 0x7ffd1669c300, reg_save_area = 0x7ffd1669c240}} #6 0x00007fbac0cba6f1 in print_exception (value=None, f=<_io.TextIOWrapper at remote 0x7fbac03ec708>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Python/pythonrun.c:753 line = <optimized out> message = None filename = b'sf.xml' text = 0x0 lineno = 0 offset = -1 err = <optimized out> type = <type at remote 0x560bb3d57208> tb = <traceback at remote 0x7fbabfae0608> err = <optimized out> type = <optimized out> tb = <optimized out> PyId_print_file_and_line = <optimized out> message = <optimized out> filename = <optimized out> text = <optimized out> lineno = <optimized out> offset = <optimized out> line = <optimized out> _py_decref_tmp = <optimized out> _py_decref_tmp = <optimized out> _py_decref_tmp = <optimized out> _py_decref_tmp = <optimized out> --Type <RET> for more, q to quit, c to continue without paging--c moduleName = <optimized out> className = <optimized out> PyId___module__ = <optimized out> dot = <optimized out> _py_xdecref_tmp = <optimized out> _py_decref_tmp = <optimized out> _py_decref_tmp = <optimized out> s = <optimized out> _py_xdecref_tmp = <optimized out> _py_decref_tmp = <optimized out> _py_xdecref_tmp = <optimized out> _py_decref_tmp = <optimized out> _py_decref_tmp = <optimized out> #7 print_exception_recursive (f=<_io.TextIOWrapper at remote 0x7fbac03ec708>, value=<XMLSyntaxError(error_log=<lxml.etree._ListErrorLog at remote 0x7fbabfae11d8>, code=1) at remote 0x7fbabfb43d38>, seen=<optimized out>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Python/pythonrun.c:901 err = 0 res = <optimized out> cause = <optimized out> context = <optimized out> #8 0x00007fbac0cbceeb in PyErr_Display (exception=<optimized out>, value=<XMLSyntaxError(error_log=<lxml.etree._ListErrorLog at remote 0x7fbabfae11d8>, code=1) at remote 0x7fbabfb43d38>, tb=<optimized out>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Python/pythonrun.c:935 seen = {140440056905016} f = <_io.TextIOWrapper at remote 0x7fbac03ec708> #9 0x00007fbac0cbcf6c in sys_excepthook (self=self@entry=<module at remote 0x7fbac0432f98>, args=args@entry=(<type at remote 0x560bb3d57208>, <XMLSyntaxError(error_log=<lxml.etree._ListErrorLog at remote 0x7fbabfae11d8>, code=1) at remote 0x7fbabfb43d38>, <traceback at remote 0x7fbabfae0608>)) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Python/sysmodule.c:321 exc = <type at remote 0x560bb3d57208> value = <XMLSyntaxError(error_log=<lxml.etree._ListErrorLog at remote 0x7fbabfae11d8>, code=1) at remote 0x7fbabfb43d38> tb = <traceback at remote 0x7fbabfae0608> #10 0x00007fbac0bc7ef3 in _PyMethodDef_RawFastCallDict (method=<optimized out>, self=<optimized out>, args=<optimized out>, nargs=<optimized out>, kwargs=<optimized out>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/call.c:518 argstuple = (<type at remote 0x560bb3d57208>, <XMLSyntaxError(error_log=<lxml.etree._ListErrorLog at remote 0x7fbabfae11d8>, code=1) at remote 0x7fbabfb43d38>, <traceback at remote 0x7fbabfae0608>) meth = <optimized out> flags = 1 result = 0x0 #11 0x00007fbac0bc8150 in _PyCFunction_FastCallDict (func=<built-in method excepthook of module object at remote 0x7fbac0432f98>, args=<optimized out>, nargs=<optimized out>, kwargs=<optimized out>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/call.c:582 result = <optimized out> #12 0x00007fbac0cd073e in PyErr_PrintEx (set_sys_last_vars=1) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Python/pythonrun.c:679 stack = {<type at remote 0x560bb3d57208>, <XMLSyntaxError(error_log=<lxml.etree._ListErrorLog at remote 0x7fbabfae11d8>, code=1) at remote 0x7fbabfb43d38>, <traceback at remote 0x7fbabfae0608>} result = <optimized out> exception = <type at remote 0x560bb3d57208> v = <XMLSyntaxError(error_log=<lxml.etree._ListErrorLog at remote 0x7fbabfae11d8>, code=1) at remote 0x7fbabfb43d38> tb = <traceback at remote 0x7fbabfae0608> hook = <optimized out> exception = <optimized out> v = <optimized out> tb = <optimized out> hook = <optimized out> stack = <optimized out> result = <optimized out> exception2 = <optimized out> v2 = <optimized out> tb2 = <optimized out> _py_decref_tmp = <optimized out> _py_decref_tmp = <optimized out> _py_xdecref_tmp = <optimized out> _py_decref_tmp = <optimized out> _py_xdecref_tmp = <optimized out> _py_decref_tmp = <optimized out> _py_xdecref_tmp = <optimized out> _py_decref_tmp = <optimized out> _py_xdecref_tmp = <optimized out> _py_decref_tmp = <optimized out> _py_xdecref_tmp = <optimized out> _py_decref_tmp = <optimized out> #13 PyErr_PrintEx (set_sys_last_vars=<optimized out>, set_sys_last_vars=<optimized out>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Python/pythonrun.c:641 exception = <optimized out> v = <optimized out> tb = <optimized out> hook = <optimized out> _py_decref_tmp = <optimized out> _py_decref_tmp = <optimized out> _py_decref_tmp = <optimized out> _py_decref_tmp = <optimized out> _py_xdecref_tmp = <optimized out> _py_decref_tmp = <optimized out> _py_xdecref_tmp = <optimized out> _py_xdecref_tmp = <optimized out> _py_xdecref_tmp = <optimized out> _py_xdecref_tmp = <optimized out> _py_decref_tmp = <optimized out> _py_decref_tmp = <optimized out> exception2 = <optimized out> v2 = <optimized out> tb2 = <optimized out> stack = <optimized out> result = <optimized out> exception = <optimized out> v = <optimized out> tb = <optimized out> hook = <optimized out> _py_xdecref_tmp = <optimized out> _py_xdecref_tmp = <optimized out> _py_xdecref_tmp = <optimized out> _py_xdecref_tmp = <optimized out> _py_decref_tmp = <optimized out> _py_decref_tmp = <optimized out> exception2 = <optimized out> v2 = <optimized out> tb2 = <optimized out> stack = <optimized out> result = <optimized out> _py_xdecref_tmp = <optimized out> _py_decref_tmp = <optimized out> _py_xdecref_tmp = <optimized out> _py_decref_tmp = <optimized out> _py_xdecref_tmp = <optimized out> _py_decref_tmp = <optimized out> #14 0x00007fbac0b703a3 in PyRun_SimpleFileExFlags (fp=<optimized out>, filename=<optimized out>, closeit=<optimized out>, flags=0x7ffd1669c540) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Python/pythonrun.c:435 m = 0x0 d = {'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <SourceFileLoader(name='__main__', path='sf.py') at remote 0x7fbac03385c0>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7fbac0438c28>, '__file__': 'sf.py', '__cached__': None, 'etree': <module at remote 0x7fbac02b1688>, 'xmlsec': <module at remote 0x7fbabfad4228>, 'xml1': <lxml.etree._ElementTree at remote 0x7fbabfae0388>} v = <optimized out> ext = <optimized out> set_file_name = 1 ret = -1 len = <optimized out> _py_decref_tmp = <optimized out> #15 0x00007fbac0cd27f1 in pymain_run_file (p_cf=0x7ffd1669c540, filename=0x560bb3c86df0 L"sf.py", fp=0x560bb3ce8ef0) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Modules/main.c:427 unicode = <optimized out> bytes = b'sf.py' filename_str = <optimized out> run = <optimized out> unicode = <optimized out> bytes = <optimized out> filename_str = <optimized out> run = <optimized out> _py_decref_tmp = <optimized out> _py_xdecref_tmp = <optimized out> _py_decref_tmp = <optimized out> #16 pymain_run_filename (cf=0x7ffd1669c540, pymain=0x7ffd1669c650) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Modules/main.c:1627 fp = 0x560bb3ce8ef0 fp = <optimized out> #17 pymain_run_python (pymain=0x7ffd1669c650) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Modules/main.c:2876 cf = {cf_flags = 0} cf = <optimized out> #18 pymain_main (pymain=<optimized out>, pymain=<optimized out>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Modules/main.c:3037 res = <optimized out> res = <optimized out> #19 0x00007fbac0cd2a2c in _Py_UnixMain (argc=<optimized out>, argv=<optimized out>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Modules/main.c:3072 pymain = {argc = 2, use_bytes_argv = 1, bytes_argv = 0x7ffd1669c7a8, wchar_argv = 0x0, status = 0, err = {prefix = 0x0, msg = 0x0, user_err = 0}, stdin_is_interactive = 1, skip_first_line = 0, filename = 0x560bb3c86df0 L"sf.py", command = 0x0, module = 0x0, main_importer_path = 0x0} #20 0x00007fbac0740413 in __libc_start_main (main=0x560bb3778050 <main>, argc=2, argv=0x7ffd1669c7a8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffd1669c798) at ../csu/libc-start.c:308 self = <optimized out> result = <optimized out> unwind_buf = {cancel_jmp_buf = {{jmp_buf = {0, -2611185483236398270, 94608255582304, 140724979484576, 0, 0, -8633824211270784190, -8601362210542343358}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x7ffd1669c7c0, 0x7fbac0e42150}, data = {prev = 0x0, cleanup = 0x0, canceltype = 376031168}}} not_first_call = <optimized out> #21 0x0000560bb377808e in _start ()
When I change the second etree.parse call to xml2 = etree.parse("sf1.xml") to try to open nonexistent file, I still get the segfault, even if I'd expect something like <lxml.etree._ElementTree object at 0x7f29e4e65e08> Traceback (most recent call last): File "sf.py", line 5, in <module> xml2 = etree.parse("sf1.xml") File "src/lxml/etree.pyx", line 3426, in lxml.etree.parse File "src/lxml/parser.pxi", line 1840, in lxml.etree._parseDocument File "src/lxml/parser.pxi", line 1866, in lxml.etree._parseDocumentFromURL File "src/lxml/parser.pxi", line 1770, in lxml.etree._parseDocFromFile File "src/lxml/parser.pxi", line 1163, in lxml.etree._BaseParser._parseDocFromFile File "src/lxml/parser.pxi", line 601, in lxml.etree._ParserContext._handleParseResultDoc File "src/lxml/parser.pxi", line 711, in lxml.etree._handleParseResult File "src/lxml/parser.pxi", line 638, in lxml.etree._raiseParseError OSError: Error reading file 'sf1.xml': failed to load external entity "sf1.xml" So one issue is why that second etree.parse on perfectly valid XML (as described in comment 0) fails and causes the exception. But second issue is that when xmlsec is imported, python itself is not able to display that OSError: Error reading file 'sf1.xml': failed to load external entity "sf1.xml" message at all.
I've now filed bug 1665490 for the second issue, python failing to show the exception message. This bugzilla stays for the problem of parsing multiple files, which should not produce any exception.
When I try to run the code with python2, with python2-lxml-4.2.5-1.fc29.x86_64 python2-xmlsec-1.3.3-3.fc29.x86_64 I get # python2 sf.py <lxml.etree._ElementTree object at 0x7fa2f910d098> Traceback (most recent call last): File "sf.py", line 5, in <module> xml2 = etree.parse("sf.xml") File "src/lxml/etree.pyx", line 3426, in lxml.etree.parse File "src/lxml/parser.pxi", line 1840, in lxml.etree._parseDocument File "src/lxml/parser.pxi", line 1866, in lxml.etree._parseDocumentFromURL File "src/lxml/parser.pxi", line 1770, in lxml.etree._parseDocFromFile File "src/lxml/parser.pxi", line 1163, in lxml.etree._BaseParser._parseDocFromFile File "src/lxml/parser.pxi", line 601, in lxml.etree._ParserContext._handleParseResultDoc File "src/lxml/parser.pxi", line 711, in lxml.etree._handleParseResult File "src/lxml/parser.pxi", line 651, in lxml.etree._raiseParseError File "sf.xml", line 0 lxml.etree.XMLSyntaxError
In bug 1665490 comment 1, Petr suggests that the segfault under python 3 is caused by the fact that the filename is passed to the exception as bytes object, rather than unicode string. If xmlsec is able to do that, maybe it also modifies other string behaviours, even under python 2.
Hi Jan, My recommendation would be to ensure you can reproduce this with pip-install packages and report it upstream. I might have time to look at it on the weekend, but it looks like upstream is responsive and you've got a super simple reproducer so that should help.
Thanks for the suggestion. I've reproduced it with pip3 and reported upstream as https://github.com/mehcode/python-xmlsec/issues/96.
I see the segfault on Fedora 30 and 31 as well.
This message is a reminder that Fedora 29 is nearing its end of life. Fedora will stop maintaining and issuing updates for Fedora 29 on 2019-11-26. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a Fedora 'version' of '29'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 29 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete.
Updating version per comment 10.
With bug 1665490 fixed in Fedora 30, the segfault is now gone but traceback stays: <lxml.etree._ElementTree object at 0x7ff2b5a8cb40> Traceback (most recent call last): File "sf.py", line 5, in <module> xml2 = etree.parse("sf.xml") File "src/lxml/etree.pyx", line 3426, in lxml.etree.parse File "src/lxml/parser.pxi", line 1840, in lxml.etree._parseDocument File "src/lxml/parser.pxi", line 1866, in lxml.etree._parseDocumentFromURL File "src/lxml/parser.pxi", line 1770, in lxml.etree._parseDocFromFile File "src/lxml/parser.pxi", line 1163, in lxml.etree._BaseParser._parseDocFromFile File "src/lxml/parser.pxi", line 601, in lxml.etree._ParserContext._handleParseResultDoc File "src/lxml/parser.pxi", line 711, in lxml.etree._handleParseResult File "src/lxml/parser.pxi", line 651, in lxml.etree._raiseParseError File "b'sf.xml'", line 0 lxml.etree.XMLSyntaxError This is with python3-3.7.6-2.fc30.x86_64 python3-xmlsec-1.3.3-5.fc30.x86_64 python3-lxml-4.2.5-2.fc30.x86_64
This message is a reminder that Fedora 30 is nearing its end of life. Fedora will stop maintaining and issuing updates for Fedora 30 on 2020-05-26. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a Fedora 'version' of '30'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 30 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete.
This package has changed maintainer in the Fedora. Reassigning to the new maintainer of this component.
On Fedora 31 with python3-3.7.7-1.fc31.x86_64 python3-xmlsec-1.3.3-6.fc31.x86_64 python3-lxml-4.4.0-1.fc31.x86_64 the traceback is still there: Traceback (most recent call last): File "sf.py", line 3, in <module> xml1 = etree.parse("sf.xml") File "src/lxml/etree.pyx", line 3467, in lxml.etree.parse File "src/lxml/parser.pxi", line 1839, in lxml.etree._parseDocument File "src/lxml/parser.pxi", line 1865, in lxml.etree._parseDocumentFromURL File "src/lxml/parser.pxi", line 1769, in lxml.etree._parseDocFromFile File "src/lxml/parser.pxi", line 1163, in lxml.etree._BaseParser._parseDocFromFile File "src/lxml/parser.pxi", line 601, in lxml.etree._ParserContext._handleParseResultDoc File "src/lxml/parser.pxi", line 711, in lxml.etree._handleParseResult File "src/lxml/parser.pxi", line 640, in lxml.etree._raiseParseError File "sf.xml", line 1 lxml.etree.XMLSyntaxError: Extra content at the end of the document, line 1, column 8
Oops, sorry, I had some garbled sf.xml content. The traceback on Fedora 31 matches the one on Fedora 30: <lxml.etree._ElementTree object at 0x7f4cbaf9ba50> Traceback (most recent call last): File "sf.py", line 5, in <module> xml2 = etree.parse("sf.xml") File "src/lxml/etree.pyx", line 3467, in lxml.etree.parse File "src/lxml/parser.pxi", line 1839, in lxml.etree._parseDocument File "src/lxml/parser.pxi", line 1865, in lxml.etree._parseDocumentFromURL File "src/lxml/parser.pxi", line 1769, in lxml.etree._parseDocFromFile File "src/lxml/parser.pxi", line 1163, in lxml.etree._BaseParser._parseDocFromFile File "src/lxml/parser.pxi", line 601, in lxml.etree._ParserContext._handleParseResultDoc File "src/lxml/parser.pxi", line 711, in lxml.etree._handleParseResult File "src/lxml/parser.pxi", line 651, in lxml.etree._raiseParseError File "b'sf.xml'", line 0 lxml.etree.XMLSyntaxError
Hello, upstream mentions in https://github.com/mehcode/python-xmlsec/issues/96#issuecomment-632123881 that latest python-xmlsec should have the problem resolved. Could you make a (scratch) build so that we could test it in Fedora?
Nope, the issue is not fixed - sorry for the unnecessary noise. I've tested with the lxml manylinux wheel from PyPI, not with the distro package.
(In reply to Jan Pazdziora from comment #18) > Hello, upstream mentions in > https://github.com/mehcode/python-xmlsec/issues/96#issuecomment-632123881 > that latest python-xmlsec should have the problem resolved. Could you make a > (scratch) build so that we could test it in Fedora? Hi, the latest python-xmlsec upstream release (1.3.8) is now available in rawhide. As Oleg mentioned in the GitHub issue it seems to be an issue in lxml [1]. Reassigning to python-lxml component. [1] https://bugs.launchpad.net/lxml/+bug/1880251
Fixed in python-lxml-4.5.1-1.fc33.
Please note that the issue is currently only partially resolved on lxml side. The code where lxml is imported after xmlsec: ```python import xmlsec from lxml import etree etree.parse('doc.xml') etree.parse('doc.xml') ``` will continue to fail with python3-lxml-4.5.1-1.fc33 (also reported in https://bugs.launchpad.net/lxml/+bug/1880251/comments/3).