Bug 1665459

Summary: Importing xmlsec causes traceback when second file is etree.parsed
Product: [Fedora] Fedora Reporter: Jan Pazdziora (Red Hat) <jpazdziora>
Component: python-lxmlAssignee: Igor Raits <igor.raits>
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 31CC: igor.raits, jcline, jeremy, jpazdziora, lilydjwg, me, mizdebsk, mkfedora, oleg.hoefling
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: python-lxml-4.5.1-1.fc33 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1665490 (view as bug list) Environment:
Last Closed: 2020-06-01 18:15:39 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1665490    

Description Jan Pazdziora (Red Hat) 2019-01-11 13:28:42 UTC 
Description of problem:

The documentation of python xmlsec module https://github.com/mehcode/python-xmlsec/blob/master/doc/source/examples.rst#sign shows that it should work just fine with lxml. However, when xmlsec module is imported in python code (not even used), attempt to parse second xml file with lxml's etree.parse results in segfault.

Version-Release number of selected component (if applicable):

python3-xmlsec-1.3.3-3.fc29.x86_64
python3-lxml-4.2.5-1.fc29.x86_64

How reproducible:

Deterministic.

Steps to Reproduce:
1. cat > sf.xml
<data/>
Ctrl-D here

2. cat > sf.py
from lxml import etree
import xmlsec
xml1 = etree.parse("sf.xml")
print(xml1)
xml2 = etree.parse("sf.xml")
print(xml2)
Ctrl-D here

3. python3 sf.py

Actual results:

<lxml.etree._ElementTree object at 0x7f39227536c8>
Traceback (most recent call last):
  File "sf.py", line 5, in <module>
    xml2 = etree.parse("sf.xml")
  File "src/lxml/etree.pyx", line 3426, in lxml.etree.parse
  File "src/lxml/parser.pxi", line 1840, in lxml.etree._parseDocument
  File "src/lxml/parser.pxi", line 1866, in lxml.etree._parseDocumentFromURL
  File "src/lxml/parser.pxi", line 1770, in lxml.etree._parseDocFromFile
  File "src/lxml/parser.pxi", line 1163, in lxml.etree._BaseParser._parseDocFromFile
  File "src/lxml/parser.pxi", line 601, in lxml.etree._ParserContext._handleParseResultDoc
  File "src/lxml/parser.pxi", line 711, in lxml.etree._handleParseResult
  File "src/lxml/parser.pxi", line 651, in lxml.etree._raiseParseError
Segmentation fault (core dumped)

Expected results:

<lxml.etree._ElementTree object at 0x7f246c117c08>
<lxml.etree._ElementTree object at 0x7f246c117c88>

Additional info:

When the "import xmlsec" line is commented out, I get the expected result.

This used to work on Fedora 27 with

python3-lxml-4.1.1-1.fc27.x86_64
python3-xmlsec-1.3.3-1.fc27.x86_64

It started to fail on Fedora 28 with

python3-lxml-4.2.5-1.fc28.x86_64
python3-xmlsec-1.3.3-1.fc28.x86_64

and fails up to rawhide.
Comment 1 Jan Pazdziora (Red Hat) 2019-01-11 13:30:30 UTC 
Backtrace is:

Core was generated by `python3 sf.py'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  find_maxchar_surrogates (num_surrogates=<synthetic pointer>, 
    maxchar=<synthetic pointer>, 
    end=0xfffffffffffffffd <error: Cannot access memory at address 0xfffffffffffffffd>, begin=0x1 <error: Cannot access memory at address 0x1>)
    at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/unicodeobject.c:1660
1660	            ch = *iter;
Missing separate debuginfos, use: dnf debuginfo-install libgpg-error-1.31-2.fc29.x86_64 libtool-ltdl-2.4.6-25.fc29.x86_64 libxml2-2.9.8-4.fc29.x86_64 libxslt-1.1.32-3.fc29.x86_64 openssl-libs-1.1.1-0.pre9.3.fc29.x86_64 python3-xmlsec-1.3.3-3.fc29.x86_64 xmlsec1-openssl-1.2.25-5.fc29.x86_64 xz-libs-5.2.4-3.fc29.x86_64 zlib-1.2.11-10.fc29.x86_64
(gdb) bt
#0  find_maxchar_surrogates (num_surrogates=<synthetic pointer>, 
    maxchar=<synthetic pointer>, 
    end=0xfffffffffffffffd <error: Cannot access memory at address 0xfffffffffffffffd>, begin=0x1 <error: Cannot access memory at address 0x1>)
    at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/unicodeobject.c:1660
#1  _PyUnicode_Ready (unicode=b'sg.xml')
    at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/unicodeobject.c:1699
#2  0x00007f7f28965651 in unicode_fromformat_write_str (precision=-1, 
    width=-1, str=<optimized out>, writer=0x7ffd3b6ecab0)
    at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/unicodeobject.c:2909
#3  unicode_fromformat_arg (vargs=0x7ffd3b6ecb10, 
    f=0x7f7f28b34964 "U\", line %d\n", writer=0x7ffd3b6ecab0)
    at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/unicodeobject.c:2801
#4  PyUnicode_FromFormatV (format=<optimized out>, vargs=<optimized out>)
    at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/unicodeobject.c:2908
#5  0x00007f7f289ffd09 in PyUnicode_FromFormat (
    format=format@entry=0x7f7f28b3495b "  File \"%U\", line %d\n")
    at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/unicodeobject.c:2960
#6  0x00007f7f28b176f1 in print_exception (value=None, 
    f=<_io.TextIOWrapper at remote 0x7f7f28248708>)
    at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Python/pythonrun.c:753
#7  print_exception_recursive (f=<_io.TextIOWrapper at remote 0x7f7f28248708>, 
    value=<XMLSyntaxError(error_log=<lxml.etree._ListErrorLog at remote 0x7f7f279380e8>, code=1) at remote 0x7f7f27999d38>, seen=<optimized out>)
    at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Python/pythonrun.c:901
#8  0x00007f7f28b19eeb in PyErr_Display (exception=<optimized out>, 
    value=<XMLSyntaxError(error_log=<lxml.etree._ListErrorLog at remote 0x7f7f279380e8>, code=1) at remote 0x7f7f27999d38>, tb=<optimized out>)
    at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Python/pythonrun.c:935
#9  0x00007f7f28b19f6c in sys_excepthook (
    self=self@entry=<module at remote 0x7f7f2828ef98>, 
    args=args@entry=(<type at remote 0x56025b2e1208>, <XMLSyntaxError(error_log=<lxml.etree._ListErrorLog at remote 0x7f7f279380e8>, code=1) at remote 0x7f7f27999d38>, <traceback at remote 0x7f7f27935688>))
    at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Python/sysmodule.c:321
#10 0x00007f7f28a24ef3 in _PyMethodDef_RawFastCallDict (
    method=<optimized out>, self=<optimized out>, args=<optimized out>, 
    nargs=<optimized out>, kwargs=<optimized out>)
    at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/call.c:518
#11 0x00007f7f28a25150 in _PyCFunction_FastCallDict (
    func=<built-in method excepthook of module object at remote 0x7f7f2828ef98>, args=<optimized out>, nargs=<optimized out>, kwargs=<optimized out>)
    at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/call.c:582
#12 0x00007f7f28b2d73e in PyErr_PrintEx (set_sys_last_vars=1)
    at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Python/pythonrun.c:679
#13 PyErr_PrintEx (set_sys_last_vars=<optimized out>, 
    set_sys_last_vars=<optimized out>)
    at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Python/pythonrun.c:641
#14 0x00007f7f289cd3a3 in PyRun_SimpleFileExFlags (fp=<optimized out>, 
    filename=<optimized out>, closeit=<optimized out>, flags=0x7ffd3b6ece90)
    at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Python/pythonrun.c:435
#15 0x00007f7f28b2f7f1 in pymain_run_file (p_cf=0x7ffd3b6ece90, 
    filename=0x56025b210df0 L"sf.py", fp=0x56025b272ef0)
    at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Modules/main.c:427
#16 pymain_run_filename (cf=0x7ffd3b6ece90, pymain=0x7ffd3b6ecfa0)
    at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Modules/main.c:1627
#17 pymain_run_python (pymain=0x7ffd3b6ecfa0)
    at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Modules/main.c:2876
#18 pymain_main (pymain=<optimized out>, pymain=<optimized out>)
    at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Modules/main.c:3037
#19 0x00007f7f28b2fa2c in _Py_UnixMain (argc=<optimized out>, 
    argv=<optimized out>)
    at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Modules/main.c:3072
#20 0x00007f7f2859d413 in __libc_start_main (main=0x56025a8fc050 <main>, 
    argc=2, argv=0x7ffd3b6ed0f8, init=<optimized out>, fini=<optimized out>, 
    rtld_fini=<optimized out>, stack_end=0x7ffd3b6ed0e8)
    at ../csu/libc-start.c:308
#21 0x000056025a8fc08e in _start ()
Comment 2 Jan Pazdziora (Red Hat) 2019-01-11 14:25:27 UTC 
Building python3-xmlsec from https://github.com/mehcode/python-xmlsec master (1.3.6) does not seem to improve the situation.
Comment 3 Jan Pazdziora (Red Hat) 2019-01-11 15:04:48 UTC 
Here by full, from different coredump though:

(gdb) bt full
#0  find_maxchar_surrogates (num_surrogates=<synthetic pointer>, maxchar=<synthetic pointer>, 
    end=0xfffffffffffffffd <error: Cannot access memory at address 0xfffffffffffffffd>, begin=0x1 <error: Cannot access memory at address 0x1>)
    at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/unicodeobject.c:1660
        iter = 0x1 <error: Cannot access memory at address 0x1>
        ch = <optimized out>
        iter = <optimized out>
        ch = <optimized out>
#1  _PyUnicode_Ready (unicode=b'sf.xml') at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/unicodeobject.c:1699
        end = 0xfffffffffffffffd <error: Cannot access memory at address 0xfffffffffffffffd>
        maxchar = 0
        num_surrogates = 0
#2  0x00007fbac0b08651 in unicode_fromformat_write_str (precision=-1, width=-1, str=<optimized out>, writer=0x7ffd1669c160)
    at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/unicodeobject.c:2909
        length = <optimized out>
        fill = <optimized out>
        arglen = <optimized out>
        maxchar = <optimized out>
        length = <optimized out>
        fill = <optimized out>
        arglen = <optimized out>
        maxchar = <optimized out>
#3  unicode_fromformat_arg (vargs=0x7ffd1669c1c0, f=0x7fbac0cd7964 "U\", line %d\n", writer=0x7ffd1669c160)
    at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/unicodeobject.c:2801
        obj = b'sf.xml'
        len = <optimized out>
        zeropad = <optimized out>
        longflag = <optimized out>
        longlongflag = <optimized out>
        size_tflag = <optimized out>
        p = <optimized out>
        width = <optimized out>
        precision = -1
        fill = <optimized out>
        p = <optimized out>
        len = <optimized out>
        zeropad = <optimized out>
        width = <optimized out>
        precision = <optimized out>
        longflag = <optimized out>
        longlongflag = <optimized out>
        size_tflag = <optimized out>
        fill = <optimized out>
        ordinal = <optimized out>
        buffer = <optimized out>
        arglen = <optimized out>
        fillchar = <optimized out>
        number = <optimized out>
        s = <optimized out>
        obj = <optimized out>
        obj = <optimized out>
        str = <optimized out>
        obj = <optimized out>
        str = <optimized out>
        _py_decref_tmp = <optimized out>
        _py_decref_tmp = <optimized out>
        obj = <optimized out>
        repr = <optimized out>
        _py_decref_tmp = <optimized out>
        _py_decref_tmp = <optimized out>
        obj = <optimized out>
        ascii = <optimized out>
        _py_decref_tmp = <optimized out>
        _py_decref_tmp = <optimized out>
#4  PyUnicode_FromFormatV (format=<optimized out>, vargs=<optimized out>)
    at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/unicodeobject.c:2908
        vargs2 = {{gp_offset = 16, fp_offset = 48, overflow_arg_area = 0x7ffd1669c300, reg_save_area = 0x7ffd1669c240}}
        f = <optimized out>
        writer = {
          buffer = '  File "\x01\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00w\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00ÿÿÿÿÿÿÿÿ\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x10\x00\x00\x00\x00\x00\x00ÿ\x0f\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', data = 0x7fbabfbbeab0, kind = PyUnicode_1BYTE_KIND, maxchar = 127, size = 121, pos = 8, min_length = 121, 
          min_char = 127, overallocate = 1 '\001', readonly = 0 '\000'}
#5  0x00007fbac0ba2d09 in PyUnicode_FromFormat (format=format@entry=0x7fbac0cd795b "  File \"%U\", line %d\n")
    at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/unicodeobject.c:2960
        ret = <optimized out>
        vargs = {{gp_offset = 8, fp_offset = 48, overflow_arg_area = 0x7ffd1669c300, reg_save_area = 0x7ffd1669c240}}
#6  0x00007fbac0cba6f1 in print_exception (value=None, f=<_io.TextIOWrapper at remote 0x7fbac03ec708>)
    at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Python/pythonrun.c:753
        line = <optimized out>
        message = None
        filename = b'sf.xml'
        text = 0x0
        lineno = 0
        offset = -1
        err = <optimized out>
        type = <type at remote 0x560bb3d57208>
        tb = <traceback at remote 0x7fbabfae0608>
        err = <optimized out>
        type = <optimized out>
        tb = <optimized out>
        PyId_print_file_and_line = <optimized out>
        message = <optimized out>
        filename = <optimized out>
        text = <optimized out>
        lineno = <optimized out>
        offset = <optimized out>
        line = <optimized out>
        _py_decref_tmp = <optimized out>
        _py_decref_tmp = <optimized out>
        _py_decref_tmp = <optimized out>
        _py_decref_tmp = <optimized out>
--Type <RET> for more, q to quit, c to continue without paging--c
        moduleName = <optimized out>
        className = <optimized out>
        PyId___module__ = <optimized out>
        dot = <optimized out>
        _py_xdecref_tmp = <optimized out>
        _py_decref_tmp = <optimized out>
        _py_decref_tmp = <optimized out>
        s = <optimized out>
        _py_xdecref_tmp = <optimized out>
        _py_decref_tmp = <optimized out>
        _py_xdecref_tmp = <optimized out>
        _py_decref_tmp = <optimized out>
        _py_decref_tmp = <optimized out>
#7  print_exception_recursive (f=<_io.TextIOWrapper at remote 0x7fbac03ec708>, value=<XMLSyntaxError(error_log=<lxml.etree._ListErrorLog at remote 0x7fbabfae11d8>, code=1) at remote 0x7fbabfb43d38>, seen=<optimized out>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Python/pythonrun.c:901
        err = 0
        res = <optimized out>
        cause = <optimized out>
        context = <optimized out>
#8  0x00007fbac0cbceeb in PyErr_Display (exception=<optimized out>, value=<XMLSyntaxError(error_log=<lxml.etree._ListErrorLog at remote 0x7fbabfae11d8>, code=1) at remote 0x7fbabfb43d38>, tb=<optimized out>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Python/pythonrun.c:935
        seen = {140440056905016}
        f = <_io.TextIOWrapper at remote 0x7fbac03ec708>
#9  0x00007fbac0cbcf6c in sys_excepthook (self=self@entry=<module at remote 0x7fbac0432f98>, args=args@entry=(<type at remote 0x560bb3d57208>, <XMLSyntaxError(error_log=<lxml.etree._ListErrorLog at remote 0x7fbabfae11d8>, code=1) at remote 0x7fbabfb43d38>, <traceback at remote 0x7fbabfae0608>)) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Python/sysmodule.c:321
        exc = <type at remote 0x560bb3d57208>
        value = <XMLSyntaxError(error_log=<lxml.etree._ListErrorLog at remote 0x7fbabfae11d8>, code=1) at remote 0x7fbabfb43d38>
        tb = <traceback at remote 0x7fbabfae0608>
#10 0x00007fbac0bc7ef3 in _PyMethodDef_RawFastCallDict (method=<optimized out>, self=<optimized out>, args=<optimized out>, nargs=<optimized out>, kwargs=<optimized out>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/call.c:518
        argstuple = (<type at remote 0x560bb3d57208>, <XMLSyntaxError(error_log=<lxml.etree._ListErrorLog at remote 0x7fbabfae11d8>, code=1) at remote 0x7fbabfb43d38>, <traceback at remote 0x7fbabfae0608>)
        meth = <optimized out>
        flags = 1
        result = 0x0
#11 0x00007fbac0bc8150 in _PyCFunction_FastCallDict (func=<built-in method excepthook of module object at remote 0x7fbac0432f98>, args=<optimized out>, nargs=<optimized out>, kwargs=<optimized out>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/call.c:582
        result = <optimized out>
#12 0x00007fbac0cd073e in PyErr_PrintEx (set_sys_last_vars=1) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Python/pythonrun.c:679
        stack = {<type at remote 0x560bb3d57208>, <XMLSyntaxError(error_log=<lxml.etree._ListErrorLog at remote 0x7fbabfae11d8>, code=1) at remote 0x7fbabfb43d38>, <traceback at remote 0x7fbabfae0608>}
        result = <optimized out>
        exception = <type at remote 0x560bb3d57208>
        v = <XMLSyntaxError(error_log=<lxml.etree._ListErrorLog at remote 0x7fbabfae11d8>, code=1) at remote 0x7fbabfb43d38>
        tb = <traceback at remote 0x7fbabfae0608>
        hook = <optimized out>
        exception = <optimized out>
        v = <optimized out>
        tb = <optimized out>
        hook = <optimized out>
        stack = <optimized out>
        result = <optimized out>
        exception2 = <optimized out>
        v2 = <optimized out>
        tb2 = <optimized out>
        _py_decref_tmp = <optimized out>
        _py_decref_tmp = <optimized out>
        _py_xdecref_tmp = <optimized out>
        _py_decref_tmp = <optimized out>
        _py_xdecref_tmp = <optimized out>
        _py_decref_tmp = <optimized out>
        _py_xdecref_tmp = <optimized out>
        _py_decref_tmp = <optimized out>
        _py_xdecref_tmp = <optimized out>
        _py_decref_tmp = <optimized out>
        _py_xdecref_tmp = <optimized out>
        _py_decref_tmp = <optimized out>
#13 PyErr_PrintEx (set_sys_last_vars=<optimized out>, set_sys_last_vars=<optimized out>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Python/pythonrun.c:641
        exception = <optimized out>
        v = <optimized out>
        tb = <optimized out>
        hook = <optimized out>
        _py_decref_tmp = <optimized out>
        _py_decref_tmp = <optimized out>
        _py_decref_tmp = <optimized out>
        _py_decref_tmp = <optimized out>
        _py_xdecref_tmp = <optimized out>
        _py_decref_tmp = <optimized out>
        _py_xdecref_tmp = <optimized out>
        _py_xdecref_tmp = <optimized out>
        _py_xdecref_tmp = <optimized out>
        _py_xdecref_tmp = <optimized out>
        _py_decref_tmp = <optimized out>
        _py_decref_tmp = <optimized out>
        exception2 = <optimized out>
        v2 = <optimized out>
        tb2 = <optimized out>
        stack = <optimized out>
        result = <optimized out>
        exception = <optimized out>
        v = <optimized out>
        tb = <optimized out>
        hook = <optimized out>
        _py_xdecref_tmp = <optimized out>
        _py_xdecref_tmp = <optimized out>
        _py_xdecref_tmp = <optimized out>
        _py_xdecref_tmp = <optimized out>
        _py_decref_tmp = <optimized out>
        _py_decref_tmp = <optimized out>
        exception2 = <optimized out>
        v2 = <optimized out>
        tb2 = <optimized out>
        stack = <optimized out>
        result = <optimized out>
        _py_xdecref_tmp = <optimized out>
        _py_decref_tmp = <optimized out>
        _py_xdecref_tmp = <optimized out>
        _py_decref_tmp = <optimized out>
        _py_xdecref_tmp = <optimized out>
        _py_decref_tmp = <optimized out>
#14 0x00007fbac0b703a3 in PyRun_SimpleFileExFlags (fp=<optimized out>, filename=<optimized out>, closeit=<optimized out>, flags=0x7ffd1669c540) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Python/pythonrun.c:435
        m = 0x0
        d = {'__name__': '__main__', '__doc__': None, '__package__': None, '__loader__': <SourceFileLoader(name='__main__', path='sf.py') at remote 0x7fbac03385c0>, '__spec__': None, '__annotations__': {}, '__builtins__': <module at remote 0x7fbac0438c28>, '__file__': 'sf.py', '__cached__': None, 'etree': <module at remote 0x7fbac02b1688>, 'xmlsec': <module at remote 0x7fbabfad4228>, 'xml1': <lxml.etree._ElementTree at remote 0x7fbabfae0388>}
        v = <optimized out>
        ext = <optimized out>
        set_file_name = 1
        ret = -1
        len = <optimized out>
        _py_decref_tmp = <optimized out>
#15 0x00007fbac0cd27f1 in pymain_run_file (p_cf=0x7ffd1669c540, filename=0x560bb3c86df0 L"sf.py", fp=0x560bb3ce8ef0) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Modules/main.c:427
        unicode = <optimized out>
        bytes = b'sf.py'
        filename_str = <optimized out>
        run = <optimized out>
        unicode = <optimized out>
        bytes = <optimized out>
        filename_str = <optimized out>
        run = <optimized out>
        _py_decref_tmp = <optimized out>
        _py_xdecref_tmp = <optimized out>
        _py_decref_tmp = <optimized out>
#16 pymain_run_filename (cf=0x7ffd1669c540, pymain=0x7ffd1669c650) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Modules/main.c:1627
        fp = 0x560bb3ce8ef0
        fp = <optimized out>
#17 pymain_run_python (pymain=0x7ffd1669c650) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Modules/main.c:2876
        cf = {cf_flags = 0}
        cf = <optimized out>
#18 pymain_main (pymain=<optimized out>, pymain=<optimized out>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Modules/main.c:3037
        res = <optimized out>
        res = <optimized out>
#19 0x00007fbac0cd2a2c in _Py_UnixMain (argc=<optimized out>, argv=<optimized out>) at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Modules/main.c:3072
        pymain = {argc = 2, use_bytes_argv = 1, bytes_argv = 0x7ffd1669c7a8, wchar_argv = 0x0, status = 0, err = {prefix = 0x0, msg = 0x0, user_err = 0}, stdin_is_interactive = 1, skip_first_line = 0, filename = 0x560bb3c86df0 L"sf.py", command = 0x0, module = 0x0, main_importer_path = 0x0}
#20 0x00007fbac0740413 in __libc_start_main (main=0x560bb3778050 <main>, argc=2, argv=0x7ffd1669c7a8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffd1669c798) at ../csu/libc-start.c:308
        self = <optimized out>
        result = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {0, -2611185483236398270, 94608255582304, 140724979484576, 0, 0, -8633824211270784190, -8601362210542343358}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x7ffd1669c7c0, 0x7fbac0e42150}, data = {prev = 0x0, cleanup = 0x0, canceltype = 376031168}}}
        not_first_call = <optimized out>
#21 0x0000560bb377808e in _start ()
Comment 4 Jan Pazdziora (Red Hat) 2019-01-11 15:15:10 UTC 
When I change the second etree.parse call to

xml2 = etree.parse("sf1.xml")

to try to open nonexistent file, I still get the segfault, even if I'd expect something like

<lxml.etree._ElementTree object at 0x7f29e4e65e08>
Traceback (most recent call last):
  File "sf.py", line 5, in <module>
    xml2 = etree.parse("sf1.xml")
  File "src/lxml/etree.pyx", line 3426, in lxml.etree.parse
  File "src/lxml/parser.pxi", line 1840, in lxml.etree._parseDocument
  File "src/lxml/parser.pxi", line 1866, in lxml.etree._parseDocumentFromURL
  File "src/lxml/parser.pxi", line 1770, in lxml.etree._parseDocFromFile
  File "src/lxml/parser.pxi", line 1163, in lxml.etree._BaseParser._parseDocFromFile
  File "src/lxml/parser.pxi", line 601, in lxml.etree._ParserContext._handleParseResultDoc
  File "src/lxml/parser.pxi", line 711, in lxml.etree._handleParseResult
  File "src/lxml/parser.pxi", line 638, in lxml.etree._raiseParseError
OSError: Error reading file 'sf1.xml': failed to load external entity "sf1.xml"

So one issue is why that second etree.parse on perfectly valid XML (as described in comment 0) fails and causes the exception.

But second issue is that when xmlsec is imported, python itself is not able to display that

OSError: Error reading file 'sf1.xml': failed to load external entity "sf1.xml"

message at all.
Comment 5 Jan Pazdziora (Red Hat) 2019-01-11 15:22:16 UTC 
I've now filed bug 1665490 for the second issue, python failing to show the exception message. This bugzilla stays for the problem of parsing multiple files, which should not produce any exception.
Comment 6 Jan Pazdziora (Red Hat) 2019-01-11 15:45:16 UTC 
When I try to run the code with python2, with

python2-lxml-4.2.5-1.fc29.x86_64
python2-xmlsec-1.3.3-3.fc29.x86_64

I get

# python2 sf.py
<lxml.etree._ElementTree object at 0x7fa2f910d098>
Traceback (most recent call last):
  File "sf.py", line 5, in <module>
    xml2 = etree.parse("sf.xml")
  File "src/lxml/etree.pyx", line 3426, in lxml.etree.parse
  File "src/lxml/parser.pxi", line 1840, in lxml.etree._parseDocument
  File "src/lxml/parser.pxi", line 1866, in lxml.etree._parseDocumentFromURL
  File "src/lxml/parser.pxi", line 1770, in lxml.etree._parseDocFromFile
  File "src/lxml/parser.pxi", line 1163, in lxml.etree._BaseParser._parseDocFromFile
  File "src/lxml/parser.pxi", line 601, in lxml.etree._ParserContext._handleParseResultDoc
  File "src/lxml/parser.pxi", line 711, in lxml.etree._handleParseResult
  File "src/lxml/parser.pxi", line 651, in lxml.etree._raiseParseError
  File "sf.xml", line 0
lxml.etree.XMLSyntaxError
Comment 7 Jan Pazdziora (Red Hat) 2019-01-11 15:47:17 UTC 
In bug 1665490 comment 1, Petr suggests that the segfault under python 3 is caused by the fact that the filename is passed to the exception as bytes object, rather than unicode string. If xmlsec is able to do that, maybe it also modifies other string behaviours, even under python 2.
Comment 8 Jeremy Cline 2019-01-11 15:53:39 UTC 
Hi Jan,

My recommendation would be to ensure you can reproduce this with pip-install packages and report it upstream. I might have time to look at it on the weekend, but it looks like upstream is responsive and you've got a super simple reproducer so that should help.
Comment 9 Jan Pazdziora (Red Hat) 2019-01-11 16:23:17 UTC 
Thanks for the suggestion. I've reproduced it with pip3 and reported upstream as https://github.com/mehcode/python-xmlsec/issues/96.
Comment 10 Jan Pazdziora (Red Hat) 2019-09-25 12:26:22 UTC 
I see the segfault on Fedora 30 and 31 as well.
Comment 11 Ben Cotton 2019-10-31 18:48:57 UTC 
This message is a reminder that Fedora 29 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 29 on 2019-11-26.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '29'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 29 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.
Comment 12 Jan Pazdziora (Red Hat) 2019-11-01 09:23:16 UTC 
Updating version per comment 10.
Comment 13 Jan Pazdziora (Red Hat) 2020-03-24 09:51:31 UTC 
With bug 1665490 fixed in Fedora 30, the segfault is now gone but traceback stays:

<lxml.etree._ElementTree object at 0x7ff2b5a8cb40>
Traceback (most recent call last):
  File "sf.py", line 5, in <module>
    xml2 = etree.parse("sf.xml")
  File "src/lxml/etree.pyx", line 3426, in lxml.etree.parse
  File "src/lxml/parser.pxi", line 1840, in lxml.etree._parseDocument
  File "src/lxml/parser.pxi", line 1866, in lxml.etree._parseDocumentFromURL
  File "src/lxml/parser.pxi", line 1770, in lxml.etree._parseDocFromFile
  File "src/lxml/parser.pxi", line 1163, in lxml.etree._BaseParser._parseDocFromFile
  File "src/lxml/parser.pxi", line 601, in lxml.etree._ParserContext._handleParseResultDoc
  File "src/lxml/parser.pxi", line 711, in lxml.etree._handleParseResult
  File "src/lxml/parser.pxi", line 651, in lxml.etree._raiseParseError
  File "b'sf.xml'", line 0
lxml.etree.XMLSyntaxError

This is with

python3-3.7.6-2.fc30.x86_64
python3-xmlsec-1.3.3-5.fc30.x86_64
python3-lxml-4.2.5-2.fc30.x86_64
Comment 14 Ben Cotton 2020-04-30 20:19:08 UTC 
This message is a reminder that Fedora 30 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 30 on 2020-05-26.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '30'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 30 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.
Comment 15 Fedora Admin user for bugzilla script actions 2020-05-13 02:48:24 UTC 
This package has changed maintainer in the Fedora.
Reassigning to the new maintainer of this component.
Comment 16 Jan Pazdziora (Red Hat) 2020-05-13 07:41:14 UTC 
On Fedora 31 with

python3-3.7.7-1.fc31.x86_64
python3-xmlsec-1.3.3-6.fc31.x86_64
python3-lxml-4.4.0-1.fc31.x86_64

the traceback is still there:

Traceback (most recent call last):
  File "sf.py", line 3, in <module>
    xml1 = etree.parse("sf.xml")
  File "src/lxml/etree.pyx", line 3467, in lxml.etree.parse
  File "src/lxml/parser.pxi", line 1839, in lxml.etree._parseDocument
  File "src/lxml/parser.pxi", line 1865, in lxml.etree._parseDocumentFromURL
  File "src/lxml/parser.pxi", line 1769, in lxml.etree._parseDocFromFile
  File "src/lxml/parser.pxi", line 1163, in lxml.etree._BaseParser._parseDocFromFile
  File "src/lxml/parser.pxi", line 601, in lxml.etree._ParserContext._handleParseResultDoc
  File "src/lxml/parser.pxi", line 711, in lxml.etree._handleParseResult
  File "src/lxml/parser.pxi", line 640, in lxml.etree._raiseParseError
  File "sf.xml", line 1
lxml.etree.XMLSyntaxError: Extra content at the end of the document, line 1, column 8
Comment 17 Jan Pazdziora (Red Hat) 2020-05-13 07:43:32 UTC 
Oops, sorry, I had some garbled sf.xml content. The traceback on Fedora 31 matches the one on Fedora 30:

<lxml.etree._ElementTree object at 0x7f4cbaf9ba50>
Traceback (most recent call last):
  File "sf.py", line 5, in <module>
    xml2 = etree.parse("sf.xml")
  File "src/lxml/etree.pyx", line 3467, in lxml.etree.parse
  File "src/lxml/parser.pxi", line 1839, in lxml.etree._parseDocument
  File "src/lxml/parser.pxi", line 1865, in lxml.etree._parseDocumentFromURL
  File "src/lxml/parser.pxi", line 1769, in lxml.etree._parseDocFromFile
  File "src/lxml/parser.pxi", line 1163, in lxml.etree._BaseParser._parseDocFromFile
  File "src/lxml/parser.pxi", line 601, in lxml.etree._ParserContext._handleParseResultDoc
  File "src/lxml/parser.pxi", line 711, in lxml.etree._handleParseResult
  File "src/lxml/parser.pxi", line 651, in lxml.etree._raiseParseError
  File "b'sf.xml'", line 0
lxml.etree.XMLSyntaxError
Comment 18 Jan Pazdziora (Red Hat) 2020-05-21 15:52:20 UTC 
Hello, upstream mentions in https://github.com/mehcode/python-xmlsec/issues/96#issuecomment-632123881 that latest python-xmlsec should have the problem resolved. Could you make a (scratch) build so that we could test it in Fedora?
Comment 19 Oleg Höfling 2020-05-22 11:51:58 UTC 
Nope, the issue is not fixed - sorry for the unnecessary noise. I've tested with the lxml manylinux wheel from PyPI, not with the distro package.
Comment 20 Martin Kutlak 2020-06-01 17:36:11 UTC 
(In reply to Jan Pazdziora from comment #18)
> Hello, upstream mentions in
> https://github.com/mehcode/python-xmlsec/issues/96#issuecomment-632123881
> that latest python-xmlsec should have the problem resolved. Could you make a
> (scratch) build so that we could test it in Fedora?

Hi, the latest python-xmlsec upstream release (1.3.8) is now available in rawhide.

As Oleg mentioned in the GitHub issue it seems to be an issue in lxml [1]. 

Reassigning to python-lxml component.


[1] https://bugs.launchpad.net/lxml/+bug/1880251
Comment 21 Igor Raits 2020-06-01 18:15:39 UTC 
Fixed in python-lxml-4.5.1-1.fc33.
Comment 22 Oleg Höfling 2020-07-09 10:21:55 UTC 
Please note that the issue is currently only partially resolved on lxml side. The code where lxml is imported after xmlsec:

```python
import xmlsec
from lxml import etree

etree.parse('doc.xml')
etree.parse('doc.xml')
```

will continue to fail with python3-lxml-4.5.1-1.fc33 (also reported in https://bugs.launchpad.net/lxml/+bug/1880251/comments/3).