Bug 1665490
| Summary: | Importing xmlsec causes segfault when exception message should be displayed [wait for Python 3.7.5] | ||
|---|---|---|---|
| Product: | [Fedora] Fedora | Reporter: | Jan Pazdziora (Red Hat) <jpazdziora> |
| Component: | python3 | Assignee: | Victor Stinner <vstinner> |
| Status: | CLOSED ERRATA | QA Contact: | Fedora Extras Quality Assurance <extras-qa> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | low | ||
| Version: | 29 | CC: | cstratak, dmalcolm, extras-qa, jeremy, jpazdziora, mcyprian, mhroncok, pviktori, rkuska, shcherbina.iryna, tomspur, torsava |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | python3-3.7.5-1.fc30 python3-3.7.5-1.fc29 python3-3.7.5-1.fc31 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | 1665459 | Environment: | |
| Last Closed: | 2019-11-02 01:12:40 UTC | Type: | --- |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | 1665459 | ||
| Bug Blocks: | |||
|
Description
Jan Pazdziora (Red Hat)
2019-01-11 15:17:47 UTC
Let's make this bugzilla the one for the problem of Objects/unicodeobject.c with code
int
_PyUnicode_Ready(PyObject *unicode)
{
wchar_t *end;
Py_UCS4 maxchar = 0;
Py_ssize_t num_surrogates;
#if SIZEOF_WCHAR_T == 2
Py_ssize_t length_wo_surrogates;
#endif
/* _PyUnicode_Ready() is only intended for old-style API usage where
strings were created using _PyObject_New() and where no canonical
representation (the str field) has been set yet aka strings
which are not yet ready. */
assert(_PyUnicode_CHECK(unicode));
assert(_PyUnicode_KIND(unicode) == PyUnicode_WCHAR_KIND);
assert(_PyUnicode_WSTR(unicode) != NULL);
assert(_PyUnicode_DATA_ANY(unicode) == NULL);
assert(_PyUnicode_UTF8(unicode) == NULL);
/* Actually, it should neither be interned nor be anything else: */
assert(_PyUnicode_STATE(unicode).interned == SSTATE_NOT_INTERNED);
end = _PyUnicode_WSTR(unicode) + _PyUnicode_WSTR_LENGTH(unicode);
if (find_maxchar_surrogates(_PyUnicode_WSTR(unicode), end,
&maxchar, &num_surrogates) == -1)
return -1;
turning that unicode parameter into something with end = 0xfffffffffffffffd on which find_maxchar_surrogates fails.
#1 _PyUnicode_Ready (unicode=b'sf.xml') at /usr/src/debug/python3-3.7.2-1.fc29.x86_64/Objects/unicodeobject.c:1699
end = 0xfffffffffffffffd <error: Cannot access memory at address 0xfffffffffffffffd>
maxchar = 0
num_surrogates = 0
The original bug 1665459 would then be for the original issue -- why can't lxml load things twice.
#1 _PyUnicode_Ready (unicode=b'sg.xml') Somehow, a bytes object, rather than a (Unicode) string, is used as the filename in a backtrace. That's an entirely different datatype at the C level. I don't think Python would be setting b'sg.xml' as the filename of running code, but I don't know enough about lxml or xmlsec to tell why/where they would do it. This is currently assigned to python-maint, and we have it in low priority. If you (anyone) can get to it faster than we can, please take the bug. I reported this bug upstream and I proposed a fix: * https://bugs.python.org/issue37467 * https://github.com/python/cpython/pull/14504 I consider that this low priority bug can wait for the next Python 3.7.4 release (scheduled for next days). FEDORA-2019-9bfb4a3e4b has been submitted as an update to Fedora 30. https://bodhi.fedoraproject.org/updates/FEDORA-2019-9bfb4a3e4b FEDORA-2019-60a1defcd1 has been submitted as an update to Fedora 29. https://bodhi.fedoraproject.org/updates/FEDORA-2019-60a1defcd1 python3-3.7.4-1.fc30, python3-docs-3.7.4-1.fc30 has been pushed to the Fedora 30 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-9bfb4a3e4b python3-3.7.4-1.fc29, python3-docs-3.7.4-1.fc29 has been pushed to the Fedora 29 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-60a1defcd1 $ I don't see the issue fixed with python3-3.7.4-1.fc30.x86_64, I'm still getting
python3 sf.py
<lxml.etree._ElementTree object at 0x7f80cc91d640>
Traceback (most recent call last):
File "sf.py", line 5, in <module>
xml2 = etree.parse("sf1.xml")
File "src/lxml/etree.pyx", line 3426, in lxml.etree.parse
File "src/lxml/parser.pxi", line 1840, in lxml.etree._parseDocument
File "src/lxml/parser.pxi", line 1866, in lxml.etree._parseDocumentFromURL
File "src/lxml/parser.pxi", line 1770, in lxml.etree._parseDocFromFile
File "src/lxml/parser.pxi", line 1163, in lxml.etree._BaseParser._parseDocFromFile
File "src/lxml/parser.pxi", line 601, in lxml.etree._ParserContext._handleParseResultDoc
File "src/lxml/parser.pxi", line 711, in lxml.etree._handleParseResult
File "src/lxml/parser.pxi", line 651, in lxml.etree._raiseParseError
Segmentation fault (core dumped)
Sadly, Python 3.7 release manager decided to not pick my fix into 3.7.4 final release :-( https://github.com/python/cpython/commit/8cbffc4d96d1da0fbc38da6f34f2da30c5ffd601 The fix will be part of next 3.7.5 release scheduled for the end of next September ("3.7.5 final: 2019-09-30 (expected)" says the PEP 537). 3.7.5 has been updated. Could you check if it fixes the problem? I tested the python37 in Rawhide (Python 3.7.5): the script doesn't crash anymore.
sudo dnf install -y xmlsec1 xmlsec1-devel libtool-ltdl-devel
python3.7 -m venv env
env/bin/python -m pip install lxml xmlsec
env/bin/python sf.py
Output:
func=xmlSecCryptoDLLibraryCreate:file=dl.c:line=135:obj=unknown:subj=lt_dlopenext:error=7:io function failed:name="libxmlsec1-openssl"; errno=2
func=xmlSecCryptoDLGetLibraryFunctions:file=dl.c:line=442:obj=unknown:subj=xmlSecCryptoDLLibraryCreate:error=1:xmlsec library function failed:crypto=openssl
func=xmlSecCryptoDLLoadLibrary:file=dl.c:line=400:obj=unknown:subj=xmlSecCryptoDLGetLibraryFunctions:error=1:xmlsec library function failed:
Traceback (most recent call last):
File "sf.py", line 3, in <module>
import xmlsec
SystemError: null argument to internal routine
FEDORA-2019-232f092db0 has been submitted as an update to Fedora 31. https://bodhi.fedoraproject.org/updates/FEDORA-2019-232f092db0 FEDORA-2019-aba3cca74a has been submitted as an update to Fedora 30. https://bodhi.fedoraproject.org/updates/FEDORA-2019-aba3cca74a FEDORA-2019-986622833f has been submitted as an update to Fedora 29. https://bodhi.fedoraproject.org/updates/FEDORA-2019-986622833f This message is a reminder that Fedora 29 is nearing its end of life. Fedora will stop maintaining and issuing updates for Fedora 29 on 2019-11-26. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a Fedora 'version' of '29'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 29 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete. Confirming the fix in Fedora 29 (container) as well, with python3-3.7.5-1.fc29.x86_64. python3-3.7.5-1.fc30 has been pushed to the Fedora 30 stable repository. If problems still persist, please make note of it in this bug report. python3-3.7.5-1.fc29 has been pushed to the Fedora 29 stable repository. If problems still persist, please make note of it in this bug report. python3-3.7.5-1.fc31 has been pushed to the Fedora 31 stable repository. If problems still persist, please make note of it in this bug report. python3-3.7.5-1.fc30 has been pushed to the Fedora 30 stable repository. If problems still persist, please make note of it in this bug report. |