Bug 1544019 - lxml crashes when mixed with libvirt which uses libxml2
Summary: lxml crashes when mixed with libvirt which uses libxml2
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Fedora
Classification: Fedora
Component: python-lxml
Version: 27
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Igor Gnatenko
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-02-09 21:36 UTC by Cole Robinson
Modified: 2018-02-23 19:08 UTC (History)
3 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2018-02-23 19:08:38 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Cole Robinson 2018-02-09 21:36:10 UTC
$ cat test.py
import os
import libvirt
import lxml.etree
libvirt.open("test://%s/testdriver.xml" % os.getcwd())

$ wget https://raw.githubusercontent.com/virt-manager/virt-manager/master/tests/testdriver.xml
$ python test.py
Segmentation fault (core dumped)

Backtrace is:

#0 __pyx_f_4lxml_5etree__local_resolver (__pyx_v_c_url=0x55555578e630 "/home/crobinso/src/virt-manager/testdriver.xml", __pyx_v_c_pubid=0x0, __pyx_v_c_context=0x55555595ea30) at src/lxml/etree.c:40478
#1 0x00007fffee17d88d in xmlLoadExternalEntity () from /lib64/libxml2.so.2
#2 0x00007fffee16a767 in xmlCtxtReadFile () from /lib64/libxml2.so.2
#3 0x00007fffef6d1494 in virXMLParseHelper () from /lib64/libvirt.so.0
#4 0x00007fffef7b6281 in testConnectOpen () from /lib64/libvirt.so.0
#5 0x00007fffef76eb3f in virConnectOpenInternal () from /lib64/libvirt.so.0
#6 0x00007fffef76fd70 in virConnectOpen () from /lib64/libvirt.so.0
#7 0x00007fffefc46470 in libvirt_virConnectOpen () from /usr/lib64/python2.7/site-packages/libvirtmod.so
#8 0x00007ffff7afe62e in PyEval_EvalFrameEx () from /lib64/libpython2.7.so.1.0
#9 0x00007ffff7aff288 in PyEval_EvalCodeEx () from /lib64/libpython2.7.so.1.0
#10 0x00007ffff7afc283 in PyEval_EvalFrameEx () from /lib64/libpython2.7.so.1.0
#11 0x00007ffff7aff288 in PyEval_EvalCodeEx () from /lib64/libpython2.7.so.1.0
#12 0x00007ffff7aff499 in PyEval_EvalCode () from /lib64/libpython2.7.so.1.0
#13 0x00007ffff7b0578f in run_mod () from /lib64/libpython2.7.so.1.0
#14 0x00007ffff7b0573a in PyRun_FileExFlags () from /lib64/libpython2.7.so.1.0
#15 0x00007ffff7b0562e in PyRun_SimpleFileExFlags () from /lib64/libpython2.7.so.1.0
#16 0x00007ffff7b0b8ce in Py_Main () from /lib64/libpython2.7.so.1.0
#17 0x00007ffff6c5900a in __libc_start_main () from /lib64/libc.so.6
#18 0x000055555555478a in _start ()


$ rpm -q python2-lxml python2-libvirt python2-libxml2
python2-lxml-4.1.1-1.fc27.x86_64
python2-libvirt-3.7.0-1.fc27.x86_64
python2-libxml2-2.9.7-1.fc27.x86_64

Reproduced with python3 as well, also tested python2 with 3.7.2 and 4.0.0 fedora packages. But here's the interesting bit, using python2 lxml from pip I couldn't reproduce this issue, so I wonder if this is something distro specific.

Comment 1 Cole Robinson 2018-02-09 23:04:06 UTC
to be a bit more clear, libvirt does not use lxml, it directly uses libxml2 from C code. dropping the lxml.etree import makes the crash go away and things work correctly. I reproduced this manual lxml builds on fedora and even with an f24 VM so this issue appears to be around for a while

Comment 2 Cole Robinson 2018-02-13 19:38:51 UTC
lxml devs pointed to this:

http://lxml.de/installation.html#using-lxml-with-python-libxml2


Which makes it sound like the only way to safely use lxml in a general purpose way is if lxml is statically compiled against libxml2 and doesn't use system versions. This is what pypi lxml builds do. I'm sure this isn't acceptable by fedora packaging standards, also doesn't look like debian does that either

Comment 3 Cole Robinson 2018-02-23 19:08:38 UTC
FWIW here's the lxml bug that was closed: https://bugs.launchpad.net/lxml/+bug/1748019

I'm closing this too since I don't think lxml will end up statically compiled in fedora, and I think we can work around this in libvirt


Note You need to log in before you can comment on or make changes to this bug.