User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:25.0) Gecko/20100101 Firefox/25.0 Build Identifier: When I'm using python-debug, I can't import numpy. If I use ordinary python interpreter, there is no problem. They both have the same sys.path. This problem happens both in Fedora 19 and in up to date vanilla (i.e. no additional packages installed except python-debug and numpy) Fedora 20. Here is the output from Fedora 20: [miha@localhost ~]$ python-debug Python 2.7.5 (default, Nov 12 2013, 16:45:15) [GCC 4.8.2 20131017 (Red Hat 4.8.2-1)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import numpy Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib64/python2.7/site-packages/numpy/__init__.py", line 153, in <module> from . import add_newdocs File "/usr/lib64/python2.7/site-packages/numpy/add_newdocs.py", line 13, in <module> from numpy.lib import add_newdoc File "/usr/lib64/python2.7/site-packages/numpy/lib/__init__.py", line 8, in <module> from .type_check import * File "/usr/lib64/python2.7/site-packages/numpy/lib/type_check.py", line 11, in <module> import numpy.core.numeric as _nx File "/usr/lib64/python2.7/site-packages/numpy/core/__init__.py", line 6, in <module> from . import multiarray ImportError: cannot import name multiarray [70322 refs] >>> Reproducible: Always
The reason why this fails is, that the debug build dynload_shlib always looks for module_d.so, instead of just module.so - and there is no multiarray_d.so. The patch that helps us build the python-debug version comes from Debian and has been adapted by Dave Malcolm in the way to _not_ fall back to multiarray.so, unlike the original Debian patch. According to a comment in python.spec: # * Debian's patch to dynload_shlib.c looks for module_d.so, then module.so, # but this can potentially find a module built against the wrong DSO ABI. We # instead search for just module_d.so in a debug build I don't totally understand this - Dave, could you please explain what you mean by "wrong DSO ABI"? Does that mean that you wanted to make this bulletproof against cases where Python 2.6 "module.so" would be present and Python 2.7 would try to load "module_d.so"? Some different concerns?
Thanks for the response! Do you know of any workaround (hack) I can use to make this work for me? I tried to copy multiarray.so to multiarray_d.so, but that also fails: [miha@localhost ~]$ sudo cp /usr/lib64/python2.7/site-packages/numpy/core/multiarray.so /usr/lib64/python2.7/site-packages/numpy/core/multiarray_d.so [miha@localhost ~]$ python-debug Python 2.7.5 (default, Nov 12 2013, 16:45:15) [GCC 4.8.2 20131017 (Red Hat 4.8.2-1)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import numpy python-debug: /builddir/build/BUILD/Python-2.7.5/Objects/object.c:65: _Py_AddToAllObjects: Assertion `(op->_ob_prev == ((void *)0)) == (op->_ob_next == ((void *)0))' failed. Aborted (core dumped)
(In reply to Bohuslav "Slavek" Kabrda from comment #1) > The reason why this fails is, that the debug build dynload_shlib always > looks for module_d.so, instead of just module.so - and there is no > multiarray_d.so. > > The patch that helps us build the python-debug version comes from Debian and > has been adapted by Dave Malcolm in the way to _not_ fall back to > multiarray.so, unlike the original Debian patch. According to a comment in > python.spec: > > # * Debian's patch to dynload_shlib.c looks for module_d.so, then > module.so, > # but this can potentially find a module built against the wrong DSO ABI. We > # instead search for just module_d.so in a debug build > > I don't totally understand this - Dave, could you please explain what you > mean by "wrong DSO ABI"? The layout of the PyObject struct is different between the optimized and the debug build: the debug build adds two extra pointers near the top (_ob_next and _ob_prev), before the ob_refcnt. There are also some changes to PyTypeObject. See object.h, and look for #ifdef Py_TRACE_REFS. If the debug libpython tries to use a module built for the optimized libpython (or vice versa), it will likely immediately segfault - things won't be in memory in their expected locations: even changing the reference count will be accessing the wrong memory location. > Does that mean that you wanted to make this > bulletproof against cases where Python 2.6 "module.so" would be present and > Python 2.7 would try to load "module_d.so"? Some different concerns? More about bulletproofing optimized vs debug within a release e.g. 2.7 debug vs 2.7 optimized. As noted above, 2.7 debug modules aren't compatible with 2.7 optimized libpython.
(In reply to Miha from comment #2) > Thanks for the response! Do you know of any workaround (hack) I can use to > make this work for me? > > I tried to copy multiarray.so to multiarray_d.so, but that also fails: > > [miha@localhost ~]$ sudo cp > /usr/lib64/python2.7/site-packages/numpy/core/multiarray.so > /usr/lib64/python2.7/site-packages/numpy/core/multiarray_d.so > [miha@localhost ~]$ python-debug > Python 2.7.5 (default, Nov 12 2013, 16:45:15) > [GCC 4.8.2 20131017 (Red Hat 4.8.2-1)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>> import numpy > python-debug: /builddir/build/BUILD/Python-2.7.5/Objects/object.c:65: > _Py_AddToAllObjects: Assertion `(op->_ob_prev == ((void *)0)) == > (op->_ob_next == ((void *)0))' failed. > Aborted (core dumped) Yes: simply copying them won't work (for the reasons noted in comment #3. I had expected a segfault, but you got a C-level assertion failure instead. In any case it's not going to work that way). The modules need to be rebuilt against the correct configuration of python to be usable. The fix may be to update how we package numpy, to build against both configurations, providing a -debug subpackage - though I'm not the numpy/python maintainer.
Thanks for the detailed info, Dave! Miha, I'd probably suggest asking maintainer of numpy to create a debug build (maybe reassign this bug). A problem might be that there is no standardized way of building debug builds of extension packages. I'm currently pretty occupied, but I'll try to take this to FPC before end of the week to get extension debug builds standardized in some way.
Thanks for the help! I opened a new bug report for numpy: https://bugzilla.redhat.com/show_bug.cgi?id=1031998
Instructions as to how to build a debug version of numpy (or other python extensions) would be useful. I have no idea how to go about it.
This package has changed ownership in the Fedora Package Database. Reassigning to the new owner of this component.
This message is a reminder that Fedora 20 is nearing its end of life. Approximately 4 (four) weeks from now Fedora will stop maintaining and issuing updates for Fedora 20. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a Fedora 'version' of '20'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 20 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete.
Fedora 20 changed to end-of-life (EOL) status on 2015-06-23. Fedora 20 is no longer maintained, which means that it will not receive any further security or bug fix updates. As a result we are closing this bug. If you can reproduce this bug against a currently maintained version of Fedora please feel free to reopen this bug against that version. If you are unable to reopen this bug, please file a new report against the current release. If you experience problems, please add a comment to this bug. Thank you for reporting this bug and we are sorry it could not be fixed.