Bug 1030830

The reason why this fails is, that the debug build dynload_shlib always looks for module_d.so, instead of just module.so - and there is no multiarray_d.so.

The patch that helps us build the python-debug version comes from Debian and has been adapted by Dave Malcolm in the way to _not_ fall back to multiarray.so, unlike the original Debian patch. According to a comment in python.spec:

#   * Debian's patch to dynload_shlib.c looks for module_d.so, then module.so,
# but this can potentially find a module built against the wrong DSO ABI.  We
# instead search for just module_d.so in a debug build

I don't totally understand this - Dave, could you please explain what you mean by "wrong DSO ABI"?  Does that mean that you wanted to make this bulletproof against cases where Python 2.6 "module.so" would be present and Python 2.7 would try to load "module_d.so"? Some different concerns?

Thanks for the response! Do you know of any workaround (hack) I can use to make this work for me?

I tried to copy multiarray.so to multiarray_d.so, but that also fails:

[miha@localhost ~]$ sudo cp /usr/lib64/python2.7/site-packages/numpy/core/multiarray.so /usr/lib64/python2.7/site-packages/numpy/core/multiarray_d.so
[miha@localhost ~]$ python-debug
Python 2.7.5 (default, Nov 12 2013, 16:45:15) 
[GCC 4.8.2 20131017 (Red Hat 4.8.2-1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
python-debug: /builddir/build/BUILD/Python-2.7.5/Objects/object.c:65: _Py_AddToAllObjects: Assertion `(op->_ob_prev == ((void *)0)) == (op->_ob_next == ((void *)0))' failed.
Aborted (core dumped)

(In reply to Bohuslav "Slavek" Kabrda from comment #1)
> The reason why this fails is, that the debug build dynload_shlib always
> looks for module_d.so, instead of just module.so - and there is no
> multiarray_d.so.
> 
> The patch that helps us build the python-debug version comes from Debian and
> has been adapted by Dave Malcolm in the way to _not_ fall back to
> multiarray.so, unlike the original Debian patch. According to a comment in
> python.spec:
> 
> #   * Debian's patch to dynload_shlib.c looks for module_d.so, then
> module.so,
> # but this can potentially find a module built against the wrong DSO ABI.  We
> # instead search for just module_d.so in a debug build
> 
> I don't totally understand this - Dave, could you please explain what you
> mean by "wrong DSO ABI"? 

The layout of the PyObject struct is different between the optimized and the debug build: the debug build adds two extra pointers near the top (_ob_next and _ob_prev), before the ob_refcnt.  There are also some changes to PyTypeObject.  See object.h, and look for #ifdef Py_TRACE_REFS.

If the debug libpython tries to use a module built for the optimized libpython (or vice versa), it will likely immediately segfault - things won't be in memory in their expected locations: even changing the reference count will be accessing the wrong memory location.

> Does that mean that you wanted to make this
> bulletproof against cases where Python 2.6 "module.so" would be present and
> Python 2.7 would try to load "module_d.so"? Some different concerns?
More about bulletproofing optimized vs debug within a release e.g. 2.7 debug vs 2.7 optimized.  As noted above, 2.7 debug modules aren't compatible with 2.7 optimized libpython.

(In reply to Miha from comment #2)
> Thanks for the response! Do you know of any workaround (hack) I can use to
> make this work for me?
> 
> I tried to copy multiarray.so to multiarray_d.so, but that also fails:
> 
> [miha@localhost ~]$ sudo cp
> /usr/lib64/python2.7/site-packages/numpy/core/multiarray.so
> /usr/lib64/python2.7/site-packages/numpy/core/multiarray_d.so
> [miha@localhost ~]$ python-debug
> Python 2.7.5 (default, Nov 12 2013, 16:45:15) 
> [GCC 4.8.2 20131017 (Red Hat 4.8.2-1)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import numpy
> python-debug: /builddir/build/BUILD/Python-2.7.5/Objects/object.c:65:
> _Py_AddToAllObjects: Assertion `(op->_ob_prev == ((void *)0)) ==
> (op->_ob_next == ((void *)0))' failed.
> Aborted (core dumped)

Yes: simply copying them won't work (for the reasons noted in comment #3.  I had expected a segfault, but you got a C-level assertion failure instead.  In any case it's not going to work that way).

The modules need to be rebuilt against the correct configuration of python to be usable.   The fix may be to update how we package numpy, to build against both configurations, providing a -debug subpackage - though I'm not the numpy/python maintainer.

Thanks for the detailed info, Dave!
Miha, I'd probably suggest asking maintainer of numpy to create a debug build (maybe reassign this bug). A problem might be that there is no standardized way of building debug builds of extension packages.
I'm currently pretty occupied, but I'll try to take this to FPC before end of the week to get extension debug builds standardized in some way.

Thanks for the help! I opened a new bug report for numpy: https://bugzilla.redhat.com/show_bug.cgi?id=1031998

Instructions as to how to build a debug version of numpy (or other python extensions) would be useful.  I have no idea how to go about it.

This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

This message is a reminder that Fedora 20 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 20. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '20'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 20 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Fedora 20 changed to end-of-life (EOL) status on 2015-06-23. Fedora 20 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.