Bug 684991

Seen with:
  rhythmbox-0.12.8-1.el6.x86_64
  python-2.6.6-9.el6.x86_64
with this backtrace:
(gdb) bt
#0  0x000000337ea67f9d in rb_python_module_load_with_gil (module=0x86d4c0) at rb-python-module.c:366
#1  0x0000003b6fc2d624 in g_type_module_use () from /lib64/libgobject-2.0.so.0
#2  0x000000337ea66814 in load_plugin_module (info=0x9a30d0) at rb-plugins-engine.c:492
#3  rb_plugins_engine_activate_plugin_real (info=0x9a30d0) at rb-plugins-engine.c:521
#4  rb_plugins_engine_activate_plugin (info=0x9a30d0) at rb-plugins-engine.c:541
#5  0x000000337ea670f5 in rb_plugins_engine_load_cb (file=<value optimized out>, dir=<value optimized out>, userdata=<value optimized out>) at rb-plugins-engine.c:316
#6  0x000000337ead194e in _uri_handle_recurse (dir=0x9833a0, cancel=0x0, handled=0x9552d0, func=0x337ea66cb0 <rb_plugins_engine_load_cb>, user_data=0x0) at rb-file-helpers.c:811
#7  0x000000337ead19bc in _uri_handle_recurse (dir=0x952220, cancel=0x0, handled=0x9552d0, func=0x337ea66cb0 <rb_plugins_engine_load_cb>, user_data=0x0) at rb-file-helpers.c:814
#8  0x000000337ead1a81 in rb_uri_handle_recursively (uri=<value optimized out>, cancel=0x0, func=0x337ea66cb0 <rb_plugins_engine_load_cb>, user_data=0x0) at rb-file-helpers.c:850
#9  0x000000337ea66ad1 in rb_plugins_engine_load_dir (shell=<value optimized out>) at rb-plugins-engine.c:329
#10 rb_plugins_engine_load_all (shell=<value optimized out>) at rb-plugins-engine.c:342
#11 rb_plugins_engine_init (shell=<value optimized out>) at rb-plugins-engine.c:375
#12 0x000000337ea4b222 in rb_shell_constructed (object=<value optimized out>) at rb-shell.c:1556
#13 0x0000003b6fc11f86 in g_object_newv () from /lib64/libgobject-2.0.so.0
#14 0x0000003b6fc12809 in g_object_new_valist () from /lib64/libgobject-2.0.so.0
#15 0x0000003b6fc12a4c in g_object_new () from /lib64/libgobject-2.0.so.0
#16 0x000000337ea47372 in rb_shell_new (no_registration=0, no_update=0, dry_run=0, b=0x0, playlists=0x0) at rb-shell.c:1115
#17 0x0000000000403972 in main ()

Crash is happening within pyg_gil_state_ensure:
   360  static gboolean
   361  rb_python_module_load_with_gil (GTypeModule *module)
   362  {
   363          PyGILState_STATE state;
   364          gboolean ret;
   365  
***366          state = pyg_gil_state_ensure ();
   367          ret = rb_python_module_load (module);
   368          pyg_gil_state_release (state);
   369  
   370          return ret;
   371  }
   372  

(This is on a 4-core machine)

/usr/include/pygtk-2.0/pygobject.h within pygobject2-devel-2.20.0-5.el6.x86_64 has:
  #define pyg_gil_state_ensure        (_PyGObject_API->gil_state_ensure)
  #define pyg_gil_state_release       (_PyGObject_API->gil_state_release)

Looks like this isn't set up yet, leading to a read through NULL:
  (gdb) p _PyGObject_API 
  $4 = (struct _PyGObject_Functions *) 0x0

oh I forgot I disabled a few plugins locally here, to get to where I crashed, but I suspect its the same problem.

*** Bug 672984 has been marked as a duplicate of this bug. ***

Works for me with python-2.6.5-3.el6.x86_64.rpm

Creating a hardware watchpoint:
  (gdb) watch _PyGObject_API
shows that this gets set up (with this version of python) here:

Old value = (struct _PyGObject_Functions *) 0x0
New value = (struct _PyGObject_Functions *) 0x7fffeddf3da0
rb_python_module_init_python () at rb-python-module.c:176
176		if (PyErr_Occurred ()) {
(gdb) bt
#0  rb_python_module_init_python () at rb-python-module.c:176
#1  0x000000337ea66a97 in rb_plugins_engine_init (shell=0x7521e0) at rb-plugins-engine.c:372
#2  0x000000337ea4b222 in rb_shell_constructed (object=<value optimized out>) at rb-shell.c:1556
#3  0x00007ffff76fdf86 in g_object_newv () from /lib64/libgobject-2.0.so.0
#4  0x00007ffff76fe809 in g_object_new_valist () from /lib64/libgobject-2.0.so.0
#5  0x00007ffff76fea4c in g_object_new () from /lib64/libgobject-2.0.so.0
#6  0x000000337ea47372 in rb_shell_new (no_registration=0, no_update=0, dry_run=0, b=0x0, playlists=0x0) at rb-shell.c:1115
#7  0x0000000000403972 in main (argc=1, argv=0x7fffffffe568) at main.c:272

With python 2.6.5, the init code gets called at line 175 of /usr/src/debug/rhythmbox-0.12.8/shell/rb-python-module.c within rb_python_module_init_python:

   148          /* Python initialization */
   149          Py_Initialize ();
   150  
   151          /* Restore old handler */
   152          res = sigaction (SIGINT, &old_sigint, NULL);
   153          if (res != 0) {
   154                  g_warning ("Error initializing Python interpreter: cannot restore "
   155                             "handler to SIGINT signal (%s)",
   156                             strerror (errno));
   157                  return;
   158          }
   159  
   160          PySys_SetArgv (1, argv);
   161  
   162          /* pygtk.require("2.8") */
   163          pygtk = PyImport_ImportModule ("pygtk");
   164          if (pygtk == NULL) {
   165                  g_warning ("Could not import pygtk");
   166                  PyErr_Print();
   167                  return;
   168          }
   169  
   170          mdict = PyModule_GetDict (pygtk);
   171          require = PyDict_GetItemString (mdict, "require");
   172          PyObject_CallObject (require, Py_BuildValue ("(S)", PyString_FromString ("2.8")));
   173  
   174          /* import gobject */
***175          init_pygobject ();
   176          if (PyErr_Occurred ()) {
   177                  g_warning ("Could not initialize pygobject");
   178                  PyErr_Print();
   179                  return;
   180          }
   181  

With python 2.6.6, 
the init_pygobject fails:

  exceptions.AssertionError("required version '2.8' not found on system",)

PyObject_Repr (v=exceptions.AssertionError("required version '2.8' not found on system",)) at Objects/object.c:360

It appears to be raising ImportError("could not import gobject (error was: 'AssertionError("required version \'2.8\' not found on system",))")
from inside pygobject_init () at /usr/include/pygtk-2.0/pygobject.h:322

which leaves  _PyGObject_API uninitialized as NULL, leaving to later crashes when code read through it to get jump tables.

rb_plugins_engine_init calls 
#ifdef ENABLE_PYTHON
	rb_python_module_init_python ();
#endif
but there doesn't seem to be any handling for the possibility that rb_python_module_init_python could fail.

I'm investigating why the import is failing.

The problem is at line 172 of rb-python-module.c (see comment #9 above): it calls
  pygtk.require("2.8")

This fails with both Python 2.6.5 and 2.6.6: in both cases, it raises:
  AssertionError: required version '2.8' not found on system

Indeed, it fails when run directly from /usr/bin/python:
>>> import pygtk
>>> pygtk.require('2.8')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python2.6/site-packages/pygtk.py", line 89, in require
    "required version '%s' not found on system" % version
AssertionError: required version '2.8' not found on system

Note that line 172 does not check to see if the call succeeded, silently throwing away the error; the exception is preserved within the per-thread state.

Line 175: init_pygobject leads to:
  - an import of gobject, which leads to:
    - an import of gobject/__init__.py
      "import sys" from gobject/__init__.py  (A)
      import of _gobject.so (B)

It looks like for some reason 2.6.5 survives this bug, but with 2.6.6 the exception is eventually detected, leading to it being raised by the "import sys" (A) from within gobject/__init__.py, leading to that import failing before the import of _gobject.so (B) (which is what would have initialized the jump tables)

Investigating the assertion failure within pygtk.require:
  versions = _get_available_versions()
is returning (for both Python 2.6.5 and 2.6.6):
  versions: {'2.0': '/usr/lib64/python2.6/site-packages/gtk-2.0'}
which means that this assertion fails:
   assert versions.has_key(version), \
           "required version '%s' not found on system" % version
since version is 2.8

In both cases, sys.path on this machine is:
['', '/usr/lib64/python26.zip', '/usr/lib64/python2.6', '/usr/lib64/python2.6/plat-linux2', '/usr/lib64/python2.6/lib-tk', '/usr/lib64/python2.6/lib-old', '/usr/lib64/python2.6/lib-dynload', '/home/david/.local/lib/python2.6/site-packages', '/usr/lib64/python2.6/site-packages', '/usr/lib64/python2.6/site-packages/PIL', '/usr/lib64/python2.6/site-packages/gst-0.10', '/usr/lib64/python2.6/site-packages/gtk-2.0', '/usr/lib64/python2.6/site-packages/webkit-1.0', '/usr/lib/python2.6/site-packages', '/usr/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg-info']

Question for Rhythmbox/PyGTK experts: see the code in comment #9 marked:
    /* pygtk.require("2.8") */

What is that fragment of pygtk code meant to do, and did it ever work?

It appears that it's always been picking up "2.0" as the version, and always failing with an assertion failure (as per comment #10).  Am I reading this right?

It may be possible to patch around this within the Python runtime, to special-case this exact failure (which would be ugly, but effective).

(In reply to comment #12)
> Question for Rhythmbox/PyGTK experts: see the code in comment #9 marked:
>     /* pygtk.require("2.8") */
> 
> What is that fragment of pygtk code meant to do, and did it ever work?

(/usr/lib64/python2.6/site-packages/pygtk.py is actually part of pygobject2-2.20.0-5.el6.x86_64; it looks like it's meant to ensure that a "gtk-2.8" directory would become part of the sys.path; however, that package also contains a /usr/lib64/python2.6/site-packages/pygtk.pth containing the line "gtk-2.0\n", ensuring that /usr/lib64/python2.6/site-packages/gtk-2.0 is part of sys.path).

Was Rhythmbox perhaps turning off Python-level assertions before?

Seems to have been changed upstream as:
http://git.gnome.org/browse/rhythmbox/commit/?id=4394826f36fad0ad36ea773b6d4525dfcfcd389b

which has:
>  pygtk.require("2.8") doesn't work - it's only after a major version,
>  so we should pass in "2.0" instead
and a reference to:
  https://bugzilla.gnome.org/show_bug.cgi?id=617587
which has:
> *Thanks to* python-2.6-r79204.patch, the import machinery of Python doesn't
> swallow errors anymore and so init_pygobject() fail because of the previous
> error and raise a new error.

That bug suggests that totem may also have the same issue.  However, I was able to successfully run totem-2.28.6-2.el6.x86_64

(In reply to comment #14)
> > *Thanks to* python-2.6-r79204.patch, the import machinery of Python doesn't
> > swallow errors anymore and so init_pygobject() fail because of the previous
> > error and raise a new error.
This is: http://svn.python.org/view?view=revision&revision=79204
where the significant change seems to be:
http://svn.python.org/view/python/branches/release26-maint/Python/import.c?r1=79204&r2=79203&pathrev=79204
apparently motivated by http://bugs.python.org/issue3137

*** Bug 690339 has been marked as a duplicate of this bug. ***

*** Bug 691909 has been marked as a duplicate of this bug. ***

*** Bug 692208 has been marked as a duplicate of this bug. ***

    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
Upgrading Python removed a call to the PyErr_Clear() method, which exposed an assertion failure in RhythmBox that resulted in RhythmBox crashing. Python now compensates for the RhythmBox assertion failure.

An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2011-0554.html

An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2011-0554.html