Bug 842403 - modules command in python help is hanging
modules command in python help is hanging
Status: CLOSED WONTFIX
Product: Fedora
Classification: Fedora
Component: python (Show other bugs)
17
x86_64 Linux
unspecified Severity medium
: ---
: ---
Assigned To: Bohuslav "Slavek" Kabrda
Fedora Extras Quality Assurance
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-07-23 14:54 EDT by kushaldas@gmail.com
Modified: 2013-07-31 15:33 EDT (History)
8 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-07-31 15:33:06 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
bt output (38.58 KB, application/octet-stream)
2012-07-23 14:54 EDT, kushaldas@gmail.com
no flags Details
IRC log of the discussion (7.90 KB, text/plain)
2012-07-23 15:07 EDT, kushaldas@gmail.com
no flags Details
LD_DEBUG=all python (24.76 MB, text/plain)
2012-07-23 15:24 EDT, kushaldas@gmail.com
no flags Details
Log from a fresh Fedora installation (9.96 KB, text/plain)
2012-07-23 16:12 EDT, kushaldas@gmail.com
no flags Details
patch for pkgutil (512 bytes, patch)
2012-07-24 11:56 EDT, kushaldas@gmail.com
no flags Details | Diff

  None (edit)
Description kushaldas@gmail.com 2012-07-23 14:54:08 EDT
Created attachment 599840 [details]
bt output

Description of problem: 
If I try to see all installed modules in python help, the interpreter hangs.


Version-Release number of selected component (if applicable): pygtk2-2.24.0-4.fc17.x86_64


How reproducible:


Steps to Reproduce:
1. open python
2. type help()
3. type modules
  
Actual results:

python hangs.

Expected results:

Supposed to show us all installed modules.

Additional info:
While talking about the issue in #fedora-desktop, halfline suggested to try 
$  LD_PRELOAD=/usr/lib64/libgtk-x11-2.0.so python

and after that if I try to view the installed modules I get teh following

$ LD_PRELOAD=/usr/lib64/libgtk-x11-2.0.so python
Python 2.7.3 (default, Apr 30 2012, 21:18:11)
[GCC 4.7.0 20120416 (Red Hat 4.7.0-2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> help()
 
Welcome to Python 2.7! This is the online help utility.
 
If this is your first time using Python, you should definitely check out
the tutorial on the Internet at http://docs.python.org/tutorial/.
 
Enter the name of any module, keyword, or topic to get help on writing
Python programs and using Python modules. To quit this help utility and
return to the interpreter, just type "quit".
 
To get a list of available modules, keywords, or topics, type "modules",
"keywords", or "topics". Each module also comes with a one-line summary
of what it does; to list the modules whose summaries contain a given word
such as "spam", type "modules spam".
 
help> modules
 
Please wait a moment while I gather a list of all available modules...
 
dm.c: 1693: not running as root returning empty list
 
** (process:10626): WARNING **: Trying to register gtype 'GMountMountFlags' as enum when in fact it is of type 'GFlags'
 
** (process:10626): WARNING **: Trying to register gtype 'GDriveStartFlags' as enum when in fact it is of type 'GFlags'
 
** (process:10626): WARNING **: Trying to register gtype 'GSocketMsgFlags' as enum when in fact it is of type 'GFlags'
Gtk-Message: Failed to load module "pk-gtk-module"
/usr/lib64/python2.7/pkgutil.py:110: UserWarning: libevent version mismatch: system version is '2.0.18-stable' but this gevent is compiled against '2.0.14-stable'
__import__(name)
** Message: pygobject_register_sinkfunc is deprecated (GstObject)
Use bootstrap.py and buildout to create executable


I am also attaching bt output from the first try (got it from gdb)
Comment 1 kushaldas@gmail.com 2012-07-23 15:07:30 EDT
Created attachment 599842 [details]
IRC log of the discussion

Attaching the IRC log discussion on the topic.
Comment 2 kushaldas@gmail.com 2012-07-23 15:24:56 EDT
Created attachment 599847 [details]
LD_DEBUG=all python

Attaching LD_DEBUG=all python output
Comment 3 kushaldas@gmail.com 2012-07-23 16:12:22 EDT
Created attachment 599852 [details]
Log from a fresh Fedora installation

This is output from a fresh F17 installation
Comment 4 kushaldas@gmail.com 2012-07-24 11:30:19 EDT
Changing the component to Python to track the actual development in python. Will file indivitual bugs for each packages which are causing the issue.
Comment 5 Dave Malcolm 2012-07-24 11:34:21 EDT
To summarize:  "help> modules" tries to import every Python module on the system, so that it can print metadata about them.  These imports leads to a chain of dynamic loading in which the process tries to import parts of both gtk2 and gtk3 stacks, and it hangs initializing one of gtk stacks (presumably the second stack).

Doing so isn't supported, but clearly Python shouldn't hang like this.

Suggested improvements:
  * somehow detect when both gtk2 and gtk3 components are attempted to be loaded, and trigger a runtime error that can propagate back up as an exception (since there may be other Python code out there that tries to import things that would cause both stacks to be loaded).  Is there a way of doing this within the gtk stack?  (or maybe easier to do in pygobject/pygtk?  though that wouldn't cover the case where a 3rd-party module has indirectly caused a gtk stack to be dynamically linked into the process).

  * reimplement pydoc so that each module is loaded in a subprocess, sandboxing the python process against issues like this.  We've had no end of issues over the years with buggy modules breaking the help functionality.  (IIRC the keyword search works in a similar way, so this would help with that).  Would be a non-trivial path to Python's Lib/pydoc.py
Comment 6 kushaldas@gmail.com 2012-07-24 11:42:13 EDT
With more debugging I gathered the following information :

import clutter --  hangs, package -- pyclutter-1.3.2-10.fc17.x86_64
import gnomedvb -- hangs, package -- gnome-dvb-daemon-0.2.8-1.fc17.x86_64

import rst2pdf.tests -- python interpreter exists, package -- rst2pdf-0.92-1.fc17.noarch.rpm
It also prints "Use bootstrap.py and buildout to create executable" before exiting.


import liveusb -- python interpreter exists, package --
liveusb-creator-3.11.7-1.fc17.noarch
Line number 52 in liveusb/__init__.py , doing a sys.exit(1) if it is non-root user


I also had to patch pkgutil.py after removing all the above mention packages to make modules command work.
http://kushal.fedorapeople.org/pkgutil.patch

One can use the following code to find out which module is causing trouble:

import pkgutil
for x,y,z in pkgutil.walk_packages():
  print x,y,z
Comment 7 kushaldas@gmail.com 2012-07-24 11:56:12 EDT
Created attachment 600062 [details]
patch for pkgutil
Comment 8 Pierre-YvesChibon 2012-07-26 05:36:07 EDT
https://bugzilla.redhat.com/show_bug.cgi?id=843416 might give some more info, I could generate some backtrace with abrt
Comment 9 Luke Macken 2013-01-04 13:59:05 EST
(In reply to comment #6)
> import liveusb -- python interpreter exists, package --
> liveusb-creator-3.11.7-1.fc17.noarch
> Line number 52 in liveusb/__init__.py , doing a sys.exit(1) if it is
> non-root user

I just fixed this issue upstream (Bug #842878)
Comment 10 Fedora Admin XMLRPC Client 2013-05-10 01:03:05 EDT
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.
Comment 11 Fedora Admin XMLRPC Client 2013-05-10 01:03:35 EDT
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.
Comment 12 Fedora Admin XMLRPC Client 2013-05-10 01:07:03 EDT
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.
Comment 13 Fedora Admin XMLRPC Client 2013-05-10 01:14:45 EDT
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.
Comment 14 Fedora Admin XMLRPC Client 2013-05-10 01:15:03 EDT
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.
Comment 15 Fedora End Of Life 2013-07-03 16:08:15 EDT
This message is a reminder that Fedora 17 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 17. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '17'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 17's end of life.

Bug Reporter:  Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 17 is end of life. If you 
would still like  to see this bug fixed and are able to reproduce it 
against a later version  of Fedora, you are encouraged  change the 
'version' to a later Fedora version prior to Fedora 17's end of life.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.
Comment 16 Fedora End Of Life 2013-07-31 15:33:12 EDT
Fedora 17 changed to end-of-life (EOL) status on 2013-07-30. Fedora 17 is 
no longer maintained, which means that it will not receive any further 
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of 
Fedora please feel free to reopen this bug against that version.

Thank you for reporting this bug and we are sorry it could not be fixed.

Note You need to log in before you can comment on or make changes to this bug.