Bug 239893 - Signal handling and poll() in threaded python don't work well
Signal handling and poll() in threaded python don't work well
Product: Fedora
Classification: Fedora
Component: pygtk2 (Show other bugs)
All Linux
medium Severity medium
: ---
: ---
Assigned To: Matthew Barnes
Fedora Extras Quality Assurance
Depends On:
Blocks: wakeup
  Show dependency treegraph
Reported: 2007-05-11 19:33 EDT by Bill Nottingham
Modified: 2014-03-16 23:06 EDT (History)
7 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2008-01-29 09:28:14 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description Bill Nottingham 2007-05-11 19:33:39 EDT
Description of problem:

According to http://www.linuxpowertop.org/, sealert is one of
the top offenders on my system. Actually considering it's doing
this on a box when SELinux isn't enabled, it's even worse.

Version-Release number of selected component (if applicable):

Comment 1 John Dennis 2007-05-14 13:11:14 EDT
By design setroubleshoot will wake up often, it is an "audit daemon watcher",
reading every audit message emitted from the kernel. We probaby can't fix the
fact  it wakes up to receive an audit message. Sealert is also a DBus client,
it's going to listen for DBus messages as well.

We could, and perhaps should, add a check to setroubleshoot to prevent it from
starting up if selinux is disabled or in permissive mode (not sure about the
latter since it can still be useful in permissive mode). For the time being you
can always disable the service with chkconfig and the desktop component with the
desktop 'services' applet.

On another note, as a RH developer SELinux should not be disabled and the
troubleshooter is there to help make running with SELinux enabled as painless as
possible to encourage the use of SELinux.

Exactly how and when the troublehooter should be installed/enabled is an open
question which has yet to be addressed and I suspect is as much a topic of this
bug report as is the resources it's consuming, it's a fair question.
Comment 2 Bill Nottingham 2007-05-14 14:03:17 EDT
Is it directly attaching to the kernel audit layer, or is it watching the logs?
Comment 3 John Dennis 2007-05-14 14:50:36 EDT
Log files are not watched. The troubleshooter is composed of two parts, a
daemon, setroubleshootd, and a desktop client, sealert. The daemon directly
attaches via a unix socket to the audit layer and receives every audit message.
When the daemon sees an audit message which is an AVC it runs analysis on the
AVC and sends the result to the sealert desktop process. Thus only the daemon
will wake up for every audit message. The sealert desktop client will wake up
for every AVC. The original bz said sealert was waking up too often, This
suggests one of two things (which I have not investiaged yet).

1) Your system is getting a lot of AVC's (selinux is not disabled, but is
permissive, BTW did you truly mean SELinux was disabled, or is it just
permissive?). One way to see if you're getting a lot of AVC is to open up
sealert by clicking on the troubleshooter icon in the notification area, or via
System-->Administration-->SELinux Troubleshooter. Do you see an AVC's whose
counts are really high?

2) Sealert is not receiving any AVC alerts from setroubleshootd, but because it
also functions as a DBus server the DBus protocol is causing it to wake up on
broadcast DBux messages which it then ignores. This is only a theory.

Comment 4 Bill Nottingham 2007-05-14 15:01:16 EDT
Wasn't #1, SELinux was disabled, not permissive.
Comment 5 John Dennis 2007-05-14 15:20:17 EDT
O.K. then it shouldn't be due to any audit activity. I'll investigate what else
might make it wake up and run.
Comment 6 Arjan van de Ven 2007-06-03 16:21:09 EDT
sealert is doing a nice "10 times per second" wakeup...

could this be the same missetting of python that yum-updatesd used to suffer from?
Comment 7 Steve Grubb 2007-07-26 17:41:43 EDT
If selinux is disabled, it should probably exit since it will require a reboot
to switch back to enforcing or permissive.
Comment 8 Steve Grubb 2007-07-26 20:15:40 EDT
The bug mentioned in comment 7 is Bug 204946. It was closed because gobject was
rebuilt with a promise that the timeout is now 1000. Well, its not:

poll([{fd=11, events=POLLIN}, {fd=10, events=POLLIN}, {fd=18,
events=POLLIN|POLLPRI}, {fd=14, events=POLLIN|POLLPRI}, {fd=15,
events=POLLIN|POLLPRI}, {fd=20, events=POLLIN|POLLPRI}, {fd=3, events=POLLIN},
{fd=13, events=POLLIN}, {fd=9, events=POLLIN}], 9, 100) = 0

The timeout is 100. The return value shows that its timing out with nothing to
do. My testing shows its running 10 times a second.

So, I wonder if gobject reverted its timeout back to 100?
Comment 9 John Dennis 2007-08-09 17:36:43 EDT
This appears to be exactly the problem cited in bug 204946. The wake up is going
to happen to any python program utilizing gobject and threads, of which sealert
is just a single example.

Bug 204946 was closed after bumping the timeout from 100ms to 1000ms, hardly a
fix IMHO. We're never going to achieve power saving when system libraries
constantly timeout to ask the question "did a signal arrive?".

I see two possible solutions:

1) re-code sealert to not use threads (it's threading which toggles signal
polling on)

2) fix pygobject to not poll for signals.

Option 2 seems preferable because it's the more general case fix which could
apply to many programs.

I'll dig a bit deeper on why signal polling is being done, it fundamentally
feels wrong to me. Perhaps at one time it was a quick fix which was not thought
Comment 10 John Dennis 2007-08-14 15:45:52 EDT
First of all thanks to Ray Strode and Matthew Barnes who helped research this

The problem is exactly the one cited in bug 204946. Here are some more details
and background information.

Python only receives signals in its main thread. Upon receipt of the signal it
records the signal in an internal data structure and queues it's processing for
later when the Python interpreter resumes execution. Python extensions may block
indefinitely if they use clib calls such as select() or poll(). GObject is a
good example of such an extension. GObject's have "sources" which are watched
using the poll() function. If a GObject mainloop does not have any other work to
perform it blocks in the system call. This means although a signal might have
been received and queued by Python's main thread there is nothing to cause it to
wake up and check if it needs to execute any of the queued signal handlers.
Pygtk solves the problem by timing out on a regular interval and manually
calling the function to check and run the signal queue (PyErr_CheckSignals()).
It is this timeout which is causing the wake ups.

Some of this is documented here:

The behavior is only turned on if threads are enabled in the pygtk application.
sealert is threaded and does call the pygtk thread init function which toggles
this behavior on.

Is there a generic solution for the system libraries (e.g. pygtk, gobject,
python)? Yes, there has been a proposed fix for about a year now which is
described here along with a patch.


Please note that both Python and PyGTK have to be patched. First Python has to
be patched to modify how it handles signals and to introduce a watchable
descriptior (py_signal_pipe) which become read ready upon signal delivery. Then
PyGTK has to be patched to watch this file descriptor in poll() and relinguish
control when it becomes read ready.

There is some minor controversy over the patch, however it seems sane to me.

Unless we patch Python and PyGTK using the above patch all threaded python apps
using GObject and/or PyGTK are going to run afoul of this problem.

The patch is tentatively scheduled for Python 2.6 if it gets approved. Should we
patch our version now? Hmm... tough call. The patch has the potential to
introduce regressions as it modifies Python's core signal handling
implementation across the board. Then again on the other hand the Python folks
would probably be thrilled to have us field test the patch.

This bug was originally assigned to setroubleshoot, can it be fixed there
instead? Yes and No. We could re-code sealert to not use threads, probably in a
manner described here:


This describes a technique that could be used to replace threads with what could
be described as co-routines. However, at the moment there are higher priority
tasks for setroubleshoot than re-writing how it handles background tasks and
solving the problem at the core makes more sense because all threaded PyGTK
applications benefit. 

I'm going to reassign this to Python with the hope they try applying the patch.
If that patch is applied then PyGTK will need to be patched to use it.

Independently as time permits I'll try to re-code setroubleshoot to not use threads.
Comment 11 John Dennis 2007-08-14 15:56:45 EDT
In the above comment I referred to PyGTK, the problem is manifest in
gobject.thread_init() as well. This is the actual call sealert is making which
triggers the behavior. FWIW, PyGTK and gobject are closely related extension
Comment 12 John Dennis 2007-09-20 18:21:52 EDT
sealert has been rewritten so as not to use threads which should address this
issue. However, this does not mean python is off the hook :-) Signal handling in
threads remains seriously broken and need to be fixed.
Comment 13 Need Real Name 2007-11-11 17:10:09 EST
Updated bug link:

Comment 14 Adam Olsen 2007-12-12 01:58:28 EST
The mentioned patch has been superseded by a simplified variation:


This should end up getting into python 2.6.  That just leaves patching pygtk to
use it.
Comment 15 James Antill 2007-12-12 09:09:26 EST
 Re-assigning to pygtk then, the patch looks like it might be able to go into
2.5.x too ... although I'm not promising anything atm.
Comment 16 Adam Olsen 2007-12-12 12:32:09 EST
Python has a policy of not adding new features (such as new external APIs) in
maintenance releases, meaning 2.5.x is unlikely.
Comment 17 Matthew Barnes 2008-01-29 09:28:14 EST
PyGTK 2.12.1 now uses the new PySignal_SetWakeupFd().  Closing as RAWHIDE.


Note You need to log in before you can comment on or make changes to this bug.