Description of problem: According to http://www.linuxpowertop.org/, sealert is one of the top offenders on my system. Actually considering it's doing this on a box when SELinux isn't enabled, it's even worse. Version-Release number of selected component (if applicable): setroubleshoot-server-1.9.4-1.fc7
By design setroubleshoot will wake up often, it is an "audit daemon watcher", reading every audit message emitted from the kernel. We probaby can't fix the fact it wakes up to receive an audit message. Sealert is also a DBus client, it's going to listen for DBus messages as well. We could, and perhaps should, add a check to setroubleshoot to prevent it from starting up if selinux is disabled or in permissive mode (not sure about the latter since it can still be useful in permissive mode). For the time being you can always disable the service with chkconfig and the desktop component with the desktop 'services' applet. On another note, as a RH developer SELinux should not be disabled and the troubleshooter is there to help make running with SELinux enabled as painless as possible to encourage the use of SELinux. Exactly how and when the troublehooter should be installed/enabled is an open question which has yet to be addressed and I suspect is as much a topic of this bug report as is the resources it's consuming, it's a fair question.
Is it directly attaching to the kernel audit layer, or is it watching the logs?
Log files are not watched. The troubleshooter is composed of two parts, a daemon, setroubleshootd, and a desktop client, sealert. The daemon directly attaches via a unix socket to the audit layer and receives every audit message. When the daemon sees an audit message which is an AVC it runs analysis on the AVC and sends the result to the sealert desktop process. Thus only the daemon will wake up for every audit message. The sealert desktop client will wake up for every AVC. The original bz said sealert was waking up too often, This suggests one of two things (which I have not investiaged yet). 1) Your system is getting a lot of AVC's (selinux is not disabled, but is permissive, BTW did you truly mean SELinux was disabled, or is it just permissive?). One way to see if you're getting a lot of AVC is to open up sealert by clicking on the troubleshooter icon in the notification area, or via System-->Administration-->SELinux Troubleshooter. Do you see an AVC's whose counts are really high? 2) Sealert is not receiving any AVC alerts from setroubleshootd, but because it also functions as a DBus server the DBus protocol is causing it to wake up on broadcast DBux messages which it then ignores. This is only a theory.
Wasn't #1, SELinux was disabled, not permissive.
O.K. then it shouldn't be due to any audit activity. I'll investigate what else might make it wake up and run.
sealert is doing a nice "10 times per second" wakeup... could this be the same missetting of python that yum-updatesd used to suffer from?
If selinux is disabled, it should probably exit since it will require a reboot to switch back to enforcing or permissive.
The bug mentioned in comment 7 is Bug 204946. It was closed because gobject was rebuilt with a promise that the timeout is now 1000. Well, its not: poll([{fd=11, events=POLLIN}, {fd=10, events=POLLIN}, {fd=18, events=POLLIN|POLLPRI}, {fd=14, events=POLLIN|POLLPRI}, {fd=15, events=POLLIN|POLLPRI}, {fd=20, events=POLLIN|POLLPRI}, {fd=3, events=POLLIN}, {fd=13, events=POLLIN}, {fd=9, events=POLLIN}], 9, 100) = 0 The timeout is 100. The return value shows that its timing out with nothing to do. My testing shows its running 10 times a second. So, I wonder if gobject reverted its timeout back to 100?
This appears to be exactly the problem cited in bug 204946. The wake up is going to happen to any python program utilizing gobject and threads, of which sealert is just a single example. Bug 204946 was closed after bumping the timeout from 100ms to 1000ms, hardly a fix IMHO. We're never going to achieve power saving when system libraries constantly timeout to ask the question "did a signal arrive?". I see two possible solutions: 1) re-code sealert to not use threads (it's threading which toggles signal polling on) 2) fix pygobject to not poll for signals. Option 2 seems preferable because it's the more general case fix which could apply to many programs. I'll dig a bit deeper on why signal polling is being done, it fundamentally feels wrong to me. Perhaps at one time it was a quick fix which was not thought through.
First of all thanks to Ray Strode and Matthew Barnes who helped research this problem. The problem is exactly the one cited in bug 204946. Here are some more details and background information. Python only receives signals in its main thread. Upon receipt of the signal it records the signal in an internal data structure and queues it's processing for later when the Python interpreter resumes execution. Python extensions may block indefinitely if they use clib calls such as select() or poll(). GObject is a good example of such an extension. GObject's have "sources" which are watched using the poll() function. If a GObject mainloop does not have any other work to perform it blocks in the system call. This means although a signal might have been received and queued by Python's main thread there is nothing to cause it to wake up and check if it needs to execute any of the queued signal handlers. Pygtk solves the problem by timing out on a regular interval and manually calling the function to check and run the signal queue (PyErr_CheckSignals()). It is this timeout which is causing the wake ups. Some of this is documented here: http://mail.python.org/pipermail/python-dev/2006-September/068569.html The behavior is only turned on if threads are enabled in the pygtk application. sealert is threaded and does call the pygtk thread init function which toggles this behavior on. Is there a generic solution for the system libraries (e.g. pygtk, gobject, python)? Yes, there has been a proposed fix for about a year now which is described here along with a patch. http://sourceforge.net/tracker/index.php?func=detail&aid=1564547&group_id=5470&atid=305470 Please note that both Python and PyGTK have to be patched. First Python has to be patched to modify how it handles signals and to introduce a watchable descriptior (py_signal_pipe) which become read ready upon signal delivery. Then PyGTK has to be patched to watch this file descriptor in poll() and relinguish control when it becomes read ready. There is some minor controversy over the patch, however it seems sane to me. Unless we patch Python and PyGTK using the above patch all threaded python apps using GObject and/or PyGTK are going to run afoul of this problem. The patch is tentatively scheduled for Python 2.6 if it gets approved. Should we patch our version now? Hmm... tough call. The patch has the potential to introduce regressions as it modifies Python's core signal handling implementation across the board. Then again on the other hand the Python folks would probably be thrilled to have us field test the patch. This bug was originally assigned to setroubleshoot, can it be fixed there instead? Yes and No. We could re-code sealert to not use threads, probably in a manner described here: http://faq.pygtk.org/index.py?req=show&file=faq23.020.htp This describes a technique that could be used to replace threads with what could be described as co-routines. However, at the moment there are higher priority tasks for setroubleshoot than re-writing how it handles background tasks and solving the problem at the core makes more sense because all threaded PyGTK applications benefit. I'm going to reassign this to Python with the hope they try applying the patch. If that patch is applied then PyGTK will need to be patched to use it. Independently as time permits I'll try to re-code setroubleshoot to not use threads.
In the above comment I referred to PyGTK, the problem is manifest in gobject.thread_init() as well. This is the actual call sealert is making which triggers the behavior. FWIW, PyGTK and gobject are closely related extension libraries.
sealert has been rewritten so as not to use threads which should address this issue. However, this does not mean python is off the hook :-) Signal handling in threads remains seriously broken and need to be fixed.
Updated bug link: http://bugs.python.org/issue1564547
The mentioned patch has been superseded by a simplified variation: http://bugs.python.org/issue1583 This should end up getting into python 2.6. That just leaves patching pygtk to use it.
Re-assigning to pygtk then, the patch looks like it might be able to go into 2.5.x too ... although I'm not promising anything atm.
Python has a policy of not adding new features (such as new external APIs) in maintenance releases, meaning 2.5.x is unlikely.
PyGTK 2.12.1 now uses the new PySignal_SetWakeupFd(). Closing as RAWHIDE. http://svn.gnome.org/viewvc/pygtk/trunk/gtk/gtk.override?r1=2926&r2=2947