Bug 470782 - sealert memory leak
Summary: sealert memory leak
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: setroubleshoot
Version: 9
Hardware: All
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Daniel Walsh
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2008-11-10 08:50 UTC by Need Real Name
Modified: 2009-06-10 11:08 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2009-06-10 11:08:17 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
dmesg (54.98 KB, text/plain)
2008-11-10 08:50 UTC, Need Real Name
no flags Details
OOM kill (58.97 KB, text/plain)
2008-11-10 16:12 UTC, Need Real Name
no flags Details
sealert file (835.20 KB, application/x-gzip)
2008-11-10 16:24 UTC, Need Real Name
no flags Details

Description Need Real Name 2008-11-10 08:50:15 UTC
Created attachment 323047 [details]
dmesg

Looks like sealert leaks memory and 8Gb machine gets OOM kill.
Originaly I thought this is ipsec problem
see meminfo & ps output at
https://bugzilla.redhat.com/show_bug.cgi?id=470738
but looks this is sealert

Comment 1 Need Real Name 2008-11-10 08:57:03 UTC
from ps axuww output 
mal       5151 80.2 84.8 6958336 6521896 ?     R    16:14  11:32
/usr/bin/python -E /usr/bin/sealert -s


sealert uses 80.2% CPU & 84.8% memory
and this is on 8Gb 3Ghz machine.
See https://bugzilla.redhat.com/show_bug.cgi?id=470738
for complete ps output

Comment 2 Need Real Name 2008-11-10 08:59:33 UTC
I am not sure this is related. From 
cat /var/log/audit/audit.log |audit2allow 

#============= setroubleshootd_t ==============
allow setroubleshootd_t self:capability { sys_rawio sys_admin sys_resource };

Comment 3 Daniel Walsh 2008-11-10 14:09:30 UTC
How large is your /var/lib/setroubleshoot/audit_listener_database.xml

file?

If you execute
# sevice setroubleshoot stop
# mv /var/lib/setroubleshoot/audit_listener_database.xml
/var/lib/setroubleshoot/audit_listener_database.xml.old

# touch /var/lib/setroubleshoot/audit_listener_database.xml
# sevice setroubleshoot start

Does the problem go away?

Comment 4 Need Real Name 2008-11-10 15:15:24 UTC
ls -l /var/lib/setroubleshoot/audit_listener_database.xml
-rw------- 1 root root 37591942 2008-11-06 03:30 /var/lib/setroubleshoot audit_listener_database.xml

37Mb is nothing compared to 8Gb computer RAM

Comment 5 Need Real Name 2008-11-10 16:12:29 UTC
Created attachment 323089 [details]
OOM kill

Sealert is such a pain, I disabled setroubleshoot, otherwise I get OOM kill every 10-15 minutes

Comment 6 Need Real Name 2008-11-10 16:24:15 UTC
Created attachment 323091 [details]
sealert file

I am attaching audit_listener_database.xml.

From my opinion current sealert should be removed (or at least disabled by default) from current fedora altogether. The bug in question of a memory leak can be fixed, but currently it just trashes the computer. sealert makes the computer plain unsable. I create more problem, than its marginal value.

In addition to this (probably fixable) OOM leak it sealert has, from my point of view, brain dead .xml format to store the data. Don't subscribe to this stupid xml hype.

XML is just plain wrong to store automatically generated data. The problem is that to read the data from xml you need to read the entire file. So that on low-memory computers the 30-40Mb xml files would just eat substantial memory.
If sealert were using say tab-separated format (or minisql) then you can read record one-by one and process them on a fly. No need to read huge files to memory.

Currently, especially for fedora 10, 
I STRONGLY RECOMMEND TO DISABLE THIS BRAIN-DEAD program.

Comment 7 Daniel Walsh 2008-11-10 19:55:22 UTC
The audit_listener_database.xml that you attached has brought forth two bugs.

One you seem to have thousands of block devices in your homedir, which selinux policy is not allowing setfiles/restorecon to relabel to user_home_t.  I will update policy to allow this, since disallowing this is crazy.  This has caused you audit_listener_database.xml to grow huge.  setroubleshoot is supposed to prevent audit_listener_database.xml from growing larger then 50 records, which we are working on a fix for.

Comment 8 Need Real Name 2008-11-10 20:16:38 UTC
The block devices in homedir are the remains of old Linux installation,
which was just copied there "as is" for a backup before Fedora 9 install.

But the size of the original audit_listener_database.xml is just 37Mb
When it is read to memory - this is nothing compared to 8Gb RAM available.
What causes the OOM kill then? I do not believe these 37Mb is the cause of the problem.

from ps axuww output 
mal       5151 80.2 84.8 6958336 6521896 ?     R    16:14  11:32
/usr/bin/python -E /usr/bin/sealert -s


where the 6.5Gb memory went to?

Comment 9 Daniel Walsh 2008-11-10 21:42:04 UTC
I have no idea this is a python application using libxml and python bindings.  I attempted to use you xml file and my machine went nuts also.

Comment 10 Need Real Name 2008-11-11 07:21:39 UTC
And this is why I do not like xml.
I converted two my applications from xml to this format:
One record occupies one line.
One line represent CGI-style escaped data

name=Name1&id=12345&ptr=8239&Desctiption=This%20is%20Name1%20Descr
name=BBBB&id=12346&ptr=8239&Desctiption=This%20is%20BBBB%20Descr
name=CCCC&id=12347&ptr=8239&Desctiption=This%20is%20CCCC%20Descr

As a result I can read records one-by one and process them while reading
(xml reads the entire file and then parses)
This format is extremely simple to parse (all you need is CGI-parameters URL
escape/unescape functions).

Speed increased & memory footprint reduced drastically.

Tab-separated format is even more efficient than mine,
but if saved records differ from one another, this may create difficulity in parsing.
The format above allows easy read/write of records of arbitrary fromats.

Comment 11 Daniel Walsh 2008-11-11 14:21:52 UTC
Sounds fine, but this is not my primary job,  :^)   I primarily work on adding features to SELinux and we are concentrating on Virtualization and GRID computing.  

My goals with enhancing setroubleshoot and sealert is to stop running them all the time.  

Phase 1:
  Add seapplet, C Program, that listens for setroubleshoot messages and puts up the "Star" in the task bar,  When user clicks on "Star" launch sealert browser.

Phase 2:
  Change setroubleshoot from a system service (initrc) to a program launched when audit detects and AVC.  It also needs to launch when sealert browser is running to handle communications.

Phase 3:
  Separate the handling of the AVC/writing of the database from the service that 

I would gladly accept patches on any of these enhancements/fixes, including moving away from XML to another format.

Comment 12 Bug Zapper 2009-06-10 03:13:53 UTC
This message is a reminder that Fedora 9 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 9.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '9'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 9's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 9 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Comment 13 Daniel Walsh 2009-06-10 11:08:17 UTC
sealert has been split into two apps seapplet which is a C program that listens for events and the browser.  The browser now closes on exit so the memory will not continue to grow,

Browser is also beening totally rewritten so closing in current release of F11.


Note You need to log in before you can comment on or make changes to this bug.