Bug 903280

Summary: libselinux multiple mmaps /etc/selinux/targeted/contexts/files/file_contexts*.bin causing big memory leak in libvirtd
Product: [Fedora] Fedora Reporter: Richard W.M. Jones <rjones>
Component: libselinuxAssignee: Daniel Walsh <dwalsh>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: unspecified    
Version: 18CC: ajia, dallan, dwalsh, dyasny, dyuan, eparis, gren, mgrepl, mzhan, rjones, rwu, whuang, ydu
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 891324 Environment:
Last Closed: 2013-03-02 20:10:15 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 890039, 891324, 903203    

Description Richard W.M. Jones 2013-01-23 16:04:04 UTC
+++ This bug was initially created as a clone of Bug #891324 +++

+++ This bug was initially created as a clone of Bug #890039 +++

Description of problem:

libvirtd grows unbounded if you continually create and
destroy transient domains:

  PID USER      PR  NI  VIRT  RES  SHR S  %CPU %MEM    TIME+  COMMAND           
25390 rjones    20   0 10.9g  10g 8.4g S   0.0 66.8  16:34.08 libvirtd          

This seems to happen because libselinux mmap's the
following files hundreds of times:

/etc/selinux/targeted/contexts/files/file_contexts.local.bin
/etc/selinux/targeted/contexts/files/file_contexts.homedirs.bin
/etc/selinux/targeted/contexts/files/file_contexts.bin

(About one mapping per second in my tests).

Methodology:

I'm testing libvirt.git (at time of writing,
commit bf62e9953c3dde35551a0c2a91d30a294516609a).

I have applied my patch to libselinux to fix bug 903203.

I use the following command (from the libvirt.git directory)
to run libvirtd from git:

killall lt-libvirtd libvirtd
./run ./daemon/libvirtd --timeout 30

while at the same time running the following command from
the libguestfs.git directory:

while true; do echo -n .; ../libvirt/run ./run ./fish/guestfish -N fs exit; done

This creates lots of transient domains, serially.

Observations:

Valgrind shows some reachable blocks, but no significant
unreachable blocks, indicating there is no memory leak.

Examining /proc/$pid/maps shows that the number of memory mappings
is growing like crazy:

$ wc -l /proc/589/maps 
822 /proc/589/maps
$ wc -l /proc/589/maps 
837 /proc/589/maps
$ wc -l /proc/589/maps 
852 /proc/589/maps
$ wc -l /proc/589/maps 
867 /proc/589/maps
$ wc -l /proc/589/maps 
942 /proc/589/maps
$ wc -l /proc/589/maps 
1032 /proc/589/maps

Examination of /proc/$pid/maps appears to point to libselinux again:

$ awk '{print $6}' /proc/589/maps | sort | uniq -c | sort -nr | head
    340 /etc/selinux/targeted/contexts/files/file_contexts.local.bin
    340 /etc/selinux/targeted/contexts/files/file_contexts.homedirs.bin
    340 /etc/selinux/targeted/contexts/files/file_contexts.bin
     67 
      4 /usr/lib64/sasl2/libsasldb.so.2.0.25
      4 /usr/lib64/sasl2/libplain.so.2.0.25
      4 /usr/lib64/sasl2/liblogin.so.2.0.25
      4 /usr/lib64/sasl2/libgssapiv2.so.2.0.25
      4 /usr/lib64/sasl2/libdigestmd5.so.2.0.25
      4 /usr/lib64/sasl2/libcrammd5.so.2.0.25

Comment 1 Daniel Walsh 2013-01-28 16:39:29 UTC
Is this the same problem as 903203?

Comment 2 Eric Paris 2013-01-28 17:03:17 UTC
No.  Different, but both binary file context memory leaks.  This should be:

http://git.infradead.org/users/eparis/selinux-userspace.git/commitdiff/f2081d3d53707389d68a9d6fdba468dd80b2c316

Comment 3 Daniel Walsh 2013-01-28 19:13:06 UTC
Ok I grabbed that also.

Fixed in 
libselinux-2.1.12-7.1.fc18

Comment 4 Fedora Update System 2013-01-28 19:14:38 UTC
libselinux-2.1.12-7.1.fc18 has been submitted as an update for Fedora 18.
https://admin.fedoraproject.org/updates/libselinux-2.1.12-7.1.fc18

Comment 5 Fedora Update System 2013-01-30 00:55:49 UTC
Package libselinux-2.1.12-7.1.fc18:
* should fix your issue,
* was pushed to the Fedora 18 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing libselinux-2.1.12-7.1.fc18'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2013-1636/libselinux-2.1.12-7.1.fc18
then log in and leave karma (feedback).

Comment 6 Fedora Update System 2013-03-02 20:10:23 UTC
libselinux-2.1.12-7.1.fc18 has been pushed to the Fedora 18 stable repository.  If problems still persist, please make note of it in this bug report.