Bug 1077897

Summary: Memory leak with proxy auth control
Product: Red Hat Enterprise Linux 7 Reporter: Noriko Hosoi <nhosoi>
Component: 389-ds-baseAssignee: Noriko Hosoi <nhosoi>
Status: CLOSED ERRATA QA Contact: Viktor Ashirov <vashirov>
Severity: unspecified Docs Contact:
Priority: medium    
Version: 7.0CC: mreynolds, nkinder, rmeggins, vashirov
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Fixed In Version: 389-ds-base- Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-03-05 09:34:06 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Description Flags
valgrind output none

Description Noriko Hosoi 2014-03-18 19:31:12 UTC
This bug is created as a clone of upstream ticket:

I've been running some load tests against version 389-ds-base- on Centos 6.5. When looking at the vmsize of the ns-slapd process it appears to grow, rather than reaching a plateau. After some modifications to the load test it appears that the memory usage is flat if we remove the portion that performs the proxied authentication control.

I've created a simplified version of the data that exhibits the problem. This backup ldif is attached to the case. It consists of 3 users and 2 acis.

The load is the following pseudocode:
create a simple connection, binding as uid=AUser,o=Test.com
In 30 threads, repeat 1000 times each (i.e. 30000 total searches):
    using this single connection, proxy as uid=PUser,o=Test.com, and search for uid=SUser

In the end, the access log would look something like:
[13/Mar/2014:16:32:57 +0000] conn=5 op=29999 SRCH base="o=test.com" scope=2 filter="(uid=SUser)" attrs=ALL authzid="uid=puser,o=test.com"
[13/Mar/2014:16:32:57 +0000] conn=5 op=29999 RESULT err=0 tag=101 nentries=1 etime=0.001000
[13/Mar/2014:16:32:57 +0000] conn=5 op=30000 SRCH base="o=test.com" scope=2 filter="(uid=SUser)" attrs=ALL authzid="uid=puser,o=test.com"
[13/Mar/2014:16:32:57 +0000] conn=5 op=30000 RESULT err=0 tag=101 nentries=1 etime=0.001000
[13/Mar/2014:16:34:04 +0000] conn=5 op=30002 UNBIND
[13/Mar/2014:16:34:04 +0000] conn=5 op=30002 fd=66 closed - U1

You can see the memory grow with this command:
for((i=0;;++i)) { echo `date` ` grep VmSize /proc/\`pidof ns-slapd\`/status | grep -o '[0-9]*'`; sleep 1 || break; }

Note that when the connection/search finishes, the memory is not freed. If you run the test again it will grow again from the new baseline. The original (more complex and slow) load test caused the ns-slapd process to be aborted due to out-of-memory after around a week.

Comment 1 mreynolds 2014-03-18 20:24:42 UTC
Fixed upstream.

Comment 3 Viktor Ashirov 2015-01-09 16:50:25 UTC
Created attachment 978294 [details]
valgrind output

Verification steps: https://bugzilla.redhat.com/show_bug.cgi?id=1077895#c5

$ rpm -qa | grep 389-ds

Database was initialized with Setup.ldif from https://bugzilla.redhat.com/attachment.cgi?id=878275

$ date; java -cp ldapbp-repackaged-4.1.jar:. LoadTest; date
Fri Jan  9 17:26:17 CET 2015
Fri Jan  9 17:41:49 CET 2015

$ sudo /usr/sbin/stop-dirsrv 
Stopping instance "rhel7ds"

$ grep proxyauth_get_dn /tmp/valgrind-20150109-172454-rhel7ds.out  | wc -l 

Marking as VERIFIED.

Comment 5 errata-xmlrpc 2015-03-05 09:34:06 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.