Bug 1028344

Summary: Slow ldapmodify operation time for large quantities of multi-valued attribute values
Product: Red Hat Enterprise Linux 6 Reporter: Arpit Tolani <atolani>
Component: 389-ds-baseAssignee: Ludwig <lkrispen>
Status: CLOSED ERRATA QA Contact: Sankar Ramalingam <sramling>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 6.5CC: ddas, degts, dsirrine, jduncan, jgalipea, klamb, lkrispen, msauton, nhosoi, nkinder, nobody, vashirov, yjog
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: 389-ds-base-1.2.11.15-34.el6 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 839344
: 1068579 (view as bug list) Environment:
Last Closed: 2014-10-14 07:52:07 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 839344    
Bug Blocks: 994246, 1056252, 1061410, 1068579    
Attachments:
Description Flags
valgrind.out none

Comment 2 Ludwig 2013-11-08 17:25:47 UTC
what are the exact operations they process to add 100k members, is this a bulk add in one modify or are they added one by one. 
can they provide their configuration and example data to reproduce.

Comment 4 Ludwig 2013-11-12 15:16:51 UTC
ok this is one modify, but the add is split into many single operations, it should be faster if it is done like this:

dn: cn=pds,ou=Groups,dc=extfac,dc=filhetallard,dc=com
changetype: modify
add: member
member: uid=P201866,ou=People,dc=extfac,dc=filhetallard,dc=com
member: uid=P201871,ou=People,dc=extfac,dc=filhetallard,dc=com
member: uid=P201873,ou=People,dc=extfac,dc=filhetallard,dc=com

the difference is that for teh single adds to check for the presence of the member a linear scan is done, but if more values are added at once a binary tree is used, which is faster. could you modify the add and test again

Comment 14 Jamie Duncan 2014-02-24 22:25:30 UTC
*** Bug 1068579 has been marked as a duplicate of this bug. ***

Comment 17 Noriko Hosoi 2014-03-04 21:28:29 UTC
To verify this bug, see https://bugzilla.redhat.com/show_bug.cgi?id=839344#c3.

Comment 25 Viktor Ashirov 2014-07-22 16:12:28 UTC
I could successfully add 100000 users as member for a single group in less than 3 secs. Hence, marking the bug as verified.

adding group: cn=g-100000
adding new entry "cn=g-100000,ou=groups,dc=example,dc=com"


real	0m2.079s
user	0m0.036s
sys	0m0.019s

Comment 26 Noriko Hosoi 2014-07-24 21:27:03 UTC
Minor memory leak was found.
https://fedorahosted.org/389/ticket/346#comment:83

Functionality wise, there's no change.

Comment 27 Viktor Ashirov 2014-07-28 15:08:10 UTC
Noriko, could you please provide additional verification steps for the minor memory leak? 

Thanks!

Comment 28 Noriko Hosoi 2014-07-28 17:42:11 UTC
If you run the test (comment 25) via valgrind and if the output does not include a function name attrlist_find_or_create_locking_optional, the fix is verified.
Thanks, Viktor!

Comment 29 Noriko Hosoi 2014-08-05 23:33:49 UTC
Hi Viktor, do you see "VERIFIED" in the status menu?  Could you change it to "VERIFIED"?

Thanks!
--noriko

Comment 30 Viktor Ashirov 2014-08-06 07:06:39 UTC
Created attachment 924368 [details]
valgrind.out

I repeated the test under valgrind. There are no mentions of attrlist_find_or_create_locking_optional in the output. 

Marking as VERIFIED.

Comment 31 errata-xmlrpc 2014-10-14 07:52:07 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2014-1385.html