Bug 1028344 - Slow ldapmodify operation time for large quantities of multi-valued attribute values
Slow ldapmodify operation time for large quantities of multi-valued attribute...
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: 389-ds-base (Show other bugs)
Unspecified Unspecified
urgent Severity urgent
: rc
: ---
Assigned To: Ludwig
Sankar Ramalingam
: 1068579 (view as bug list)
Depends On: 839344
Blocks: 994246 1056252 1061410 1068579
  Show dependency treegraph
Reported: 2013-11-08 04:21 EST by Arpit Tolani
Modified: 2014-10-14 03:52 EDT (History)
13 users (show)

See Also:
Fixed In Version: 389-ds-base-
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 839344
: 1068579 (view as bug list)
Last Closed: 2014-10-14 03:52:07 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
valgrind.out (84.65 KB, text/plain)
2014-08-06 03:06 EDT, Viktor Ashirov
no flags Details

  None (edit)
Comment 2 Ludwig 2013-11-08 12:25:47 EST
what are the exact operations they process to add 100k members, is this a bulk add in one modify or are they added one by one. 
can they provide their configuration and example data to reproduce.
Comment 4 Ludwig 2013-11-12 10:16:51 EST
ok this is one modify, but the add is split into many single operations, it should be faster if it is done like this:

dn: cn=pds,ou=Groups,dc=extfac,dc=filhetallard,dc=com
changetype: modify
add: member
member: uid=P201866,ou=People,dc=extfac,dc=filhetallard,dc=com
member: uid=P201871,ou=People,dc=extfac,dc=filhetallard,dc=com
member: uid=P201873,ou=People,dc=extfac,dc=filhetallard,dc=com

the difference is that for teh single adds to check for the presence of the member a linear scan is done, but if more values are added at once a binary tree is used, which is faster. could you modify the add and test again
Comment 14 Jamie Duncan 2014-02-24 17:25:30 EST
*** Bug 1068579 has been marked as a duplicate of this bug. ***
Comment 17 Noriko Hosoi 2014-03-04 16:28:29 EST
To verify this bug, see https://bugzilla.redhat.com/show_bug.cgi?id=839344#c3.
Comment 25 Viktor Ashirov 2014-07-22 12:12:28 EDT
I could successfully add 100000 users as member for a single group in less than 3 secs. Hence, marking the bug as verified.

adding group: cn=g-100000
adding new entry "cn=g-100000,ou=groups,dc=example,dc=com"

real	0m2.079s
user	0m0.036s
sys	0m0.019s
Comment 26 Noriko Hosoi 2014-07-24 17:27:03 EDT
Minor memory leak was found.

Functionality wise, there's no change.
Comment 27 Viktor Ashirov 2014-07-28 11:08:10 EDT
Noriko, could you please provide additional verification steps for the minor memory leak? 

Comment 28 Noriko Hosoi 2014-07-28 13:42:11 EDT
If you run the test (comment 25) via valgrind and if the output does not include a function name attrlist_find_or_create_locking_optional, the fix is verified.
Thanks, Viktor!
Comment 29 Noriko Hosoi 2014-08-05 19:33:49 EDT
Hi Viktor, do you see "VERIFIED" in the status menu?  Could you change it to "VERIFIED"?

Comment 30 Viktor Ashirov 2014-08-06 03:06:39 EDT
Created attachment 924368 [details]

I repeated the test under valgrind. There are no mentions of attrlist_find_or_create_locking_optional in the output. 

Marking as VERIFIED.
Comment 31 errata-xmlrpc 2014-10-14 03:52:07 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.