Bug 1028344 - Slow ldapmodify operation time for large quantities of multi-valued attribute values
Summary: Slow ldapmodify operation time for large quantities of multi-valued attribute...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: 389-ds-base
Version: 6.5
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: rc
: ---
Assignee: Ludwig
QA Contact: Sankar Ramalingam
URL:
Whiteboard:
: 1068579 (view as bug list)
Depends On: 839344
Blocks: 994246 1056252 1061410 1068579
TreeView+ depends on / blocked
 
Reported: 2013-11-08 09:21 UTC by Arpit Tolani
Modified: 2020-09-13 20:09 UTC (History)
13 users (show)

Fixed In Version: 389-ds-base-1.2.11.15-34.el6
Doc Type: Bug Fix
Doc Text:
Clone Of: 839344
: 1068579 (view as bug list)
Environment:
Last Closed: 2014-10-14 07:52:07 UTC
Target Upstream Version:


Attachments (Terms of Use)
valgrind.out (84.65 KB, text/plain)
2014-08-06 07:06 UTC, Viktor Ashirov
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Github 389ds 389-ds-base issues 346 0 None None None 2020-09-13 20:09:15 UTC
Red Hat Product Errata RHBA-2014:1385 0 normal SHIPPED_LIVE 389-ds-base bug fix and enhancement update 2014-10-14 01:27:42 UTC

Comment 2 Ludwig 2013-11-08 17:25:47 UTC
what are the exact operations they process to add 100k members, is this a bulk add in one modify or are they added one by one. 
can they provide their configuration and example data to reproduce.

Comment 4 Ludwig 2013-11-12 15:16:51 UTC
ok this is one modify, but the add is split into many single operations, it should be faster if it is done like this:

dn: cn=pds,ou=Groups,dc=extfac,dc=filhetallard,dc=com
changetype: modify
add: member
member: uid=P201866,ou=People,dc=extfac,dc=filhetallard,dc=com
member: uid=P201871,ou=People,dc=extfac,dc=filhetallard,dc=com
member: uid=P201873,ou=People,dc=extfac,dc=filhetallard,dc=com

the difference is that for teh single adds to check for the presence of the member a linear scan is done, but if more values are added at once a binary tree is used, which is faster. could you modify the add and test again

Comment 14 Jamie Duncan 2014-02-24 22:25:30 UTC
*** Bug 1068579 has been marked as a duplicate of this bug. ***

Comment 17 Noriko Hosoi 2014-03-04 21:28:29 UTC
To verify this bug, see https://bugzilla.redhat.com/show_bug.cgi?id=839344#c3.

Comment 25 Viktor Ashirov 2014-07-22 16:12:28 UTC
I could successfully add 100000 users as member for a single group in less than 3 secs. Hence, marking the bug as verified.

adding group: cn=g-100000
adding new entry "cn=g-100000,ou=groups,dc=example,dc=com"


real	0m2.079s
user	0m0.036s
sys	0m0.019s

Comment 26 Noriko Hosoi 2014-07-24 21:27:03 UTC
Minor memory leak was found.
https://fedorahosted.org/389/ticket/346#comment:83

Functionality wise, there's no change.

Comment 27 Viktor Ashirov 2014-07-28 15:08:10 UTC
Noriko, could you please provide additional verification steps for the minor memory leak? 

Thanks!

Comment 28 Noriko Hosoi 2014-07-28 17:42:11 UTC
If you run the test (comment 25) via valgrind and if the output does not include a function name attrlist_find_or_create_locking_optional, the fix is verified.
Thanks, Viktor!

Comment 29 Noriko Hosoi 2014-08-05 23:33:49 UTC
Hi Viktor, do you see "VERIFIED" in the status menu?  Could you change it to "VERIFIED"?

Thanks!
--noriko

Comment 30 Viktor Ashirov 2014-08-06 07:06:39 UTC
Created attachment 924368 [details]
valgrind.out

I repeated the test under valgrind. There are no mentions of attrlist_find_or_create_locking_optional in the output. 

Marking as VERIFIED.

Comment 31 errata-xmlrpc 2014-10-14 07:52:07 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2014-1385.html


Note You need to log in before you can comment on or make changes to this bug.