Bug 681379

Summary: ns-slapd segfaults if I have more than 100 DBs
Product: Red Hat Enterprise Linux 6 Reporter: Rich Megginson <rmeggins>
Component: 389-ds-baseAssignee: Rich Megginson <rmeggins>
Status: CLOSED ERRATA QA Contact: Chandrasekar Kannan <ckannan>
Severity: high Docs Contact:
Priority: unspecified    
Version: 6.1CC: benl, diego, dpal, jgalipea
Target Milestone: rcKeywords: screened
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: 389-ds-base-1.2.8-0.6.rc1.el6 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 680555 Environment:
Last Closed: 2011-05-19 12:42:26 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 680555    
Bug Blocks: 639035, 656390, 680575    

Description Rich Megginson 2011-03-01 23:42:11 UTC
+++ This bug was initially created as a clone of Bug #680555 +++

Description of problem:
If you have more than 100 sub suffixes with their databases, 389 DS segfaults.


Version-Release number of selected component (if applicable):
1.2.8.a2


How reproducible:
Execute this from command line 150 times:
ldapadd -x -D uid=superuser,ou=People,dc=domain,dc=ar -w superpass << EOF
dn: cn=ou=$1\,dc=domain\,dc=ar,cn=mapping tree,cn=config
objectclass: top
objectclass: extensibleObject
objectclass: nsMappingTree
nsslapd-state: backend 
nsslapd-backend: $1
nsslapd-parent-suffix: dc=domain,dc=ar
cn: ou=$1,dc=domain,dc=ar

dn: cn=$1,cn=ldbm database,cn=plugins,cn=config
objectclass: extensibleObject
objectclass: nsBackendInstance
nsslapd-suffix: ou=$1,dc=domain,dc=ar

dn: ou=$1,dc=domain,dc=ar
objectClass: organizationalUnit
objectClass: top
ou: $1  
description: $1
EOF     

  
Actual results:
Running ns-slapd from gdb:

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff55c11c4 in PR_RWLock_Unlock () from /usr/lib/libnspr4.so.0d
(gdb) bt
#0  0x00007ffff55c11c4 in PR_RWLock_Unlock () from /usr/lib/libnspr4.so.0d
#1  0x00007ffff7b5cb60 in slapi_mapping_tree_free_all (be_list=<value optimized out>,
    referral_list=0x7fffffffd2e0) at ldap/servers/slapd/mapping_tree.c:2272
#2  0x00007ffff7b68710 in op_shared_search (pb=0x4787410,
    send_result=<value optimized out>) at ldap/servers/slapd/opshared.c:872
#3  0x00007ffff7b72609 in search_internal_callback_pb (pb=0x4787410,
    callback_data=<value optimized out>, prc=<value optimized out>,
    psec=0x7fffed0ab760 <views_dn_views_cb>, prec=0)
    at ldap/servers/slapd/plugin_internal_op.c:761
#4  0x00007fffed0ac3ea in views_cache_add_dn_views ()
    at ldap/servers/plugins/views/views.c:1307
#5  views_cache_build_view_list () at ldap/servers/plugins/views/views.c:1177
#6  views_cache_create () at ldap/servers/plugins/views/views.c:439
#7  0x00007fffed0ac554 in views_start (pb=<value optimized out>)
    at ldap/servers/plugins/views/views.c:254
#8  0x00007ffff7b6f6cd in plugin_call_func (list=0x99a500, operation=212, pb=0x8398f0,
    call_one=1) at ldap/servers/slapd/plugin.c:1428
#9  0x00007ffff7b70046 in plugin_call_one (argc=9, argv=0x7fffffffe5e8,
    errmsg=<value optimized out>, operation=<value optimized out>)
    at ldap/servers/slapd/plugin.c:1396
#10 plugin_dependency_startall (argc=9, argv=0x7fffffffe5e8,
    errmsg=<value optimized out>, operation=<value optimized out>)
    at ldap/servers/slapd/plugin.c:1187
#11 0x000000000041dfaa in main (argc=9, argv=0x7fffffffe5e8)


I found that DS works if I set BE_LIST_SIZE to 300 in ldap/servers/slapd/slap.h

The problem is in ldap/servers/slapd/mapping_tree.c:slapi_mapping_tree_select_all() too.

Line 2181 should be: "while ((node) &&(index < BE_LIST_SIZE -1))" to let space for the NULL value assigned after the loop. A warning if we have more than BE_LIST_SIZE datases would be great.

--- Additional comment from rmeggins on 2011-03-01 17:40:58 EST ---

Created attachment 481729 [details]
0001-Bug-680555-ns-slapd-segfaults-if-I-have-more-than-10.patch

--- Additional comment from rmeggins on 2011-03-01 18:41:45 EST ---

To ssh://git.fedorahosted.org/git/389/ds.git
   e3c72d0..6c4eac9  master -> master
commit 6c4eac9ca642b99d7664d3a6b04067c3091f5694
Author: Rich Megginson <rmeggins>
Date:   Tue Mar 1 15:36:11 2011 -0700
    Reviewed by: nhosoi, nkinder (Thanks!)
    Branch: master
    Fix Description: 1) slapi_mapping_tree_select_all() does
    be_list[BE_LIST_SIZE] = NULL
    so be_list must be of size BE_LIST_SIZE+1
    2) loop counter should check be_index, not index, to see if the loop is
    completed
    3) if the search is going to hit more backends than we can process, just
    return ADMINLIMIT_EXCEEDED with an explanatory error message
    4) increase the BE_LIST_SIZE to 1000
    Platforms tested: RHEL6 x86_64
    Flag Day: no
    Doc impact: no
To ssh://git.fedorahosted.org/git/389/ds.git
   1ba8420..ef1cb3d  389-ds-base-1.2.8 -> 389-ds-base-1.2.8
commit ef1cb3d053888274a8b7d0f59c8392427b01e783

Comment 3 Chandrasekar Kannan 2011-04-17 22:59:42 UTC
Ran the following script with the latest 389-ds-base and no crash was observed. 
marking bug verified. 389-ds-base-1.2.8.2-1.el6.i686

[root@ds90-rhel6-32vm ~]# cat run.sh 
i=1
while [ $i -le 200 ];
do
ldapadd -x -D "cn=directory manager" -w Secret123 << EOF
dn: cn=ou=a-$i\,dc=idm\,dc=lab\,dc=bos\,dc=bos\,dc=redhat\,dc=com\,cn=mapping tree,cn=config
objectclass: top
objectclass: extensibleObject
objectclass: nsMappingTree
nsslapd-state: backend 
nsslapd-backend: a-$i
nsslapd-parent-suffix: dc=idm,dc=lab,dc=bos,dc=redhat,dc=com
cn: ou=a-$i,dc=idm,dc=lab,dc=bos,dc=redhat,dc=com

dn: cn=a-$i,cn=ldbm database,cn=plugins,cn=config
objectclass: extensibleObject
objectclass: nsBackendInstance
nsslapd-suffix: ou=a-$i,dc=idm,dc=lab,dc=bos,dc=redhat,dc=com

dn: ou=a-$i,dc=idm,dc=lab,dc=bos,dc=redhat,dc=com
objectClass: organizationalUnit
objectClass: top
ou: a-$i  
description: a-$i
EOF

i=`expr $i + 1`
done

Comment 4 errata-xmlrpc 2011-05-19 12:42:26 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHEA-2011-0533.html