Bug 631862 - crash - delete entries not in cache + referint
Summary: crash - delete entries not in cache + referint
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: 389
Classification: Retired
Component: Directory Server
Version: 1.2.6
Hardware: All
OS: Linux
high
medium
Target Milestone: ---
Assignee: Rich Megginson
QA Contact: Viktor Ashirov
URL:
Whiteboard:
: 636764 (view as bug list)
Depends On:
Blocks: 389_1.2.6 639035
TreeView+ depends on / blocked
 
Reported: 2010-09-08 14:43 UTC by Mike McGrath
Modified: 2015-12-07 16:35 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-12-07 16:35:19 UTC
Embargoed:


Attachments (Terms of Use)
Deletion script (393 bytes, text/x-python)
2010-09-08 14:43 UTC, Mike McGrath
no flags Details
This is the trace I got from gdb (14.31 KB, text/plain)
2010-09-08 14:44 UTC, Mike McGrath
no flags Details
0001-Bug-631862-crash-delete-entries-not-in-cache-r.patch (5.82 KB, patch)
2010-09-13 19:12 UTC, Rich Megginson
nhosoi: review-
Details | Diff
tentative patch (10.47 KB, patch)
2010-09-13 23:19 UTC, Noriko Hosoi
no flags Details | Diff
git patch file (master) (12.71 KB, patch)
2010-09-14 17:15 UTC, Noriko Hosoi
nhosoi: review?
nhosoi: review?
rmeggins: review+
Details | Diff
stack trace from crash with two search threads (28.40 KB, text/plain)
2010-09-29 21:04 UTC, Rich Megginson
no flags Details
another two search stack trace (28.94 KB, text/plain)
2010-09-29 21:04 UTC, Rich Megginson
no flags Details

Description Mike McGrath 2010-09-08 14:43:36 UTC
Created attachment 445999 [details]
Deletion script

Description of problem:

Trying to delete users causes ns-slapd to segfault after 5-10 deletes.

Version-Release number of selected component (if applicable):

I've got a standard ipa install on F13 (1.9.0-pre4-0) with 389-ds-base-1.2.6-0.8.rc3.fc13.i686


How reproducible:

Every time.

Steps to Reproduce:
1.  I've got 30,000 or so entries in the ldap db.  Then run a script (attached)
2.  It will run for 5-10 users deleting them
3.  Crash
  
Actual results:

seg fault, stack trace attached.

Additional info:

If this trace isn't enough I might be able to get you a copy of an ldap backup I'm using.

Comment 1 Mike McGrath 2010-09-08 14:44:02 UTC
Created attachment 446000 [details]
This is the trace I got from gdb

Comment 2 Rich Megginson 2010-09-08 15:20:15 UTC
(In reply to comment #1)
> Created attachment 446000 [details]
> This is the trace I got from gdb

Can you try with 389-ds-base-1.2.6-1?
Does it crash if you disable the referential integrity plugin?

Comment 3 Mike McGrath 2010-09-08 16:55:01 UTC
After upgrade it still failed, after disabling the integrity plugin it worked fine.  After re-enabling the plugin it would fail again.

Comment 4 Rich Megginson 2010-09-08 22:57:48 UTC
Tried to reproduce with Example.ldif + referint + memberof + 150 groups each containing every user - then deleted every user entry one by one - no crash

Trying with 1000 users

Comment 5 Rich Megginson 2010-09-10 20:45:04 UTC
tried with 1150 users + 100 groups - every user is in all groups + referint + memberof - no crash - running in valgrind now - no problems yet

Comment 6 Mike McGrath 2010-09-10 20:52:42 UTC
FWIW, I've got around 30,000 users.  I can provide a dump if that would be useful.

Comment 7 Rich Megginson 2010-09-10 21:55:33 UTC
Got it.  The trick is to shutdown the server, start it up, then immediately delete users before doing other ops.  I'm using a debug build with assertions enabled and got a slightly different stack trace:
(gdb) bt full
#0  0x0000003950c30265 in raise () from /lib64/libc.so.6
No symbol table info available.
#1  0x0000003950c31d10 in abort () from /lib64/libc.so.6
No symbol table info available.
#2  0x0000003950c6a84b in __libc_message () from /lib64/libc.so.6
No symbol table info available.
#3  0x0000003950c75e71 in free_check () from /lib64/libc.so.6
No symbol table info available.
#4  0x0000003950c727f1 in free () from /lib64/libc.so.6
No symbol table info available.
#5  0x00002af75f7ff888 in slapi_ch_free (ptr=0x479aebd0)
    at ../ds126.git/ldap/servers/slapd/ch_malloc.c:363
No locals.
#6  0x00002af75f7ff907 in slapi_ch_free_string (s=0x479aebd0)
    at ../ds126.git/ldap/servers/slapd/ch_malloc.c:396
No locals.
#7  0x00002af75f7fb26f in slapi_attr_is_dn_syntax_attr (attr=0x1098d750)
    at ../ds126.git/ldap/servers/slapd/attrsyntax.c:833
        syntaxoid = 0x10a87610 "1.3.6.1.4.1.1466.115.121.1.12"
        dn_syntax = 1
#8  0x00002af75f80e807 in str2entry_fast (rawdn=0x0, s=0x10aa28d7 "member", 
    flags=0, read_stateinfo=-1) at ../ds126.git/ldap/servers/slapd/entry.c:430
        value = 0x10a874a0
        a = 0x109dc150
        valuelen = 43
        value_state = 2
        type = 0x10aa28d7 "member"
        freetype = 0
        valuecharptr = 0x10aa28df "uid=user_id_149,ou=people,dc=example,dc=com"
        attr_state = 2
        maxvals = 512
        del_maxvals = 0
        e = 0x10a76330
        next = 0x10aa290b "member: uid=user_id_150,ou=people,dc=example,dc=com\nmember: uid=user_id_151,ou=people,dc=example,dc=com\nmember: uid=user_id_152,ou=people,dc=example,dc=com\nmember: uid=user_id_153,ou=people,dc=example"...
        ptype = 0x10922de0 "member"
        nvals = 297
        del_nvals = 0
        retmalloc = 0
        attr_val_cnt = 299
        attributedeletioncsn = 0x0
        valuecsnset = 0x0
        maxcsn = 0x0
        normdn = 0x109d8220 "cn=group_9,ou=Groups,dc=example,dc=com"
        strict = 0
#9  0x00002af75f8108c7 in slapi_str2entry_ext (
    dn=0x10922db0 "cn=group_9,ou=Groups,dc=example,dc=com", 
    s=0x10a9ef20 "rdn", flags=0)
    at ../ds126.git/ldap/servers/slapd/entry.c:1381
        e = 0x1096bac0
        read_stateinfo = -1
#10 0x00002af7632ee5b5 in id2entry_ext (be=0x107375a0, id=1169, txn=0x0, 
    err=0x479b0ed0, flags=1)
    at ../ds126.git/ldap/servers/slapd/back-ldbm/id2entry.c:324
        dn = 0x10922db0 "cn=group_9,ou=Groups,dc=example,dc=com"
        bdn = 0x10a0a9b0
        rdn = 0x10a68e90 "cn=group_9"
        rc = 0
        inst = 0x106ade90
        db = 0x1073b3c0
        db_txn = 0x0
        key = {data = 0x479b0e10, size = 4, ulen = 0, dlen = 0, doff = 0, 
          flags = 0}
        data = {data = 0x10a9ef20, size = 59371, ulen = 0, dlen = 0, doff = 0, 
          flags = 4}
        e = 0x0
        ee = 0x2af7632f46c5
        temp_id = "\000\000\004\221"
#11 0x00002af763327ff2 in ldbm_back_next_search_entry_ext (pb=0x10925af0, 
    use_extension=0)
    at ../ds126.git/ldap/servers/slapd/back-ldbm/ldbm_search.c:1244
        be = 0x107375a0
        inst = 0x106ade90
        li = 0x106685d0
        scope = 2
        managedsait = 0
        attr = 0x10a20b60
        filter = 0x109244f0
        base = 0x1076c290 "dc=example,dc=com"
        sr = 0x1093cb00
        id = 1169
        e = 0x2af75f811efb
        nentries = 8
        curtime = 1284153642
        stoptime = 1284153641
        optime = 1284153642
        tlimit = -1
        llimit = -1
        slimit = -1
        isroot = 1
        urls = 0x0
        err = 0
        basesdn = {flag = 0 '\000', dn = 0x1076c290 "dc=example,dc=com", 
          ndn = 0x1076c290 "dc=example,dc=com", ndn_len = 17}
        target_uniqueid = 0x0
        rc = 0
#12 0x00002af763327a8a in ldbm_back_next_search_entry (pb=0x10925af0)
    at ../ds126.git/ldap/servers/slapd/back-ldbm/ldbm_search.c:1095
No locals.
#13 0x00002af75f843dc8 in iterate (pb=0x10925af0, be=0x107375a0, 
    send_result=1, pnentries=0x479b37cc, pagesize=-1, pr_statp=0x479b37c0)
    at ../ds126.git/ldap/servers/slapd/opshared.c:1088
        gerentry = 0x0
        operation = 0x10977bd0
        rc = 0
        rval = 1
        attrsonly = 0
        done = 0
        e = 0x10a20b60
        attrs = 0x0
        pr_stat = 0
#14 0x00002af75f8445d3 in send_results_ext (pb=0x10925af0, send_result=1, 
    nentries=0x479b37cc, pagesize=-1, pr_stat=0x479b37c0)
    at ../ds126.git/ldap/servers/slapd/opshared.c:1457
        be = 0x107375a0
        rc = 1010966528
#15 0x00002af75f843595 in op_shared_search (pb=0x10925af0, send_result=1)
    at ../ds126.git/ldap/servers/slapd/opshared.c:688
        be_suffix = 0x106acfc0
        err = 0
        next_be = 0x0
        base = 0x1061ed00 "dc=example,dc=com"
        fstr = 0x109c2b00 "(member=uid=tmorris,ou=people,dc=example,dc=com)"
        scope = 2
        be = 0x107375a0
        be_single = 0x0
        be_list = {0x107375a0, 0x0 <repeats 92 times>, 0x1086bb80, 0x0, 
          0x2af75f8991b0, 0x109315d0, 0x0, 0x0, 0x0}
        referral_list = {0x0 <repeats 84 times>, 0x107375a0, 
          0x0 <repeats 15 times>}
        ebuf = '\000' <repeats 416 times>"\222, \225U\020\000\000\000\000R[\233G", '\000' <repeats 20 times>"\340, Y\233G\000\000\000\000\315\b\203_\367*\000\000\200\225U\020\000\000\000\000\370*Bb\367*\000\000\237eBb\367*\000\000\202\000\000\000\020\000\000\000H\177U\020\000\000\000\000@[\233G\000\000\000\000\060Z\233G\351\a\035\272\020\252U\020\000\000\000\000@[\233G\000\000\000\000\020\226U\020\000\000\000\000H\177U\020\000\000\000\000`\350\207\020\000\000\000\000\200Z\233G\000\000\000\000\000\000\000\000m\000\000\000\300Z\233G\000\000\000\000\001\000---Type <return> to continue, or q <return> to quit---
\000\000\000\000\000\000\002\000\000\000\000\000\000\000'`Bb\367*\000\000P8\222\020\000\000\000\000\376\254\207_\367*\000\000\340[\233G\000\000\000\000\240us\020'\000\000\000P8\222\020\000\000\000\000|\372Ab\367*\000\000"...
        attrlistbuf = "P\000\000\000\000\000\000\000\061\000\000\000\000\000\000\000H\000\000\000\000\000\000\000\300z\233G", '\000' <repeats 12 times>"\243, $!d\000\001\000\000\000\000\000\000\000\000\000\000\220\373\221\020\017\000\000\000Px\233G\000\000\000\000p\373\221\020\000\000\000\000p\373\221\020\000\000\000\000G\000\000\000\000\000\000\000\b\000\000\000\000\000\000\000G\000\000\000\000\000\000\000\320\025\223\020\000\000\000\000GT\307P9\000\000\000\033[\233", '\000' <repeats 13 times>, "!\000\000\000\000\000\000\000g\306-c\367*\000\000\000\000\000\000\000\000\000\000\262V\200_\367*\000\000\000\000\000\000\000\000\000\000p\373\221\020\000\000\000,\340z\233G\000\000\000\000\350z\233G\000\000\000\000'\000\000\000\000\000\000\000@\376\207\020\000\000\000\000\320\025\223\020\000\000\000\000NT!"...
        attrliststr = 0x0
        attrs = 0x0
        rc = 0
        internal_op = 32
        sdn = {flag = 4 '\004', dn = 0x1061ed00 "dc=example,dc=com", 
          ndn = 0x10a07c70 "dc=example,dc=com", ndn_len = 17}
        operation = 0x10977bd0
        referral = 0x0
        errorbuf = '\000' <repeats 5112 times>, "L\227*f9", '\000' <repeats 11 t
imes>"\260, L\233G\000\000\000\000\300\245j\020\000\000\000\000\313\317*f9\000\000\000p\033v\020\000\000\000\000\270L\233G\000\000\000\000p\033v\020\000\000\000\000\340\061\223\020\000\000\000\000p\033v\020\000\000\000\000\000/^f\367*\000\000\300\245j\020\000\000\000\000\002", '\000' <repeats 15 times>, "\026\232*f9", '\000' <repeats 23 times>, "\022\000\000\200xe:f\367*", '\000' <repeats 26 times>"\220, \001\000\000\000\000\000\000\210\350'f\367*\000\000\360_\024f\367*\000\000\000\000\000\000\000\000\000\000p\033v\020\000\000\000\000\300N\233G\000\000\000\000\370#\"f9\000\000\000p\033v\020", '\000' <repeats 12 times>, "\002\000\000\000\000\000\000\000\220"...
        nentries = 0
        pnentries = 8
        flag_search_base_found = 1
        flag_no_such_object = 0
        flag_referral = 0
        flag_psearch = 0
        err_code = 0
        ctrlp = 0x0
        ctl_value = 0x0
        iscritical = 0
        be_name = 0x0
        index = -1
        sent_result = 0
        pr_stat = 0
        pagesize = -1
        curr_search_count = 0
        pr_be = 0x0
        pr_search_result = 0x0
        pr_search_result_count = 0
#16 0x00002af75f851753 in search_internal_callback_pb (pb=0x10925af0, 
    callback_data=0x479b7df0, prc=0, 
    psec=0x2af763c05e7f <memberof_del_dn_type_callback>, prec=0)
    at ../ds126.git/ldap/servers/slapd/plugin_internal_op.c:761
        controls = 0x0
        op = 0x10977bd0
        filter = 0x109244f0
        fstr = 0x1076c0e0 "(member=uid=tmorris,ou=people,dc=example,dc=com)"
        callback_handler_data = {p_res_callback = 0, 
          p_srch_entry_callback = 0x2af763c05e7f <memberof_del_dn_type_callback>, p_ref_entry_callback = 0, callback_data = 0x479b7df0}
        scope = 2
        ifstr = 0x109c2b00 "(member=uid=tmorris,ou=people,dc=example,dc=com)"
        opresult = 0
        rc = 0
        original_base = 0x1061ed00 "dc=example,dc=com"
        new_base = 0x0
#17 0x00002af75f851147 in slapi_search_internal_callback_pb (pb=0x10925af0, 
    callback_data=0x479b7df0, prc=0, 
    psec=0x2af763c05e7f <memberof_del_dn_type_callback>, prec=0)
    at ../ds126.git/ldap/servers/slapd/plugin_internal_op.c:534
No locals.

#18 0x00002af763c06072 in memberof_call_foreach_dn (pb=0x1086c0f0, 
    dn=0x109315d0 "uid=tmorris,ou=people,dc=example,dc=com", 
    type=0x10922e00 "member", 
    callback=0x2af763c05e7f <memberof_del_dn_type_callback>, 
    callback_data=0x479b7df0)
    at ../ds126.git/ldap/servers/plugins/memberof/memberof.c:454
        rc = 0
        search_pb = 0x10925af0
        be = 0x107375a0
        sdn = 0x1091fde0
        base_sdn = 0x106acfc0
        filter_str = 0x109c2b00 "(member=uid=tmorris,ou=people,dc=example,dc=com)"
#19 0x00002af763c05e7d in memberof_del_dn_from_groups (pb=0x1086c0f0, 
    config=0x479b7e20, dn=0x109315d0 "uid=tmorris,ou=people,dc=example,dc=com")
    at ../ds126.git/ldap/servers/plugins/memberof/memberof.c:374
        data = {dn = 0x109315d0 "uid=tmorris,ou=people,dc=example,dc=com", 
          type = 0x10922e00 "member"}
#20 0x00002af763c05db3 in memberof_postop_del (pb=0x1086c0f0)
    at ../ds126.git/ldap/servers/plugins/memberof/memberof.c:341
        e = 0x10925d70
        ret = 0
        configCopy = {groupattr = 0x10922e00 "member", 
          memberof_attr = 0x10943500 "memberOf", group_filter = 0x10924d70, 
          group_slapiattr = 0x10924c70}
        dn = 0x109315d0 "uid=tmorris,ou=people,dc=example,dc=com"
#21 0x00002af75f84d07b in plugin_call_func (list=0x10677a50, operation=508, 
    pb=0x1086c0f0, call_one=0) at ../ds126.git/ldap/servers/slapd/plugin.c:1424
        n = 0x10677670 "MemberOf Plugin"
        func = 0x2af763c05cdb <memberof_postop_del>
        rc = 0
        return_value = 0
        count = 6
#22 0x00002af75f84cf5c in plugin_call_list (list=0x10693710, operation=508, 
    pb=0x1086c0f0) at ../ds126.git/ldap/servers/slapd/plugin.c:1386
No locals.
#23 0x00002af75f84b53f in plugin_call_plugins (pb=0x1086c0f0, 
    whichfunction=508) at ../ds126.git/ldap/servers/slapd/plugin.c:379
        p = 0x106677c0
        plugin_list_number = 2
        rc = 0
        do_op = 1
#24 0x00002af75f804235 in op_shared_delete (pb=0x1086c0f0)
    at ../ds126.git/ldap/servers/slapd/delete.c:370
        rc = 0
        dn = 0x10932030 "uid=tmorris,ou=People,dc=example,dc=com"
        be = 0x107375a0
        ebuf = '\000' <repeats 7464 times>"\243, \065\307P9", '\000' <repeats 67 times>, ")", '\000' <repeats 31 times>, "`\350\207\020\000\000\000\000\360\275\233G\000\000\000\000P\035\222\020\000\000\000\000P\035\222\020\000\000\000\000\020\300\233G\000\000\000\000\002\227\202R9\000\000\000`\350\207\020\000\000\000\000\360\275\233G\000\000\000JP\035\222\020\000\000\000\000=\037\222\020\000\000\000\000\060 \223\020\000\000\000\000/\252\307P9\000\000\000J\000\000\000\000\000\000\000'\000\000\000\000\000\000\000P\035\222\020\000\000\000\000'", '\000' <repeats 15 times>, "\023ƂR9\000\000\000P\035\222\020\000\000\000\000\262V\200_\367*\000\000\060 \223\020\000\000\000\000)\233\202R9\000\000,\000\300\233G\000\000\000\000\b\300\233G\000\000\000\000'\000\000\000\000\000\000\000\060 \223\020\000\000\000\000"...
        internal_op = 0
        sdn = {flag = 4 '\004', 
          dn = 0x10932030 "uid=tmorris,ou=People,dc=example,dc=com", 
          ndn = 0x109315d0 "uid=tmorris,ou=people,dc=example,dc=com", 
          ndn_len = 39}
        operation = 0x10921bd0
        referral = 0x0
        ecopy = 0x10925d70
        errorbuf = '\000' <repeats 8191 times>
        err = 0
#25 0x00002af75f803b93 in do_delete (pb=0x1086c0f0)
    at ../ds126.git/ldap/servers/slapd/delete.c:143
        operation = 0x10921bd0
        ber = 0x10921d50
        rawdn = 0x10932030 "uid=tmorris,ou=People,dc=example,dc=com"
        dn = 0x10932030 "uid=tmorris,ou=People,dc=example,dc=com"
        dnlen = 39
        err = 0
#26 0x00000000004136af in connection_dispatch_operation (conn=0x2aaaaaab0010, 
    op=0x10921bd0, pb=0x1086c0f0)
    at ../ds126.git/ldap/servers/slapd/connection.c:568
        minssf = 0
#27 0x0000000000414dd7 in connection_threadmain ()
    at ../ds126.git/ldap/servers/slapd/connection.c:2313
        is_timedout = 0
        curtime = 1284153641
        pb = 0x1086c0f0
        interval = 10000
        conn = 0x2aaaaaab0010
        op = 0x10921bd0
        tag = 74
        need_wakeup = 0
        thread_turbo_flag = 0
        ret = 0
        more_data = 0
        replication_connection = 0
#28 0x000000396422867d in _pt_root (arg=<value optimized out>)
    at ../../../mozilla/nsprpub/pr/src/pthreads/ptthread.c:228
        thred = 0x1087e860
#29 0x000000395180673d in start_thread () from /lib64/libpthread.so.0
No symbol table info available.
#30 0x0000003950cd3d1d in clone () from /lib64/libc.so.6
No symbol table info available.

Comment 8 Rich Megginson 2010-09-10 22:12:30 UTC
valgrind shows a stack trace similar to the original one:
==15875== Invalid read of size 1
==15875==    at 0x4C4AD01: strcmpi_fast (intrinsics.h:95)
==15875==    by 0x4C4AE0C: attrlist_find (attrlist.c:126)
==15875==    by 0x4C62EFA: slapi_entry_attr_find (entry.c:2222)
==15875==    by 0x96A315E: ldbm_back_next_search_entry_ext (ldbm_search.c:1282)
==15875==    by 0x96A2A89: ldbm_back_next_search_entry (ldbm_search.c:1095)
==15875==    by 0x4C94DC7: iterate (opshared.c:1088)
==15875==    by 0x4C955D2: send_results_ext (opshared.c:1457)
==15875==    by 0x4C94594: op_shared_search (opshared.c:688)
==15875==    by 0x4CA2752: search_internal_callback_pb (plugin_internal_op.c:761)
==15875==    by 0x4CA2304: search_internal_pb (plugin_internal_op.c:611)
==15875==    by 0x4CA20BA: slapi_search_internal_pb (plugin_internal_op.c:515)
==15875==    by 0xA79E004: update_integrity (referint.c:705)
==15875==    by 0xA79D103: referint_postop_del (referint.c:202)
==15875==    by 0x4C9E07A: plugin_call_func (plugin.c:1424)
==15875==    by 0x4C9DF5B: plugin_call_list (plugin.c:1386)
==15875==    by 0x4C9C53E: plugin_call_plugins (plugin.c:379)
==15875==    by 0x4C55234: op_shared_delete (delete.c:370)
==15875==    by 0x4C54B92: do_delete (delete.c:143)
==15875==    by 0x4136AE: connection_dispatch_operation (connection.c:568)
==15875==    by 0x414DD6: connection_threadmain (connection.c:2313)

Comment 9 Rich Megginson 2010-09-13 19:12:21 UTC
Created attachment 447020 [details]
0001-Bug-631862-crash-delete-entries-not-in-cache-r.patch

Comment 10 Rich Megginson 2010-09-13 19:13:52 UTC
The patch applies to 1.2.7, not to 1.2.6.  I believe the workaround for 1.2.6 is to make sure the entry is in the cache before deleting it.  You can ensure this by searching for the entry before deleting it.

Comment 11 Noriko Hosoi 2010-09-13 19:38:43 UTC
Comment on attachment 447020 [details]
0001-Bug-631862-crash-delete-entries-not-in-cache-r.patch

add_update_entry_operational_attributes adds entrydn to the entry before writing it to the primary db file id2entry.db4.

# dbscan -f /path/to/id2entry.db4 | egrep entrydn:
	entrydn: dc=example,dc=com
	entrydn: ...

Comment 12 Noriko Hosoi 2010-09-13 23:19:05 UTC
Created attachment 447086 [details]
tentative patch

This is still tentative, but this patch solves the problem flushing entrydn to the id2entry.db4 accidentally.  I had to change the algorithm to ...
1) entry in the entry cache always have entrydn
2) removing entrydn before writing the entry to db

Rich, could it be possible to run the crash test against the server that includes this patch?

Comment 13 Noriko Hosoi 2010-09-14 17:15:06 UTC
Created attachment 447272 [details]
git patch file (master)

This patch is based upon the Rich's patch (447020: 0001-Bug-631862-crash-delete-entries-not-in-cache-r.patch) with 2 additions:
1) entrydn attribute is always added to the entry in memory before putting
   it in the entry cache, and the attribute is removed before writing the
   entry to the database.
2) eliminating id2entry_ext, which was introduced to pass flags, but it is
   no longer needed since only a flag ID2ENTRY_ADD_ENTRYDN was removed.

Comment 14 Rich Megginson 2010-09-14 17:36:55 UTC
Comment on attachment 447272 [details]
git patch file (master)

Tested on RHEL5 x86_64 - works.

Comment 15 Noriko Hosoi 2010-09-14 20:45:55 UTC
Reviewed by Rich (Thank you!!)

Pushed to master.

$ git merge 631862
Updating 09baba5..9098fc7
Fast-forward
 ldap/servers/slapd/back-ldbm/back-ldbm.h       |    3 -
 ldap/servers/slapd/back-ldbm/id2entry.c        |   86 +++++++++++++++---------
 ldap/servers/slapd/back-ldbm/ldbm_add.c        |   17 ++---
 ldap/servers/slapd/back-ldbm/ldbm_search.c     |    2 +-
 ldap/servers/slapd/back-ldbm/proto-back-ldbm.h |    2 -
 5 files changed, 60 insertions(+), 50 deletions(-)

$ git push
Counting objects: 20, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (11/11), done.
Writing objects: 100% (11/11), 9.38 KiB, done.
Total 11 (delta 8), reused 0 (delta 0)
To ssh://git.fedorahosted.org/git/389/ds.git
   09baba5..9098fc7  master -> master

Comment 17 Rich Megginson 2010-09-23 21:15:02 UTC
*** Bug 636764 has been marked as a duplicate of this bug. ***

Comment 18 Rich Megginson 2010-09-29 21:04:28 UTC
Created attachment 450587 [details]
stack trace from crash with two search threads

Comment 19 Rich Megginson 2010-09-29 21:04:52 UTC
Created attachment 450588 [details]
another two search stack trace

Comment 20 Noriko Hosoi 2010-09-29 22:21:42 UTC
The line number 393 shows this version does not include the patch "attachment 447272 [details]"...

#6  0x00007f5f01723c8a in slapi_entry_attr_set_charptr (e=0x7f5e6e969d50, type=0x7f5ef8387470 "entrydn", value=<value optimized out>) at ldap/servers/slapd/entry.c:2757
#7  0x00007f5ef834195e in id2entry_ext (be=0xb906f0, id=154598, txn=0x0, err=<value optimized out>, flags=1) at ldap/servers/slapd/back-ldbm/id2entry.c:393 <== 

id2entry.c before applying the patch "attachment 447272 [details]"
378 bail:
379     /* 
380      * If return entry exists AND adding entrydn is requested AND
381      * entryrdn switch is on, add the entrydn value.
382      */
383     if (e && e->ep_entry && (flags & ID2ENTRY_ADD_ENTRYDN) &&
384         entryrdn_get_switch()) {
385         Slapi_Attr *eattr = NULL;
386         /* Check if entrydn is in the entry or not */ 
387         if (slapi_entry_attr_find(e->ep_entry, "entrydn", &eattr)) {
388             /* entrydn does not exist in the entry */
389             char *entrydn = NULL;
390             /* slapi_ch_strdup and slapi_dn_ignore_case never returns NULL *    /       
391             entrydn = slapi_ch_strdup(slapi_entry_get_dn_const(e->ep_entry))    ;       
392             entrydn = slapi_dn_ignore_case(entrydn);
393             slapi_entry_attr_set_charptr (e->ep_entry, "entrydn", entrydn);
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Comment 23 Noriko Hosoi 2010-10-08 00:01:19 UTC
It was verified that this crash problem does not occur on 389 v1.2.6.1.

Changing the bug status to MODIFIED.

Comment 24 Amita Sharma 2011-05-16 10:04:12 UTC
I tried 
1. Stop server
2. Start Server
3. ldapdelete , but no crash..

anything else I need to do to test this bug?

Comment 25 Noriko Hosoi 2011-05-16 16:33:34 UTC
(In reply to comment #24)
> I tried 
> 1. Stop server
> 2. Start Server
> 3. ldapdelete , but no crash..
> 
> anything else I need to do to test this bug?

This bug was observed when the memberof plugin and the referential integrity plugin are enabled.  To verify this bug, please enable the 2 plugins.  Then, add user entries and group entries which sets up the member - memberof relationships.  Finally, delete all the user and group entries.  If the server does not get crashed, we could mark this bug verified.

Comment 26 Amita Sharma 2011-05-17 11:45:24 UTC
dn: cn=referential integrity postoperation,cn=plugins,cn=config
objectClass: top
objectClass: nsSlapdPlugin
objectClass: extensibleObject
cn: referential integrity postoperation
nsslapd-pluginPath: libreferint-plugin
nsslapd-pluginInitfunc: referint_postop_init
nsslapd-pluginType: postoperation
nsslapd-pluginEnabled: on


dn: cn=MemberOf Plugin,cn=plugins,cn=config
objectClass: top
objectClass: nsSlapdPlugin
objectClass: extensibleObject
cn: MemberOf Plugin
nsslapd-pluginPath: libmemberof-plugin
nsslapd-pluginInitfunc: memberof_postop_init
nsslapd-pluginType: postoperation
nsslapd-pluginEnabled: on
nsslapd-plugin-depends-on-type: database
memberofgroupattr: member
memberofattr: memberOf
nsslapd-pluginId: memberof
nsslapd-pluginVersion: 1.2.8.3
nsslapd-pluginVendor: 389 Project
nsslapd-pluginDescription: memberof plugin

Did the rest steps from console, Server did not crash.

But few observations here :
1. On DS console, I have one suffix dc=test,dc=redhat,dc=com..then under dc=test.. I have two OUs..
1. ou=Users
having users abc, xyz ... etc.
2. ou=Groups
having groups like admin, domain, unix ..etc
3. When I have added users to the groups from console.. using group's properties wizard.. members tab...
4. Saw that all the users disappeared from the ou=users.. then they came after I did a refresh.. same thing happened with group also.. when I clicked on ou=groups.. all groups disappeared..after a refresh they came... bizarre..?!?

......

then I have noticed that.. there is no field on the user's properties (DS Console)to tell to which group this user belongs to.. 
I could not see that under advanced properties too?
..............

Certainly, these are not related to the original issue but if these are valid to open a new bug ?? Please advice

Comment 27 Rich Megginson 2011-05-17 17:01:54 UTC
(In reply to comment #26)
> dn: cn=referential integrity postoperation,cn=plugins,cn=config
> objectClass: top
> objectClass: nsSlapdPlugin
> objectClass: extensibleObject
> cn: referential integrity postoperation
> nsslapd-pluginPath: libreferint-plugin
> nsslapd-pluginInitfunc: referint_postop_init
> nsslapd-pluginType: postoperation
> nsslapd-pluginEnabled: on
> 
> 
> dn: cn=MemberOf Plugin,cn=plugins,cn=config
> objectClass: top
> objectClass: nsSlapdPlugin
> objectClass: extensibleObject
> cn: MemberOf Plugin
> nsslapd-pluginPath: libmemberof-plugin
> nsslapd-pluginInitfunc: memberof_postop_init
> nsslapd-pluginType: postoperation
> nsslapd-pluginEnabled: on
> nsslapd-plugin-depends-on-type: database
> memberofgroupattr: member
> memberofattr: memberOf
> nsslapd-pluginId: memberof
> nsslapd-pluginVersion: 1.2.8.3
> nsslapd-pluginVendor: 389 Project
> nsslapd-pluginDescription: memberof plugin
> 
> Did the rest steps from console, Server did not crash.
> 
> But few observations here :
> 1. On DS console, I have one suffix dc=test,dc=redhat,dc=com..then under
> dc=test.. I have two OUs..
> 1. ou=Users
> having users abc, xyz ... etc.
> 2. ou=Groups
> having groups like admin, domain, unix ..etc
> 3. When I have added users to the groups from console.. using group's
> properties wizard.. members tab...
> 4. Saw that all the users disappeared from the ou=users.. then they came after
> I did a refresh.. same thing happened with group also.. when I clicked on
> ou=groups.. all groups disappeared..after a refresh they came... bizarre..?!?
> 
> ......
> 
> then I have noticed that.. there is no field on the user's properties (DS
> Console)to tell to which group this user belongs to.. 
> I could not see that under advanced properties too?
> ..............
> 
> Certainly, these are not related to the original issue but if these are valid
> to open a new bug ?? Please advice

Yes.  Please open a new bug for the console issues.

Comment 28 Noriko Hosoi 2011-05-17 17:12:41 UTC
Hmmm, is it a console problem?

I thought Amita hit this bug... (see the second paragraph.)

> Subject: [PATCH] Bug 697961 - memberOf needs to be triggered by internal 
> operations

> The memberOf plug-in is not currently triggered by internal modify,
> add, delete, or rename operations.  This patch makes it work with
> internal operations so it can play nicely with other plug-ins that
> make membership changes, such as the new Auto Membership plug-in.

> The precedence for the referential integrity plug-in needed to be
> lowered to ensure consistency of membership attributes when both
> memberOf and referential integrity plug-ins are in use.

Could you lower the priority of the referential integrity plugin's precedence
as the Nathan's patch does?  Then, run the test?
Steps:
1. shutdown the server
2. edit dse.ldif as follows
 nsslapd-pluginpath: libreferint-plugin
 nsslapd-plugininitfunc: referint_postop_init
 nsslapd-plugintype: postoperation
 nsslapd-pluginenabled: off
+nsslapd-pluginprecedence: 40
 nsslapd-pluginArg0: 0
 nsslapd-pluginArg1: %log_dir%/referint
 nsslapd-pluginArg2: 0
3. start the server
4. run the test

Comment 29 Amita Sharma 2011-05-18 10:52:10 UTC
> 
> Yes.  Please open a new bug for the console issues.

Thanks Rich, I will do that.

Comment 30 Amita Sharma 2011-05-18 10:53:32 UTC
No Noriko, I did not hit the bug..
They are the console issues, I faced. :)


Note You need to log in before you can comment on or make changes to this bug.