Bug 974719

Summary: rhds90 crash on tombstone modrdn
Product: Red Hat Enterprise Linux 6 Reporter: Marc Sauton <msauton>
Component: 389-ds-baseAssignee: Ludwig <lkrispen>
Status: CLOSED ERRATA QA Contact: Sankar Ramalingam <sramling>
Severity: medium Docs Contact:
Priority: high    
Version: 6.4CC: jgalipea, jrusnack, lkrispen, lnovich, nhosoi, nkinder
Target Milestone: rc   
Target Release: 6.5   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: 389-ds-base-1.2.11.15-22.el6 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-11-21 21:09:17 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Marc Sauton 2013-06-15 01:36:05 UTC
Description of problem:
potential security issue
crash RHDS consistently using 1 ldapmodrdn commandline on a tombstone entry


Version-Release number of selected component (if applicable):
389-ds-base-1.2.11.15-14.el6_4.x86_64
redhat-ds-base-9.0.0-0.17.el6dsrv.x86_64


How reproducible:
not yet in house, but customer can crash RHDS consistently using 1 ldapmodrdn commandline on a tombstone entry


Steps to Reproduce:

# ldapmodrdn -h localhost -p 389 -D "cn=admin" -W "nsuniqueid=69704201-cddc11e2-b5a9ba83-26a06d26,cn=00000000000000002162,ou=Dimitri,ou=Test,dc=cids" "cn=00000000000000002162" -s "ou=Dimitri,ou=Test,dc=cids"
Enter LDAP Password:
ldap_result: Can't contact LDAP server (-1)

(and believe me, the ldap server was running before executing this command). The result (always):

Jun  5 15:17:33 sucir0114 kernel: ns-slapd[22874]: segfault at 18 ip 00007f2d9934c43e sp 00007f2d769ed350 error 4 in libback-ldbm.so[7f2d9931e000+95000]

So the tombstone entry I'm trying to ressurect:
# ldapsearch -LLL -h localhost -p 389 -s sub -b "ou=Dimitri,ou=Test,dc=cids" -D "cn=admin" -W "(objectclass=nstombstone)"
Enter LDAP Password:
dn: nsuniqueid=cf10fe01-cddd11e2-b5a9ba83-26a06d26,cn=00000000000000002161,ou=
 Dimitri,ou=Test,dc=cids
description: test2
uid: 00000000000000002161
mail: s.user1.onkpn.com
sn: User1
physicalDeliveryOfficeName: NL_2700AA_3_ZOETERMEER
countryCode: 0
givenName: Scoot
cIDSDeleted: 0
cIDSSourceUID: aa5d8ea8-6607-45a2-bf8a-9e047d5f5dab
cIDSCredentials:: PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0iVVRGLTgiIHN0YW5kYWxv
 bmU9InllcyI/PjxTZXJ2aWNlQ3JlZGVudGlhbHMgeG1sbnM9Imh0dHA6Ly9ub3JkaWNlZGdlLnNlL
 2NyZWQiPjxTZXJ2aWNlIHNydmlkPSJncmlwUG9ydGFsIj48Q3JlZGVudGlhbCBlbmNhbGdvPSJBRV
 MvQ0JDL1BLQ1M1UGFkZGluZyI+PFVzZXJJRD4wMDAwMDAwMDAwMDAwMDAwMjE2MTwvVXNlcklEPjw
 vQ3JlZGVudGlhbD48L1NlcnZpY2U+PFNlcnZpY2Ugc3J2aWQ9ImdyaXBBZG1pblBvcnRhbCI+PENy
 ZWRlbnRpYWwgZW5jYWxnbz0iQUVTL0NCQy9QS0NTNVBhZGRpbmciPjxVc2VySUQ+MDAwMDAwMDAwM
 DAwMDAwMDIxNjE8L1VzZXJJRD48L0NyZWRlbnRpYWw+PC9TZXJ2aWNlPjxTZXJ2aWNlIHNydmlkPS
 JncmlwU2VydmljZU1hbmFnZW1lbnQiPjxDcmVkZW50aWFsIGVuY2FsZ289IkFFUy9DQkMvUEtDUzV
 QYWRkaW5nIj48VXNlcklEPnMudXNlcjFAVGVzdGNvbXBhbnkxLnRzdC5vbmtwbi5jb208L1VzZXJJ
 RD48L0NyZWRlbnRpYWw+PC9TZXJ2aWNlPjwvU2VydmljZUNyZWRlbnRpYWxzPg==
telephoneNumber: 0123456789
cIDSCompanyID: 111111
cIDSServiceManagementUID: s.user1.onkpn.com
distinguishedName: cn=00000000000000002161,OU=1,OU=Users,OU=111111,OU=Tenants,
 DC=CIDS
cn: 00000000000000002161
cIDSUserAlias: s.user1
cIDSPrimaryLoginID: testcompany1\s.user1
preferredLanguage: en
cIDSLoginDisabled: False
objectClass: inetOrgPerson
objectClass: cIDSUserObject
objectClass: extensibleObject
objectClass: organizationalPerson
objectClass: top
objectClass: person
objectClass: nsTombstone
displayName: Scoot User1
userPassword:: e1NTSEF9ejU1NzhwNFViRkpCUUswSk8wUUw3OTczOFM4WHRiQWZlTjZjenc9PQ=
 =
nsParentUniqueId: 45acfc01-cddc11e2-b5a9ba83-26a06d26

I have no idea how to ressurect a tombstoned entry manualy but want to know. It was suggested in the mentioned case to use ldapmodrdn.
I also tried removing the nsTombstone class in an earlier try from a different entry with the ldif:
dn: nsuniqueid=69704201-cddc11e2-b5a9ba83-26a06d26,cn=00000000000000002162,ou=
 Dimitri,ou=Test,dc=cids
changetype: modify
delete: objectclass
objectClass: nsTombstone

That succeeded but resulted in the entry to really disappear, but it's still there somewhere. Propably this is because the nsParentUniqueId is still set (and I couldn't delete that attribute in the ldif, I tried that as well)


1.
2.
3.

Actual results:


Expected results:


Additional info:


Core was generated by `/usr/sbin/ns-slapd -D /etc/dirsrv/slapd-sucir0114 -i /var/run/dirsrv/slapd-suci'.
Program terminated with signal 11, Segmentation fault.
#0  0x00007fa5c0f4943e in id2entry_add_ext (be=0xa8d900, e=0x0, txn=0x7fa59f9f5a00, encrypt=1, cache_res=0x0) at ldap/servers/slapd/back-ldbm/id2entry.c:78
78	    id_internal_to_stored(e->ep_id,temp_id);
#0  0x00007fa5c0f4943e in id2entry_add_ext (be=0xa8d900, e=0x0, txn=0x7fa59f9f5a00, encrypt=1, cache_res=0x0) at ldap/servers/slapd/back-ldbm/id2entry.c:78
        inst = 0xb3adc0
        db = 0xb87e90
        db_txn = 0x0
        data = {data = 0x7fa56800a360, size = 1, ulen = 0, dlen = 34336, doff = 0, app_data = 0x7fa5680189e0, flags = 3237084992}
        key = {data = 0x7fa5680176d0, size = 3390188309, ulen = 32677, dlen = 1744886976, doff = 32677, app_data = 0x7fa5ca122b5c, flags = 1744886976}
        len = <value optimized out>
        rc = <value optimized out>
        temp_id = "\000\260\361\300"
        encrypted_entry = 0x0
        entrydn = 0x0
#1  0x00007fa5c0f752df in modify_update_all (be=0xa8d900, pb=0xa92720, mc=0x7fa59f9f57c0, txn=0x7fa59f9f5a00) at ldap/servers/slapd/back-ldbm/ldbm_modify.c:159
        function_name = 0x7fa5c0fa17fd "modify_update_all"
        operation = 0xa929d0
        is_ruv = 0
        retval = 0
#2  0x00007fa5c0f7a454 in ldbm_back_modrdn (pb=<value optimized out>) at ldap/servers/slapd/back-ldbm/ldbm_modrdn.c:936
        be = 0xa8d900
        inst = 0xb3adc0
        li = 0xa48b60
        e = 0x7fa55c056ab0
        ec = 0x7fa56800a2f0
        ec_in_cache = 1
        e_in_cache = 0
        txn = {back_txn_txn = 0x7fa568017420}
        parent_txn = 0x0
        retval = 0
        msg = <value optimized out>
        postentry = 0x0
        errbuf = 0x0
        disk_full = 0
        retry_count = <value optimized out>
        ldap_result_code = 0
        ldap_result_message = 0x0
        ldap_result_matcheddn = 0x0
        parententry = 0x0
        newparententry = 0x7fa588003790
        original_entry = 0x7fa56800fb00
        original_parent = 0x0
        original_newparent = 0x0
        parent_modify_context = {new_entry_in_cache = 0, old_entry = 0x0, new_entry = 0x0, smods = 0x0, attr_encrypt = 1}
        newparent_modify_context = {new_entry_in_cache = 0, old_entry = 0x7fa588003790, new_entry = 0x7fa56800de00, smods = 0x7fa56800e260, attr_encrypt = 1}
---Type <return> to continue, or q <return> to quit---
        ruv_c = {new_entry_in_cache = 0, old_entry = 0xd5e290, new_entry = 0x7fa56800f4c0, smods = 0x7fa56800f0c0, attr_encrypt = 1}
        ruv_c_init = 1
        is_ruv = -1616947456
        children = 0x0
        child_entries = 0x0
        child_dns = 0x0
        sdn = 0x7fa5680015d0
        dn_newdn = {flag = 14 '\016', udn = 0x7fa568004050 "cn=", '0' <repeats 18 times>, "21,ou=Dimitri,ou=Test,dc=cids", 
          dn = 0x7fa56800df70 "cn=", '0' <repeats 18 times>, "21,ou=Dimitri,ou=Test,dc=cids", ndn = 0x7fa56800dff0 "cn=", '0' <repeats 18 times>, "21,ou=dimitri,ou=test,dc=cids", ndn_len = 50}
        dn_newrdn = {flag = 0 '\000', udn = 0x0, dn = 0x7fa568000e20 "cn=", '0' <repeats 18 times>, "21", ndn = 0x0, ndn_len = 23}
        dn_newsuperiordn = 0x7fa568001670
        dn_parentdn = {flag = 6 '\006', udn = 0x0, dn = 0x7fa5680018e0 "cn=", '0' <repeats 18 times>, "21,ou=Dimitri,ou=Test,dc=cids", 
          ndn = 0x7fa568009430 "cn=", '0' <repeats 18 times>, "21,ou=dimitri,ou=test,dc=cids", ndn_len = 50}
        orig_dn_newsuperiordn = 0x7fa5680173c0
        target_entry = 0x7fa5680025e0
        original_targetentry = 0x7fa568013900
        rc = 0
        isroot = 1
        mods = 0x7fa568001030
        smods_operation_wsi = {mods = 0x7fa568001030, num_elements = 3, num_mods = 2, iterator = 2, free_mods = 0}
        smods_generated = {mods = 0x7fa56800de90, num_elements = 5, num_mods = 1, iterator = 0, free_mods = 1}
        smods_generated_wsi = {mods = 0x7fa56800e0d0, num_elements = 5, num_mods = 0, iterator = 0, free_mods = 1}
        operation = 0xa929d0
        dblock_acquired = 1
        is_replicated_operation = 0
        is_fixup_operation = 0
        new_addr = {udn = 0x0, uniqueid = 0x0, sdn = 0x7fa59f9f5730}
        old_addr = 0xa92aa8
        oldparent_addr = {udn = 0xd5c130 "5093f1a8000000010000", uniqueid = 0x0, sdn = 0x7fa59f9f56d0}
        newsuperior_addr = 0xa92ae0
        original_newrdn = 0x7fa5680173a0 "cn=", '0' <repeats 18 times>, "21"
        opcsn = 0x7fa568003d20
        newdn = <value optimized out>
        newrdn = 0x7fa568000e20 "cn=", '0' <repeats 18 times>, "21"
        opreturn = 0
        free_modrdn_existing_entry = 1
#3  0x00007fa5ca0e9aca in op_shared_rename (pb=0xa92720, passin_args=0) at ldap/servers/slapd/modrdn.c:664
        rc = 1
        dn = 0x7fa568000fc0 "nsuniqueid=3bf03081-d2d511e2-b660ba83-26a06d26,cn=", '0' <repeats 18 times>, "21,ou=Dimitri,ou=Test,dc=cids"
        newrdn = 0x7fa568000e20 "cn=", '0' <repeats 18 times>, "21"
        newdn = 0x7fa568001820 "cn=", '0' <repeats 18 times>, "21,ou=Dimitri,ou=Test,dc=cids"
        newsuperior = <value optimized out>
        rdns = <value optimized out>
        deloldrdn = 0
        be = 0xa8d900
        origsdn = 0x7fa59f9f7c60
        smods = {mods = 0x0, num_elements = 0, num_mods = 0, iterator = 0, free_mods = 0}
        internal_op = 0
        repl_op = 0
        lastmod = 1
        operation = 0xa929d0
        referral = 0x0
        errorbuf = '\000' <repeats 6864 times>, "\006\000\000\000\000\000\000\000\001", '\000' <repeats 15 times>"\200, ", '\000' <repeats 31 times>, "\006\000\000\000\001\000\000\000[\000\000\000|\000\000\000w\000\000\000n\000\000\000p\016\000h\245\177\000\000\300\r\000h\245\177\000\000`\000\000\000\000\000\000\000 \000\000h\245\177\000\000\020\025\000h\245\177\000\000\360\r\000h\245\177\000\000\060\000\000\000\000\000\000\000\060:\273ǥ\177\000\000\200z\237\237\245\177\000\000\060\000\000\000\000\000\000\000 \000\000h\245\177\000\000\320\r\000h\245\177\000\000R\000\000\000\000\000\000\000\300\r\000h\245\177\000\---Type <return> to continue, or q <return> to quit---
000`\000\000\000\000\000\000\000\245<\273ǥ\177\000\000\003\000\000\000\000\000\000\000 \000\000h\245\177\000\000\003", '\000' <repeats 16 times>, "{\237\237\245\177\000\000\320"...
        err = <value optimized out>
        proxydn = 0x0
        proxystr = 0x0
        proxy_err = <value optimized out>
        errtext = 0x0
        sdn = 0x7fa5680015d0
        newsuperiorsdn = 0x7fa568001670
#4  0x00007fa5ca0ea38c in do_modrdn (pb=0xa92720) at ldap/servers/slapd/modrdn.c:268
        operation = 0xa929d0
        ber = 0xa93030
        rawdn = 0x7fa568000fc0 "nsuniqueid=3bf03081-d2d511e2-b660ba83-26a06d26,cn=", '0' <repeats 18 times>, "21,ou=Dimitri,ou=Test,dc=cids"
        rawnewsuperior = 0x7fa568000e40 "ou=Dimitri,ou=Test,dc=cids"
        dn = 0x7fa568000ec0 "nsuniqueid=3bf03081-d2d511e2-b660ba83-26a06d26,cn=", '0' <repeats 18 times>, "21,ou=Dimitri,ou=Test,dc=cids"
        newsuperior = 0x7fa568000f60 "ou=Dimitri,ou=Test,dc=cids"
        rawnewrdn = 0x0
        newrdn = 0x7fa568000e70 "cn=", '0' <repeats 18 times>, "21"
        err = <value optimized out>
        deloldrdn = 0
        len = 26
        newdn = <value optimized out>
        parent = 0x7fa568000d90 "cn=", '0' <repeats 18 times>, "21,ou=dimitri,ou=test,dc=cids"
        sdn = {flag = 14 '\016', udn = 0x7fa568000fc0 "nsuniqueid=3bf03081-d2d511e2-b660ba83-26a06d26,cn=", '0' <repeats 18 times>, "21,ou=Dimitri,ou=Test,dc=cids", 
          dn = 0x7fa568000ec0 "nsuniqueid=3bf03081-d2d511e2-b660ba83-26a06d26,cn=", '0' <repeats 18 times>, "21,ou=Dimitri,ou=Test,dc=cids", 
          ndn = 0x7fa568000d20 "nsuniqueid=3bf03081-d2d511e2-b660ba83-26a06d26,cn=", '0' <repeats 18 times>, "21,ou=dimitri,ou=test,dc=cids", ndn_len = 97}
        snewdn = {flag = 6 '\006', udn = 0x0, dn = 0x7fa568001510 "cn=", '0' <repeats 18 times>, "21,cn=", '0' <repeats 18 times>, "21,ou=dimitri,ou=test,dc=cids", 
          ndn = 0x7fa568001570 "cn=", '0' <repeats 18 times>, "21,cn=", '0' <repeats 18 times>, "21,ou=dimitri,ou=test,dc=cids", ndn_len = 74}
        snewsuperior = 0x7fa568000f30
#5  0x00000000004140e4 in connection_dispatch_operation () at ldap/servers/slapd/connection.c:588
        minssf = <value optimized out>
        minssf_exclude_rootdse = <value optimized out>
#6  connection_threadmain () at ldap/servers/slapd/connection.c:2338
        is_timedout = 0
        curtime = <value optimized out>
        pb = 0xa92720
        interval = 10000
        conn = 0x7fa5b80c3e10
        op = 0xa929d0
        tag = 108
        need_wakeup = <value optimized out>
        thread_turbo_flag = 0
        ret = <value optimized out>
        more_data = 0
        replication_connection = <value optimized out>
        doshutdown = 0
#7  0x00007fa5c852fa73 in _pt_root (arg=0xde1540) at ../../../mozilla/nsprpub/pr/src/pthreads/ptthread.c:199
        thred = 0xde1540
        detached = 1
#8  0x00007fa5c7ed2851 in start_thread (arg=0x7fa59f9f8700) at pthread_create.c:301
        __res = <value optimized out>
        pd = 0x7fa59f9f8700
        now = <value optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140349324363520, -1180840161032295930, 140734492886400, 140349324364224, 0, 3, 1213701544196155910, 1213544156527831558}, mask_was_saved = 0}}, priv = {
            pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <value optimized out>
---Type <return> to continue, or q <return> to quit---
        pagesize_m1 = <value optimized out>
        sp = <value optimized out>
        freesize = <value optimized out>
#9  0x00007fa5c7c2090d in nfsservctl () at ../sysdeps/unix/syscall-template.S:82
No locals.
#10 0x0000000000000000 in ?? ()
No symbol table info available.
(gdb)

Comment 5 Ludwig 2013-06-17 09:52:55 UTC
Reproduced with current master. The behaviour of modrdn for a tombstone entry is very inconsistent. Modrdn has two options: deleteoldrdn and newsuperior and the result is different for different combinations:

deleteoldrdn: 1 NO newsuperior: ==> err=53 (unwilling to perform)
deleteoldrdn: 1 newsuperior: <dn> ==> err=53 (unwilling to perform)
deleteoldrdn: 0 NO newsuperior ==> err=1 (operations error)
deleteoldrdn: 0 newsuperior: <dn> ==> CRASH

The crash has the side effect, that the entry is no longer accessable after restart, an attempt to repeat the operation gives err=32 (no such object)

What's next:
1] Define the expected behaviour, consistent to all combinations. In my opinion tombstone entries are internal and direct modifications should be prevented, so err=53 seems appropriate.
2] Implement 1]
3] Investigate if there is a proper way to resurrect tombstones by a client

Comment 6 Ludwig 2013-06-17 12:32:48 UTC
There is also a problem with modify operations with tombstone entries. It is possible to change an ordinary attribute eg uid. The change will get a csn, is written to the audit log and the index is updated although tombstone entries should have been removed from the ordinary attribute indexes. When the tombstone is purged an invalid index reference remains

Comment 7 Nathan Kinder 2013-06-18 15:39:32 UTC
Upstream ticket:
https://fedorahosted.org/389/ticket/47396

Comment 12 Ján Rusnačko 2013-08-05 08:13:09 UTC
Automated in mmrepl/accept

Comment 14 Ján Rusnačko 2013-08-23 12:31:48 UTC
Verified all 4 scenarios as per https://bugzilla.redhat.com/show_bug.cgi?id=974719#c5 on 389-ds-base-1.2.11.15-22.el6.x86_64.

Comment 15 errata-xmlrpc 2013-11-21 21:09:17 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-1653.html