Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1633111

Summary: Rename user (MODRDN) failing with err=1 leaves DB in bad state.
Product: Red Hat Enterprise Linux 7 Reporter: German Parente <gparente>
Component: 389-ds-baseAssignee: mreynolds
Status: CLOSED DUPLICATE QA Contact: RHDS QE <ds-qe-bugs>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.7-AltCC: gparente, lkrispen, mhonek, nkinder, pvoborni, rcritten, rmeggins, spichugi, tbordaz, tscherf, vashirov
Target Milestone: rcKeywords: Reopened
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-12-26 15:59:01 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description German Parente 2018-09-26 08:38:26 UTC 
Description of problem:

I am reporting this bug to the ipa component since compat-plugin component is related. And I have not managed to reproduce this issue in pure 389-ds-base. But if for a certain reason, the postop of the MODRDN fails, we could finish in a similar state.

Version-Release number of selected component (if applicable): 
389-ds-base-1.3.7.5-25.el7_5.x86_64


How reproducible: rename user when we have a very large number of users. This does not guarantee the failure but several customers manages to reproduce this already.

If we take a pstack during the MORDN operation, we can see, for instance these threads. The most important one is the one where the MODRDN is taking place, that is provoking a postop of referential integrity plugin and, in turn, a re-creation of maps in compat-plugin which is taking long time since we do a search by each user:

Thread 29 (Thread 0x7f257c2bf700 (LWP 12884)):
#0  0x00007f25cc6a3d00 in tc_malloc () at /lib64/libtcmalloc.so.4
#1  0x00007f25cd851ee3 in slapi_ch_malloc () at /usr/lib64/dirsrv/libslapd.so.0
#2  0x00007f25cd854f32 in csnset_add_csn () at /usr/lib64/dirsrv/libslapd.so.0
#3  0x00007f25cd85541f in csnset_dup () at /usr/lib64/dirsrv/libslapd.so.0
#4  0x00007f25cd8dc60e in slapi_value_dup () at /usr/lib64/dirsrv/libslapd.so.0
#5  0x00007f25cd8de9c5 in valueset_set_valueset () at /usr/lib64/dirsrv/libslapd.so.0
#6  0x00007f25cd84af57 in slapi_attr_dup () at /usr/lib64/dirsrv/libslapd.so.0
#7  0x00007f25cd861538 in slapi_entry_dup () at /usr/lib64/dirsrv/libslapd.so.0
#8  0x00007f25bd3395c1 in wrap_search_internal_get_entry_cb () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#9  0x00007f25cd8ba664 in send_ldap_search_entry_ext () at /usr/lib64/dirsrv/libslapd.so.0
#10 0x00007f25cd8bae5c in send_ldap_search_entry () at /usr/lib64/dirsrv/libslapd.so.0
#11 0x00007f25cd895cf4 in send_entry.isra.0 () at /usr/lib64/dirsrv/libslapd.so.0
#12 0x00007f25cd896258 in send_results_ext.constprop.4 () at /usr/lib64/dirsrv/libslapd.so.0
#13 0x00007f25cd898121 in op_shared_search () at /usr/lib64/dirsrv/libslapd.so.0
#14 0x00007f25cd8a9557 in search_internal_callback_pb () at /usr/lib64/dirsrv/libslapd.so.0
#15 0x00007f25bd3398dd in wrap_search_internal_get_entry () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#16 0x00007f25bd33051a in format_deref_rx.isra.6 () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#17 0x00007f25bd330ce9 in format_deref_r () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#18 0x00007f25bd32ec89 in format_expand () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#19 0x00007f25bd32fdd6 in format_format () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#20 0x00007f25bd3319e9 in format_get_data_set () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#21 0x00007f25bd3242c9 in backend_set_entry_from () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#22 0x00007f25bd32833d in backend_shr_set_config_entry_set_one_dn.isra.0 () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#23 0x00007f25bd32a926 in backend_shr_update_references_cb () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#24 0x00007f25bd33766f in map_data_foreach_map () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#25 0x00007f25bd327beb in backend_shr_update_references () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#26 0x00007f25bd32b1a1 in backend_shr_modify_cb.part.19 () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#27 0x00007f25bd32b581 in backend_shr_betxn_post_modify_cb () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#28 0x00007f25cd8a4548 in plugin_call_func () at /usr/lib64/dirsrv/libslapd.so.0
#29 0x00007f25cd8a4803 in plugin_call_plugins () at /usr/lib64/dirsrv/libslapd.so.0
#30 0x00007f25bf300a15 in ldbm_back_modify () at /usr/lib64/dirsrv/plugins/libback-ldbm.so
#31 0x00007f25cd890b36 in op_shared_modify () at /usr/lib64/dirsrv/libslapd.so.0
#32 0x00007f25cd8916ba in modify_internal_pb () at /usr/lib64/dirsrv/libslapd.so.0
#33 0x00007f25bdb62fd1 in _do_modify () at /usr/lib64/dirsrv/plugins/libreferint-plugin.so
#34 0x00007f25bdb650ff in update_integrity () at /usr/lib64/dirsrv/plugins/libreferint-plugin.so
#35 0x00007f25bdb66283 in referint_postop_modrdn () at /usr/lib64/dirsrv/plugins/libreferint-plugin.so
#36 0x00007f25cd8a4548 in plugin_call_func () at /usr/lib64/dirsrv/libslapd.so.0
#37 0x00007f25cd8a4803 in plugin_call_plugins () at /usr/lib64/dirsrv/libslapd.so.0
#38 0x00007f25bf3033ca in ldbm_back_modrdn () at /usr/lib64/dirsrv/plugins/libback-ldbm.so
#39 0x00007f25cd892f18 in op_shared_rename.constprop.0 () at /usr/lib64/dirsrv/libslapd.so.0
#40 0x00007f25cd8935cb in do_modrdn () at /usr/lib64/dirsrv/libslapd.so.0
#41 0x0000558e1a381304 in connection_threadmain ()
#42 0x00007f25cb662bab in _pt_root () at /lib64/libnspr4.so
#43 0x00007f25cb002dd5 in start_thread () at /lib64/libpthread.so.0
#44 0x00007f25ca6afb3d in clone () at /lib64/libc.so.6

This is locking other threads doing search operations and trying to access compat plugin:

Thread 26 (Thread 0x7f257aabc700 (LWP 12887)):
#0  0x00007f25cb006094 in pthread_rwlock_rdlock () at /lib64/libpthread.so.0
#1  0x00007f25bd325275 in backend_search_cb () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#2  0x00007f25cd8a4548 in plugin_call_func () at /usr/lib64/dirsrv/libslapd.so.0
#3  0x00007f25cd8a4803 in plugin_call_plugins () at /usr/lib64/dirsrv/libslapd.so.0
#4  0x00007f25cd897667 in op_shared_search () at /usr/lib64/dirsrv/libslapd.so.0
#5  0x0000558e1a39373e in do_search ()
#6  0x0000558e1a3815fa in connection_threadmain ()
#7  0x00007f25cb662bab in _pt_root () at /lib64/libnspr4.so
#8  0x00007f25cb002dd5 in start_thread () at /lib64/libpthread.so.0
#9  0x00007f25ca6afb3d in clone () at /lib64/libc.so.6

Thread 22 (Thread 0x7f2578ab8700 (LWP 12891)):
#0  0x00007f25cb006094 in pthread_rwlock_rdlock () at /lib64/libpthread.so.0
#1  0x00007f25bd325275 in backend_search_cb () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#2  0x00007f25cd8a4548 in plugin_call_func () at /usr/lib64/dirsrv/libslapd.so.0
#3  0x00007f25cd8a4803 in plugin_call_plugins () at /usr/lib64/dirsrv/libslapd.so.0
#4  0x00007f25cd897667 in op_shared_search () at /usr/lib64/dirsrv/libslapd.so.0
#5  0x0000558e1a39373e in do_search ()
#6  0x0000558e1a3815fa in connection_threadmain ()
#7  0x00007f25cb662bab in _pt_root () at /lib64/libnspr4.so
#8  0x00007f25cb002dd5 in start_thread () at /lib64/libpthread.so.0
#9  0x00007f25ca6afb3d in clone () at /lib64/libc.so.6

Thread 19 (Thread 0x7f25772b5700 (LWP 12894)):
#0  0x00007f25cb006094 in pthread_rwlock_rdlock () at /lib64/libpthread.so.0
#1  0x00007f25bd325275 in backend_search_cb () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#2  0x00007f25cd8a4548 in plugin_call_func () at /usr/lib64/dirsrv/libslapd.so.0
#3  0x00007f25cd8a4803 in plugin_call_plugins () at /usr/lib64/dirsrv/libslapd.so.0
#4  0x00007f25cd897667 in op_shared_search () at /usr/lib64/dirsrv/libslapd.so.0
#5  0x0000558e1a39373e in do_search ()
#6  0x0000558e1a3815fa in connection_threadmain ()
#7  0x00007f25cb662bab in _pt_root () at /lib64/libnspr4.so
#8  0x00007f25cb002dd5 in start_thread () at /lib64/libpthread.so.0
#9  0x00007f25ca6afb3d in clone () at /lib64/libc.so.6

Thread 17 (Thread 0x7f25762b3700 (LWP 12896)):
#0  0x00007f25cb006094 in pthread_rwlock_rdlock () at /lib64/libpthread.so.0
#1  0x00007f25bd325275 in backend_search_cb () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#2  0x00007f25cd8a4548 in plugin_call_func () at /usr/lib64/dirsrv/libslapd.so.0
#3  0x00007f25cd8a4803 in plugin_call_plugins () at /usr/lib64/dirsrv/libslapd.so.0
#4  0x00007f25cd897667 in op_shared_search () at /usr/lib64/dirsrv/libslapd.so.0
#5  0x0000558e1a39373e in do_search ()
#6  0x0000558e1a3815fa in connection_threadmain ()
#7  0x00007f25cb662bab in _pt_root () at /lib64/libnspr4.so
#8  0x00007f25cb002dd5 in start_thread () at /lib64/libpthread.so.0
#9  0x00007f25ca6afb3d in clone () at /lib64/libc.so.6

Thread 9 (Thread 0x7f25722ab700 (LWP 12904)):
#0  0x00007f25cb006094 in pthread_rwlock_rdlock () at /lib64/libpthread.so.0
#1  0x00007f25bd325275 in backend_search_cb () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#2  0x00007f25cd8a4548 in plugin_call_func () at /usr/lib64/dirsrv/libslapd.so.0
#3  0x00007f25cd8a4803 in plugin_call_plugins () at /usr/lib64/dirsrv/libslapd.so.0
#4  0x00007f25cd897667 in op_shared_search () at /usr/lib64/dirsrv/libslapd.so.0
#5  0x0000558e1a39373e in do_search ()
#6  0x0000558e1a3815fa in connection_threadmain ()
#7  0x00007f25cb662bab in _pt_root () at /lib64/libnspr4.so
#8  0x00007f25cb002dd5 in start_thread () at /lib64/libpthread.so.0
#9  0x00007f25ca6afb3d in clone () at /lib64/libc.so.6

and this operation is locked when reading an index (not related to schema compat):

Thread 30 (Thread 0x7f257cac0700 (LWP 12883)):
#0  0x00007f25cb006945 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libpthread.so.0
#1  0x00007f25c3b098b3 in __db_hybrid_mutex_suspend () at /lib64/libdb-5.3.so
#2  0x00007f25c3b08c00 in __db_tas_mutex_lock () at /lib64/libdb-5.3.so
#3  0x00007f25c3bb32fa in __lock_get_internal () at /lib64/libdb-5.3.so
#4  0x00007f25c3bb3de0 in __lock_get () at /lib64/libdb-5.3.so
#5  0x00007f25c3bdf762 in __db_lget () at /lib64/libdb-5.3.so
#6  0x00007f25c3b26bc5 in __bam_search () at /lib64/libdb-5.3.so
#7  0x00007f25c3b11816 in __bamc_search () at /lib64/libdb-5.3.so
#8  0x00007f25c3b137f3 in __bamc_get () at /lib64/libdb-5.3.so
#9  0x00007f25c3bcc286 in __dbc_iget () at /lib64/libdb-5.3.so
#10 0x00007f25c3bdb122 in __dbc_get_pp () at /lib64/libdb-5.3.so
#11 0x00007f25bf2f4924 in _entryrdn_get_elem () at /usr/lib64/dirsrv/plugins/libback-ldbm.so
#12 0x00007f25bf2f803a in _entryrdn_index_read () at /usr/lib64/dirsrv/plugins/libback-ldbm.so
#13 0x00007f25bf2f93c4 in entryrdn_index_read_ext () at /usr/lib64/dirsrv/plugins/libback-ldbm.so
#14 0x00007f25bf2c8a3c in dn2entry_ext () at /usr/lib64/dirsrv/plugins/libback-ldbm.so
#15 0x00007f25bf2cb86a in find_entry_internal.isra.0 () at /usr/lib64/dirsrv/plugins/libback-ldbm.so
#16 0x00007f25bf2cbcd3 in find_entry () at /usr/lib64/dirsrv/plugins/libback-ldbm.so
#17 0x00007f25bf306fa6 in ldbm_back_search () at /usr/lib64/dirsrv/plugins/libback-ldbm.so
#18 0x00007f25cd8979f2 in op_shared_search () at /usr/lib64/dirsrv/libslapd.so.0
#19 0x0000558e1a39373e in do_search ()
#20 0x0000558e1a3813e5 in connection_threadmain ()
#21 0x00007f25cb662bab in _pt_root () at /lib64/libnspr4.so
#22 0x00007f25cb002dd5 in start_thread () at /lib64/libpthread.so.0
#23 0x00007f25ca6afb3d in clone () at /lib64/libc.so.6

I will add more details privately in this bug.

The consequence is that operation fails with err=1 (operations error).

Then, database is sort of corrupted.

We cannot query the "renamed user" by it's DN:

ldapsearch -D "cn=directory manager" -W -b  "uid=myrenameduser,cn=users,cn=accounts,dc=example,dc=com"

err=32

But if we use filter by, for instance, uidnumber:

ldapsearch -LLLx -H ldap://server.example.com -b "dc=texample,dc=com" uidnumber=5000

we find the "renamed dn":

dn: uid=myrenameduser,cn=users,cn=accounts,dc=example,dc=com


How to repair this ?

export + re-import database. We will have the original user and not the renamed one.

But we need to clean references of the renamed user in, for instance "member" fields.

Then, either we could do a DEL + ADD, instead if acceptable.