Created attachment 1136082 [details]
abrt email send_ldap_result_ext

Sumit, could this be an issue in sssd_krb5_locator_plugin.so?

The messages about sssd_krb5_locator_plugin.so in the output above are caused by a not matching sssd-debuginfo package. The version of the OpenLDAP debuginfo package looks wrong as well.

If a matching version of sssd-debuginfo is installed I would expect that the messages related to ssd_krb5_locator_plugin.so will go way. Abhijeet, can you try to reproduce with matching debuginfo packages? The issue will not go away but it might be easier to spot with matching versions.

the crash is in DS, it is dereferencing an operation structure, which is NULL.
This should not happen and should be hardened,

 but the event is probably triggered by schema compat, the event is inside the  backend_shr_data_initialize_thread_cb() thread and an internal operation calls send_ldap_result with err=53 and the msg text "This plugin is not configured to access operation target data" which indicates that there is no operation.

Slapi-nis populates its tree (backend_shr_data_initialize_thread_c) in a delay after the server startup. It is called with plugin_state containing 'plugin_base'. But when it is called the plugin_state has empty plugin_base (backend_update_params). So slapi_search_internal_set_pb just return without setting the approriate fields and it crashes later during the search.

(gdb) where
#0  0x00007fd21487ba31 in send_ldap_result_ext (pb=pb@entry=0x7fd1fc000a40, err=err@entry=53, matched=matched@entry=0x0, 
    text=text@entry=0x7fd2148a20a8 "This plugin is not configured to access operation target data", nentries=nentries@entry=0, 
    urls=urls@entry=0x0, ber=ber@entry=0x0) at ldap/servers/slapd/result.c:350
#1  0x00007fd21487c2e1 in send_ldap_result (pb=pb@entry=0x7fd1fc000a40, err=err@entry=53, matched=matched@entry=0x0, 
    text=text@entry=0x7fd2148a20a8 "This plugin is not configured to access operation target data", nentries=nentries@entry=0, 
    urls=urls@entry=0x0) at ldap/servers/slapd/result.c:193
#2  0x00007fd21486cd40 in slapi_search_internal_callback_pb (pb=pb@entry=0x7fd1fc000a40, 
    callback_data=callback_data@entry=0x7fd1f91a3d20, prc=prc@entry=0x0, 
    psec=psec@entry=0x7fd205b29080 <wrap_search_internal_get_entry_cb>, prec=prec@entry=0x0)
    at ldap/servers/slapd/plugin_internal_op.c:559
#3  0x00007fd205b2932d in wrap_search_internal_get_entry (parent_pb=parent_pb@entry=0x7fd215b13de8, dn=0x7fd1fc000a10, 
    filter=filter@entry=0x0, attrs=attrs@entry=0x0, ret_entry=ret_entry@entry=0x7fd1f91a3d20, caller_id=0x7fd2159e2070)
    at wrap.c:300
#4  0x00007fd205b170fa in backend_update_params (pb=0x7fd215b13de8, state=state@entry=0x7fd2159e1ed0) at back-sch.c:1047
#5  0x00007fd205b1797e in backend_shr_data_initialize_thread_cb (arg=<optimized out>) at back-shr.c:697
#6  0x00007fd212a4a7bb in _pt_root () from /lib64/libnspr4.so
#7  0x00007fd2123ebdc5 in start_thread () from /lib64/libpthread.so.0
#8  0x00007fd2121191cd in clone () from /lib64/libc.so.6
(gdb) frame 4
#4  0x00007fd205b170fa in backend_update_params (pb=0x7fd215b13de8, state=state@entry=0x7fd2159e1ed0) at back-sch.c:1047
1047		wrap_search_internal_get_entry(pb, our_dn, NULL, NULL, &our_entry,
(gdb) print *stat
A syntax error in expression, near `'.
(gdb) print *state
$1 = {plugin_base = 0x0, plugin_identity = 0x7fd2159e2070, plugin_desc = 0x7fd205d34840 <plugin_description>, use_be_txns = 1, 
  ready_to_serve = 0, tid = 0x0, pmap_client_socket = 0, max_dgram_size = 0, max_value_size = 0, request_info = 0x0, 
  securenet_info = 0x0, n_listeners = 0, listener = {{fd = 0, port = 0, pf = 0, type = 0}, {fd = 0, port = 0, pf = 0, type = 0}, {
      fd = 0, port = 0, pf = 0, type = 0}, {fd = 0, port = 0, pf = 0, type = 0}}, pam_lock = 0x0, nss_context = 0x0, 
  use_entry_cache = 0, cached_entries = 0x0, cached_entries_lock = 0x0}


I do not know why state->plugin_state is empty.
for hardening, backend_update_params should check state->plugin_base before creating slapi_sdn.

I cloned the same job as Abhijeet specified and I'm yet to see any ns-slapd crash in it. Was it reproducible in any other run?

Ok, after a while I have produced the crash in beaker as well. I now have a fix that survives my tests. You may try https://copr.fedorainfracloud.org/coprs/abbra/slapi-nis-test/ for your tests.

Move to right component.

Created attachment 1147216 [details]
This patch make the slapi-nis shutdown prevent priming thread to start or if it already started wait for its completion

This is a possible fix still under review

The patch is available in attachment 1147216 [details], moving to POST.

The bug is fixed by the rebase of slapi-nis package which was done as the bug 1292148.

IPA server version: ipa-server-4.4.0-12.el7.x86_64
Slapi-nis: slapi-nis-0.56.0-4.el7.x86_64

Verified the bug on the basis of below observations:
1. Verified that IPA server is successfully upgraded to latest version.
2. No Slapd-nis crash message/mail is observed.
3. Verified the same for following upgrade paths:
- 7.2.6 > 7.3
- 7.1.z > 7.3

Thus on the basis of above observations marking status of bug to "VERIFIED".

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-2471.html