RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1461437 - crash in send_ldap_result
Summary: crash in send_ldap_result
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: 389-ds-base
Version: 7.4
Hardware: All
OS: Linux
urgent
urgent
Target Milestone: rc
: ---
Assignee: mreynolds
QA Contact: Viktor Ashirov
URL:
Whiteboard:
Depends On:
Blocks: 1489694
TreeView+ depends on / blocked
 
Reported: 2017-06-14 12:55 UTC by German Parente
Modified: 2020-12-14 08:51 UTC (History)
4 users (show)

Fixed In Version: 389-ds-base-1.3.7.5-4.el7
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1489694 (view as bug list)
Environment:
Last Closed: 2018-04-10 14:16:50 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github 389ds 389-ds-base issues 2350 0 None closed slapi_search_internal_callback_pb may SIGSEV if related pblock has not operation set 2020-12-19 03:34:11 UTC
Red Hat Product Errata RHBA-2018:0811 0 None None None 2018-04-10 14:17:45 UTC

Description German Parente 2017-06-14 12:55:29 UTC
Description of problem:

we have seen a server crash at customer site that when sending the result of an operation.

The full stacktrace is:

The whole backtrace:

#0  0x00007fa579a22711 in send_ldap_result_ext (pb=pb@entry=0x7fa528007550, err=err@entry=53, matched=matched@entry=0x0, 
    text=text@entry=0x7fa579a4c708 "This plugin is not configured to access operation target data", nentries=nentries@entry=0, urls=urls@entry=0x0, ber=ber@entry=0x0) at ldap/servers/slapd/result.c:353
#1  0x00007fa579a22fd1 in send_ldap_result (pb=pb@entry=0x7fa528007550, err=err@entry=53, matched=matched@entry=0x0, 
    text=text@entry=0x7fa579a4c708 "This plugin is not configured to access operation target data", nentries=nentries@entry=0, urls=urls@entry=0x0) at ldap/servers/slapd/result.c:194
#2  0x00007fa579a13260 in slapi_search_internal_callback_pb (pb=pb@entry=0x7fa528007550, callback_data=callback_data@entry=0x7fa53bfe5ea0, prc=prc@entry=0x0, 
    psec=psec@entry=0x7fa56a802df0 <backend_shr_note_entry_sdn_cb>, prec=prec@entry=0x0) at ldap/servers/slapd/plugin_internal_op.c:559
#3  0x00007fa56a804a61 in backend_shr_update_references_cb (group=0x7fa520007d50 "cn=compat,dc=mlbam,dc=net", set=0x7fa52000aa50 "cn=groups", flag=<optimized out>, backend_data=0x7fa52392daa0, 
    cbdata_ptr=<optimized out>) at back-shr.c:1539
#4  0x00007fa56a81203f in map_data_foreach_map (state=state@entry=0x7fa57b585a30, domain_name=domain_name@entry=0x0, fn=fn@entry=0x7fa56a803fd0 <backend_shr_update_references_cb>, 
    cbdata=cbdata@entry=0x7fa53bfe5f50) at map.c:347
#5  0x00007fa56a80279b in backend_shr_update_references (state=0x7fa57b585a30, pb=pb@entry=0x7fa53bfe6a90, e=<optimized out>, mods=mods@entry=0x0, modlist=modlist@entry=0x0) at back-shr.c:1825
#6  0x00007fa56a803a17 in backend_shr_add_cb (pb=pb@entry=0x7fa53bfe6a90) at back-shr.c:1953
#7  0x00007fa56a803b41 in backend_shr_add_cb (pb=0x7fa53bfe6a90) at back-shr.c:1883
#8  backend_shr_betxn_post_add_cb (pb=0x7fa53bfe6a90) at back-shr.c:1965
#9  0x00007fa579a0db18 in plugin_call_func (list=0x7fa57b58e910, operation=operation@entry=560, pb=pb@entry=0x7fa53bfe6a90, call_one=call_one@entry=0) at ldap/servers/slapd/plugin.c:2049
#10 0x00007fa579a0dda3 in plugin_call_list (pb=0x7fa53bfe6a90, operation=560, list=<optimized out>) at ldap/servers/slapd/plugin.c:1993
#11 plugin_call_plugins (pb=pb@entry=0x7fa53bfe6a90, whichfunction=whichfunction@entry=560) at ldap/servers/slapd/plugin.c:445
#12 0x00007fa56c7b9b94 in ldbm_back_add (pb=0x7fa53bfe6a90) at ldap/servers/slapd/back-ldbm/ldbm_add.c:1150
#13 0x00007fa5799b1ed8 in op_shared_add (pb=pb@entry=0x7fa53bfe6a90) at ldap/servers/slapd/add.c:699
#14 0x00007fa5799b31a0 in do_add (pb=pb@entry=0x7fa53bfe6a90) at ldap/servers/slapd/add.c:226
#15 0x00007fa579ee59c3 in connection_dispatch_operation (pb=0x7fa53bfe6a90, op=0x7fa57b7521b0, conn=0x7fa57bc05118) at ldap/servers/slapd/connection.c:612
#16 connection_threadmain () at ldap/servers/slapd/connection.c:1759
#17 0x00007fa577bdb9bb in _pt_root (arg=0x7fa57bbcf3d0) at ../../../nspr/pr/src/pthreads/ptthread.c:216
#18 0x00007fa57757bdc5 in start_thread (arg=0x7fa53bfe7700) at pthread_create.c:308
#19 0x00007fa5772aa76d in __lseek_nocancel () at ../sysdeps/unix/syscall-template.S:81
#20 0x0000000000000000 in ?? ()

The add operation provokes a compat plugin post operation.

When returning to server the operation is NULL:

#0  0x00007fa579a22711 in send_ldap_result_ext (pb=pb@entry=0x7fa528007550, err=err@entry=53, matched=matched@entry=0x0, 
    text=text@entry=0x7fa579a4c708 "This plugin is not configured to access operation target data", nentries=nentries@entry=0, urls=urls@entry=0x0, ber=ber@entry=0x0) at ldap/servers/slapd/result.c:353
353		if (operation->o_status == SLAPI_OP_STATUS_RESULT_SENT) {

But operation is NULL:

 print operation
$1 = (Slapi_Operation *) 0x0

As we are not checking if operation is null before accessing its filed o_status, the server crashes:

=============================================
in send_ldap_intermediate:

        slapi_pblock_get (pb, SLAPI_OPERATION, &operation);
        slapi_pblock_get (pb, SLAPI_CONNECTION, &connection);

        if (operation->o_status == SLAPI_OP_STATUS_RESULT_SENT) {
                return(rc); /* result already sent */
        }
=============================================


Version-Release number of selected component (if applicable):
      389-ds-base-1.3.5.10-21.el7_3.x86_64

How reproducible: rarely !

Expected results:

Operation should not be NULL but a check could prevent the crash.

Comment 2 thierry bordaz 2017-06-14 16:21:29 UTC
Upstream ticket https://pagure.io/389-ds-base/issue/49291

Comment 3 thierry bordaz 2017-06-15 08:07:38 UTC
upstream slapi-nis ticket https://pagure.io/slapi-nis/issue/16 (improve robustness of slapi-nis)

Comment 4 thierry bordaz 2017-06-16 09:24:18 UTC
upstream slapi-nis ticket https://pagure.io/slapi-nis/issue/17 (format_add_sdn_list was called with empty sdn)

Comment 5 thierry bordaz 2017-06-23 15:41:33 UTC
As a follow up of investigations of https://pagure.io/slapi-nis/issue/17 


    - Because base_sdn_list and base_sdn_list2 contains the same SDN with NULL DN
      we know the NULL DN was added by calling format_add_sdn_list.

    - The NULL DN SDN was the last SDN of base_sdn_list, so it was append to the list

    - format_add_sdn_list is call in several places
      backend_shr_update_references_cb
      backend_shr_note_entry_sdn_cb
      format_referred_r_entry_cb
      format_maybe_add_sdn_list
      format_deref_rx
      format_referred_r


    - In backend_shr_update_references_cb:
      backend_shr_update_references_cb was not called with an entry with NULL dn
      In fact, 'cn=groups' maps contains a 'ignore_subtree' array.
      In such condition the entry DN is tested and if it is NULL,
      backend_shr_update_references_cb exits before calling format_add_sdn_list.
(gdb) print *(struct backend_shr_set_data *) backend_data
$2 = {state = 0x7fa57b585a30, group = 0x7fa521a75df0 "cn=compat,<suffix>", set = 0x7fa5238e28a0 "cn=groups", 
  bases = 0x7fa5238ea0c0, entry_filter = 0x7fa520008220 "(objectclass=posixGroup)", rel_attrs = 0x7fa523eae650, 
  rel_attr_list = 0x7fa528007860 "cn,ipaanchoruuid,gidNumber,memberUid,ipauniqueid,member,uid", rel_attrs_list = 0x7fa523eae650, 
  ref_attrs = 0x0, inref_attrs = 0x0, ref_attr_list = 0x7fa510582480, inref_attr_list = 0x0, skip_uninteresting_updates = 1, 
(gdb) print *((struct backend_shr_set_data *) backend_data)->ignore_subtrees[0]
$5 = {flag = 7 '\a', udn = 0x0, dn = 0x7fa521a53330 "cn=tasks,cn=config", ndn = 0x7fa510f8b240 "cn=tasks,cn=config", ndn_len = 18}
(gdb) print *((struct backend_shr_set_data *) backend_data)->ignore_subtrees[1]
Cannot access memory at address 0x0
      
      So the NULL DN SDN was not added by backend_shr_update_references_cb


    -  In backend_shr_note_entry_sdn_cb
      backend_shr_note_entry_sdn_cb is a callback of a search function.
      A NULL dn entry does not exist. An internal search would not call backend_shr_note_entry_sdn_cb with an NULL DN entry

    - In format_referred_r_entry_cb
      Idem as above

    - In format_maybe_add_sdn_list
      It does not call format_add_sdn_list on NULL dn entry

    - format_deref_rx
      This is possible when looping over parents entries

    - format_referred_r
      This is possible 
            if backend_get_set_config can return NULL DN SDN
            if it is call with an entry containing NULL DN


    In conclusion:
        The most probable place to introduce NULL DN SDN in the list are
            format_deref_rx (while looping other parents)
            format_referred_r (backend_get_set_config / called with NULL DN entry)

Comment 10 thierry bordaz 2017-09-19 07:31:57 UTC
Hi Viktor,

Unfortunately I think the easiest way to reproduce is to use/configure slapi-nis.
We need to have a plugin that triggers an internal search with a base search that is SDN containing a NULL DN.

I do not know if using slapi-nis is an acceptable option to test that bug. If it is you can use the test case described in https://pagure.io/slapi-nis/issue/17 (having a group with members (member attribute) with one of the 'member' value being empty).

If it is not acceptable I will need to dig in our standard plugins to check if one of them is doing an internal search without checking the base search SDN.

Comment 12 Viktor Ashirov 2017-11-21 15:04:41 UTC
Builds tested:
389-ds-base-1.3.7.5-10.el7.x86_64
slapi-nis-0.56.0-7.el7.x86_64

[1] To verify I configured slapi-nis plugin: 
# ldapadd -D "cn=Directory Manager" -w Secret123
dn: cn=NIS Server,cn=plugins,cn=config
objectClass: top
objectClass: nsSlapdPlugin
objectClass: extensibleObject
cn: NIS Server
nsslapd-pluginPath: /usr/lib64/dirsrv/plugins/nisserver-plugin.so
nsslapd-pluginInitfunc: nis_plugin_init
nsslapd-pluginType: postoperation
nsslapd-pluginEnabled: on
nsslapd-pluginDescription: NIS Server Plugin
nsslapd-pluginVendor: redhat.com
nsslapd-pluginVersion: 0.56 (betxn support available and enabled by default)
nsslapd-pluginId: nis-plugin
nis-tcp-wrappers-name: ypserv
nsslapd-pluginarg0: 541

dn: nis-domain=example.com+nis-map=groups,cn=NIS Server,cn=plugins,cn=config
objectClass: extensibleObject
objectClass: top
nis-domain: example.com
nis-map: groups
nis-base: ou=Groups, dc=example, dc=com
nis-filter: (objectClass=groupOfNames)
nis-key-format: %{cn}
nis-value-format: %merge(" ","%deref_f(\"member\",\"(objectclass=ipanisNetgroup)\",\"cn\")","(%link(\"%ifeq(\\\"hostCategory\\\",\\\"all\\\",\\\"\\\",\\\"%collect(\\\\\\\"%{externalHost}\\\\\\\",\\\\\\\"%deref(\\\\\\\\\\\\\\\"memberHost\\\\\\\\\\\\\\\",\\\\\\\\\\\\\\\"fqdn\\\\\\\\\\\\\\\")\\\\\\\",\\\\\\\"%deref_r(\\\\\\\\\\\\\\\"member\\\\\\\\\\\\\\\",\\\\\\\\\\\\\\\"fqdn\\\\\\\\\\\\\\\")\\\\\\\",\\\\\\\"%deref_r(\\\\\\\\\\\\\\\"memberHost\\\\\\\\\\\\\\\",\\\\\\\\\\\\\\\"member\\\\\\\\\\\\\\\",\\\\\\\\\\\\\\\"fqdn\\\\\\\\\\\\\\\")\\\\\\\")\\\")\",\"%ifeq(\\\"hostCategory\\\",\\\"all\\\",\\\"\\\",\\\"-\\\")\",\",\",\"%ifeq(\\\"userCategory\\\",\\\"all\\\",\\\"\\\",\\\"%collect(\\\\\\\"%deref(\\\\\\\\\\\\\\\"memberUser\\\\\\\\\\\\\\\",\\\\\\\\\\\\\\\"uid\\\\\\\\\\\\\\\")\\\\\\\",\\\\\\\"%deref_r(\\\\\\\\\\\\\\\"member\\\\\\\\\\\\\\\",\\\\\\\\\\\\\\\"uid\\\\\\\\\\\\\\\")\\\\\\\",\\\\\\\"%deref_r(\\\\\\\\\\\\\\\"memberUser\\\\\\\\\\\\\\\",\\\\\\\\\\\\\\\"member\\\\\\\\\\\\\\\",\\\\\\\\\\\\\\\"uid\\\\\\\\\\\\\\\")\\\\\\\")\\\")\",\"%ifeq(\\\"userCategory\\\",\\\"all\\\",\\\"\\\",\\\"-\\\")\"),%{nisDomainName:-})")

adding new entry "cn=NIS Server,cn=plugins,cn=config"

adding new entry "nis-domain=example.com+nis-map=groups,cn=NIS Server,cn=plugins,cn=config"

[2] Added test entries:

# ldapadd -D "cn=Directory Manager" -w Secret123
dn: cn=tuser,ou=People,dc=example,dc=com
objectClass: inetOrgPerson
objectClass: posixAccount
objectClass: organizationalPerson
objectClass: top
objectClass: person
cn: tuser
sn: tuser
uid: tuser
gidNumber: 2000
homeDirectory: /home/tuser
uidNumber: 2000

dn: cn=tgroup,ou=Groups,dc=example,dc=com
objectClass: groupofnames
objectClass: top
cn: tgroup
member:
member: cn=tuser,ou=People,dc=example,dc=com

adding new entry "cn=tuser,ou=People,dc=example,dc=com"

adding new entry "cn=tgroup,ou=Groups,dc=example,dc=com"


[3] Removed empty member attribute without server crashing:

# ldapmodify -D "cn=Directory Manager" -w Secret123
dn: cn=tgroup,ou=Groups,dc=example,dc=com
changetype: modify
replace: member
member: cn=tuser,ou=People,dc=example,dc=com

modifying entry "cn=tgroup,ou=Groups,dc=example,dc=com"

Marking as VERIFIED.

Comment 15 errata-xmlrpc 2018-04-10 14:16:50 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:0811


Note You need to log in before you can comment on or make changes to this bug.