Bug 735217 - simple paged search + ip/dns based ACI hangs server
Summary: simple paged search + ip/dns based ACI hangs server
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: 389-ds-base
Version: 6.2
Hardware: Unspecified
OS: Unspecified
urgent
high
Target Milestone: rc
: ---
Assignee: Rich Megginson
QA Contact: Chandrasekar Kannan
URL:
Whiteboard:
Depends On: 735121
Blocks: 434915 389_1.2.9 735231
TreeView+ depends on / blocked
 
Reported: 2011-09-01 20:08 UTC by Rich Megginson
Modified: 2015-01-04 23:50 UTC (History)
6 users (show)

Fixed In Version: 389-ds-base-1.2.9.9-1.el6
Doc Type: Bug Fix
Doc Text:
Clone Of: 735121
Environment:
Last Closed: 2011-12-06 17:56:24 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2011:1711 normal SHIPPED_LIVE 389-ds-base bug fix and enhancement update 2011-12-06 01:02:20 UTC

Description Rich Megginson 2011-09-01 20:08:02 UTC
+++ This bug was initially created as a clone of Bug #735121 +++

Doing a simple paged results search against a subtree that uses ip or dns based ACIs will hang the server.

For example:
dn: dc=example,dc=com
aci: (targetattr != "userPassword") (version 3.0;acl "Anonymous access within domain";allow (read,compare,search)(userdn = "ldap:///anyone") and (dns="localhost" or dns="localdomain" or dns="*.localdomain");)

If you repeatedly do simple paged result searches against dc=example,dc=com, the server will hang with a stack trace like this:

Thread 10 (Thread 0x7f44fa9f0700 (LWP 7422)):
#0  0x000000305280dfe4 in __lll_lock_wait () from /lib64/libpthread.so.0
#1  0x000000305280934e in _L_lock_995 () from /lib64/libpthread.so.0
#2  0x00000030528092b6 in pthread_mutex_lock () from /lib64/libpthread.so.0
#3  0x0000003061023bd9 in PR_Lock (lock=0x19890f0)
    at ../../../mozilla/nsprpub/pr/src/pthreads/ptsynch.c:206
#4  0x00007f4521f17112 in slapi_pblock_get (pblock=0x193ee50, arg=850, 
    value=0x7f44fa9dfdc0) at ../ds.git/ldap/servers/slapd/pblock.c:254
#5  0x00007f451eee241f in DS_LASDnsGetter (errp=0x0, subject=0x1822d08, 
    resource=0x0, auth_info=0x0, global_auth=0x0, arg=0x0)
    at ../ds.git/ldap/servers/plugins/acl/acllas.c:376
#6  0x00007f451ec9bf3e in ACL_GetAttribute (errp=0x0, 
    attr=0x7f451ecb1093 "dns", val=0x7f44fa9dff68, subject=0x1822d08, 
    resource=0x0, auth_info=0x0, global_auth=0x0)
    at ../ds.git/lib/libaccess/method.cpp:159
#7  0x00007f451ec9996f in LASDnsEval (errp=0x0, 
    attr_name=0x7f44e0001268 "dns", comparator=CMP_OP_EQ, 
    attr_pattern=0x7f44e00011d8 "localhost", cachable=0x7f44fa9e0020, 
    LAS_cookie=0x7f44e0002ec8, subject=0x1822d08, resource=0x0, auth_info=0x0, 
    global_auth=0x0) at ../ds.git/lib/libaccess/lasdns.cpp:404
#8  0x00007f451ec9c999 in ACLEvalAce (errp=0x0, acleval=0x1822f28, 
    ace=0x7f44e000db08, cachable=0x7f44fa9e2bc8, autharray=0x0, 
    global_auth=0x0) at ../ds.git/lib/libaccess/oneeval.cpp:254
#9  0x00007f451ec9d990 in ACL_INTEvalTestRights (errp=0x0, acleval=0x1822f28, 
    rights=0x7f44fa9e2d38, map_generic=0x7f451f0fb5c0, 
    deny_type=0x7f44fa9e2d60, deny_response=0x7f44fa9e2d58, 
    acl_tag=0x7f44fa9e2d50, expr_num=0x7f44fa9e2d80, cachable=0x7f44fa9e2cc8)
    at ../ds.git/lib/libaccess/oneeval.cpp:785
#10 0x00007f451ec9e012 in ACL_EvalTestRights (errp=0x0, acleval=0x1822f28, 
    rights=0x7f44fa9e2d30, map_generic=0x7f451f0fb5c0, 
    deny_type=0x7f44fa9e2d60, deny_response=0x7f44fa9e2d58, 
    acl_tag=0x7f44fa9e2d50, expr_num=0x7f44fa9e2d80)
    at ../ds.git/lib/libaccess/oneeval.cpp:995
#11 0x00007f451eed9653 in acl__TestRights (aclpb=0x184b3c0, access=2, 
    right=0x7f44fa9e2e60, map_generic=0x7f451f0fb5c0, 
    result_reason=0x7f44fa9e2e00)
    at ../ds.git/ldap/servers/plugins/acl/acl.c:3068
#12 0x00007f451eed5113 in acl_access_allowed (pb=0x193ee50, e=0x7f44e0000a50, 
    attr=0x7f4488007850 "objectClass", val=0x0, access=2)
    at ../ds.git/ldap/servers/plugins/acl/acl.c:590
#13 0x00007f451eef01dc in acl_access_allowed_main (pb=0x193ee50, 
    e=0x7f44e0000a50, attrs=0x7f44fa9e4fe0, val=0x0, access=2, flags=0, 
    errbuf=0x0) at ../ds.git/ldap/servers/plugins/acl/aclplugin.c:381
#14 0x00007f4521f2281e in plugin_call_acl_plugin (pb=0x193ee50, 
    e=0x7f44e0000a50, attrs=0x7f44fa9e4fe0, val=0x0, access=2, flags=0, 
    errbuf=0x0) at ../ds.git/ldap/servers/slapd/plugin_acl.c:90
#15 0x00007f4521eeabb3 in test_filter_access (pb=0x193ee50, e=0x7f44e0000a50, 
    attr_type=0x7f4488007850 "objectClass", attr_val=0x0)
    at ../ds.git/ldap/servers/slapd/filterentry.c:1011
#16 0x00007f4521eea793 in slapi_vattr_filter_test_ext_internal (pb=0x193ee50, 
    e=0x7f44e0000a50, f=0x7f4488007390, verify_access=1, only_check_access=0, 
    access_check_done=0x7f44fa9e50b8)
    at ../ds.git/ldap/servers/slapd/filterentry.c:926
#17 0x00007f4521eea38d in slapi_vattr_filter_test_ext (pb=0x193ee50, 
    e=0x7f44e0000a50, f=0x7f4488007390, verify_access=1, only_check_access=0)
    at ../ds.git/ldap/servers/slapd/filterentry.c:839
#18 0x00007f4521eea2b5 in slapi_vattr_filter_test (pb=0x193ee50, 
    e=0x7f44e0000a50, f=0x7f4488007390, verify_access=1)
    at ../ds.git/ldap/servers/slapd/filterentry.c:787
#19 0x00007f451cdd847a in ldbm_back_next_search_entry_ext (pb=0x193ee50, 
    use_extension=0)
    at ../ds.git/ldap/servers/slapd/back-ldbm/ldbm_search.c:1412
#20 0x00007f451cdd78dd in ldbm_back_next_search_entry (pb=0x193ee50)
    at ../ds.git/ldap/servers/slapd/back-ldbm/ldbm_search.c:1141
#21 0x00007f4521f15344 in iterate (pb=0x193ee50, be=0x1774ea0, send_result=1, 
    pnentries=0x7f44fa9eb358, pagesize=2, pr_statp=0x7f44fa9eb334)
    at ../ds.git/ldap/servers/slapd/opshared.c:1162
#22 0x00007f4521f15c2b in send_results_ext (pb=0x193ee50, send_result=1, 
    nentries=0x7f44fa9eb358, pagesize=2, pr_stat=0x7f44fa9eb334)
    at ../ds.git/ldap/servers/slapd/opshared.c:1545
#23 0x00007f4521f144d1 in op_shared_search (pb=0x193ee50, send_result=1)
    at ../ds.git/ldap/servers/slapd/opshared.c:599
#24 0x000000000042caf7 in do_search (pb=0x193ee50)
    at ../ds.git/ldap/servers/slapd/search.c:393
#25 0x0000000000413fde in connection_dispatch_operation (conn=0x7f45181f5560, 
    op=0x1990ef0, pb=0x193ee50)
    at ../ds.git/ldap/servers/slapd/connection.c:611
#26 0x00000000004158e4 in connection_threadmain ()
    at ../ds.git/ldap/servers/slapd/connection.c:2328
#27 0x0000003061029633 in _pt_root (arg=0x197c0f0)
    at ../../../mozilla/nsprpub/pr/src/pthreads/ptthread.c:187
#28 0x00000030528077e1 in start_thread () from /lib64/libpthread.so.0
#29 0x0000003051ce577d in clone () from /lib64/libc.so.6

Thread 1 (Thread 0x7f4521c577c0 (LWP 7366)):
#0  0x000000305280dfe4 in __lll_lock_wait () from /lib64/libpthread.so.0
#1  0x000000305280934e in _L_lock_995 () from /lib64/libpthread.so.0
#2  0x00000030528092b6 in pthread_mutex_lock () from /lib64/libpthread.so.0
#3  0x0000003061023bd9 in PR_Lock (lock=0x19890f0)
    at ../../../mozilla/nsprpub/pr/src/pthreads/ptsynch.c:206
#4  0x0000000000418eda in setup_pr_read_pds (ct=0x1996fb0, n_tcps=0x171d4d0, 
    s_tcps=0x0, i_unix=0x0, num_to_read=0x7fff551bb248)
    at ../ds.git/ldap/servers/slapd/daemon.c:1193
#5  0x0000000000417f7c in slapd_daemon (ports=0x7fff551bb390)
    at ../ds.git/ldap/servers/slapd/daemon.c:663
#6  0x000000000041fc23 in main (argc=7, argv=0x7fff551bb508)
    at ../ds.git/ldap/servers/slapd/main.c:1239

The connection mutex is acquired here:
opshared.c:1160
        if ( is_paged && pb->pb_conn && pb->pb_conn->c_mutex ) {
            PR_Lock( pb->pb_conn->c_mutex );
        }

--- Additional comment from rmeggins@redhat.com on 2011-09-01 14:37:02 EDT ---

Created attachment 521072 [details]
0001-Bug-735121-simple-paged-search-ip-dns-based-ACI-hang.patch

--- Additional comment from rmeggins@redhat.com on 2011-09-01 16:06:31 EDT ---

To ssh://git.fedorahosted.org/git/389/ds.git
   576d90b..b5f77c6  master -> master
commit b5f77c693d50ca3acf58eb7a41d9eded59328632
Author: Rich Megginson <rmeggins@redhat.com>
Date:   Thu Sep 1 10:30:06 2011 -0600
    Reviewed by: nhosoi (Thanks!)
    Branch: master
    Fix Description: The pb_conn->c_mutex lock around be->be_next_search_entry()
    was too big.  Other code inside be->be_next_search_entry needs to lock
    the conn.  In this particular case, the aci code needed to access the
    client ip address from the conn object.  The fix is to remove the mutex
    around be->be_next_search_entry() in iterate() and instead make sure
    code that accesses pb_conn locks first.
    Platforms tested: RHEL6 x86_64
    Flag Day: no
    Doc impact: no

Comment 4 Amita Sharma 2011-09-13 06:28:58 UTC
Cloned - https://bugzilla.redhat.com/show_bug.cgi?id=735121 is verified hence marking this as VERIFIED.

Comment 5 errata-xmlrpc 2011-12-06 17:56:24 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHEA-2011-1711.html


Note You need to log in before you can comment on or make changes to this bug.