Bug 1791342

Summary: ns-slapd crashes during paged result search test suite
Product: Red Hat Enterprise Linux 8 Reporter: Viktor Ashirov <vashirov>
Component: 389-ds-baseAssignee: mreynolds
Status: CLOSED DUPLICATE QA Contact: RHDS QE <ds-qe-bugs>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 8.2CC: lkrispen, spichugi, tbordaz, vashirov
Target Milestone: rcKeywords: Regression
Target Release: 8.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: 389-ds-1.4-8020020200120164339.bf00efc9 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-01-21 14:19:47 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Viktor Ashirov 2020-01-15 15:31:53 UTC 
Description of problem:
Gating tests reported failures in paged results test suite:
suites/paged_results/paged_results_test.py::test_search_multiple_paging PASSED [ 75%]
suites/paged_results/paged_results_test.py::test_search_invalid_cookie[1000] FAILED [ 75%]
suites/paged_results/paged_results_test.py::test_search_invalid_cookie[-1] FAILED [ 76%]
suites/paged_results/paged_results_test.py::test_search_abandon_with_zero_size FAILED [ 76%]
suites/paged_results/paged_results_test.py::test_search_pagedsizelimit_success FAILED [ 76%]
suites/paged_results/paged_results_test.py::test_search_nspagedsizelimit[5-15-PASS] FAILED [ 76%]
suites/paged_results/paged_results_test.py::test_search_nspagedsizelimit[15-5-SIZELIMIT_EXCEEDED] FAILED [ 76%]
suites/paged_results/paged_results_test.py::test_search_paged_limits[conf_attr_values0-ADMINLIMIT_EXCEEDED] FAILED [ 76%]
suites/paged_results/paged_results_test.py::test_search_paged_limits[conf_attr_values1-PASS] FAILED [ 76%]
suites/paged_results/paged_results_test.py::test_search_paged_user_limits[conf_attr_values0-ADMINLIMIT_EXCEEDED] FAILED [ 76%]
suites/paged_results/paged_results_test.py::test_search_paged_user_limits[conf_attr_values1-PASS] FAILED [ 76%]
suites/paged_results/paged_results_test.py::test_ger_basic FAILED        [ 76%]
suites/paged_results/paged_results_test.py::test_multi_suffix_search ERROR [ 76%]
suites/paged_results/paged_results_test.py::test_maxsimplepaged_per_conn_success[None] FAILED [ 76%]
suites/paged_results/paged_results_test.py::test_maxsimplepaged_per_conn_success[-1] FAILED [ 76%]
suites/paged_results/paged_results_test.py::test_maxsimplepaged_per_conn_success[1000] FAILED [ 77%]
suites/paged_results/paged_results_test.py::test_maxsimplepaged_per_conn_failure[0] FAILED [ 77%]
suites/paged_results/paged_results_test.py::test_maxsimplepaged_per_conn_failure[1] FAILED [ 77%]
suites/paged_results/paged_results_test.py::test_maxsimplepaged_per_conn_failure[1] ERROR [ 77%]

In the first failing test server crashes: 
Thread 1 (Thread 0x7f7b331f7700 (LWP 13513)):
#0  0x00007f7b7478bb8d in attr_syntax_normalize_no_lookup () at /usr/lib64/dirsrv/libslapd.so.0
#1  0x00007f7b747888a0 in slapi_attr_init_locking_optional () at /usr/lib64/dirsrv/libslapd.so.0
#2  0x00007f7b74788941 in slapi_attr_dup () at /usr/lib64/dirsrv/libslapd.so.0
#3  0x00007f7b747a28c0 in slapi_entry_dup () at /usr/lib64/dirsrv/libslapd.so.0
#4  0x00007f7b74785eca in op_shared_add () at /usr/lib64/dirsrv/libslapd.so.0
#5  0x00007f7b74786fc4 in do_add () at /usr/lib64/dirsrv/libslapd.so.0
#6  0x000056197791be89 in connection_dispatch_operation (pb=0x7f7b37a85520, op=0x7f7b2e444000, conn=0x7f7b3a8071c0) at ldap/servers/slapd/connection.c:623
#7  0x000056197791be89 in connection_threadmain () at ldap/servers/slapd/connection.c:1767
#8  0x00007f7b7211e568 in _pt_root () at /lib64/libnspr4.so
#9  0x00007f7b71ab92de in start_thread () at /lib64/libpthread.so.0
#10 0x00007f7b7124de83 in clone () at /lib64/libc.so.6


Version-Release number of selected component (if applicable):
389-ds-base-1.4.2.4-5.module+el8.2.0+5439+e9855ef3.x86_64


How reproducible:
always

Steps to Reproduce:
1. Run suites/paged_results/paged_results_test.py

Actual results:
ns-slapd crashes

Expected results:
ns-slapd should not crash

Additional info:
Comment 1 mreynolds 2020-01-15 21:41:15 UTC 
This appears to be a regression from https://bugzilla.redhat.com/show_bug.cgi?id=1790975 / Ticket 50709 (memory leak in "IP" Aci's) 

py.test ./paged_results_test.py::test_search_dns_ip_aci


ASAN output:

==1009==ERROR: AddressSanitizer: heap-use-after-free on address 0x60b000079f60 at pc 0x7fd3d80cdd7d bp 0x7fd39cef5b20 sp 0x7fd39cef5b10
READ of size 4 at 0x60b000079f60 thread T13
    #0 0x7fd3d80cdd7c in slapi_pblock_set (/usr/lib64/dirsrv/libslapd.so.0+0x1b1d7c)
    #1 0x7fd3c94ed3cd in DS_LASIpGetter ldap/servers/plugins/acl/acllas.c:308
    #2 0x7fd3c9254128 in ACL_GetAttribute (/usr/lib64/dirsrv/libns-dshttpd-1.4.2.4.so+0x46128)
    #3 0x7fd3c92513b8 in LASIpEval lib/libaccess/lasip.cpp:496
    #4 0x7fd3c92556d5 in ACLEvalAce(NSErr_s*, ACLEvalHandle*, ACLExprHandle*, unsigned long*, PListStruct_s**, PListStruct_s*) lib/libaccess/oneeval.cpp:215
    #5 0x7fd3c9257400 in ACL_INTEvalTestRights lib/libaccess/oneeval.cpp:752
    #6 0x7fd3c925852d in ACL_EvalTestRights (/usr/lib64/dirsrv/libns-dshttpd-1.4.2.4.so+0x4a52d)
    #7 0x7fd3c94c7e81 in acl__TestRights ldap/servers/plugins/acl/acl.c:3289
    #8 0x7fd3c94d14ce in acl_access_allowed (/usr/lib64/dirsrv/plugins/libacl-plugin.so+0x224ce)
    #9 0x7fd3c950498b in acl_access_allowed_main ldap/servers/plugins/acl/aclplugin.c:371
    #10 0x7fd3d80dfb0a in plugin_call_acl_plugin (/usr/lib64/dirsrv/libslapd.so.0+0x1c3b0a)
    #11 0x7fd3d804908e in test_filter_access ldap/servers/slapd/filterentry.c:956
    #12 0x7fd3d804908e in slapi_vattr_filter_test_ext_internal ldap/servers/slapd/filterentry.c:817
    #13 0x7fd3d804aeaa in slapi_vattr_filter_test_ext (/usr/lib64/dirsrv/libslapd.so.0+0x12eeaa)
    #14 0x7fd3c5b1bcb9 in ldbm_back_next_search_entry_ext ldap/servers/slapd/back-ldbm/ldbm_search.c:1713
    #15 0x7fd3d80aa4ff in iterate ldap/servers/slapd/opshared.c:1307
    #16 0x7fd3d80aa4ff in send_results_ext ldap/servers/slapd/opshared.c:1660
    #17 0x7fd3d80afc64 in op_shared_search (/usr/lib64/dirsrv/libslapd.so.0+0x193c64)
    #18 0x56159e8f726c in do_search ldap/servers/slapd/search.c:376
    #19 0x56159e8c7092 in connection_dispatch_operation ldap/servers/slapd/connection.c:662
    #20 0x56159e8c7092 in connection_threadmain ldap/servers/slapd/connection.c:1767
    #21 0x7fd3d58e4567  (/lib64/libnspr4.so+0x2b567)
    #22 0x7fd3d527f2dd in start_thread (/lib64/libpthread.so.0+0x82dd)
    #23 0x7fd3d4a13e82 in __GI___clone (/lib64/libc.so.6+0xfbe82)

0x60b000079f60 is located 0 bytes inside of 112-byte region [0x60b000079f60,0x60b000079fd0)
freed by thread T13 here:
    #0 0x7fd3d857b7b0 in __interceptor_free (/lib64/libasan.so.5+0xef7b0)
    #1 0x7fd3d7ff7a6c in slapi_ch_free (/usr/lib64/dirsrv/libslapd.so.0+0xdba6c)
    #2 0x7fd3d80c5830 in slapi_pblock_set (/usr/lib64/dirsrv/libslapd.so.0+0x1a9830)
    #3 0x7fd3c94ed3cd in DS_LASIpGetter ldap/servers/plugins/acl/acllas.c:308
    #4 0x7fd3c9254128 in ACL_GetAttribute (/usr/lib64/dirsrv/libns-dshttpd-1.4.2.4.so+0x46128)
    #5 0x7fd3c92513b8 in LASIpEval lib/libaccess/lasip.cpp:496
    #6 0x7fd3c92556d5 in ACLEvalAce(NSErr_s*, ACLEvalHandle*, ACLExprHandle*, unsigned long*, PListStruct_s**, PListStruct_s*) lib/libaccess/oneeval.cpp:215
    #7 0x7fd3c9257400 in ACL_INTEvalTestRights lib/libaccess/oneeval.cpp:752
    #8 0x7fd3c925852d in ACL_EvalTestRights (/usr/lib64/dirsrv/libns-dshttpd-1.4.2.4.so+0x4a52d)
    #9 0x7fd3c94c7e81 in acl__TestRights ldap/servers/plugins/acl/acl.c:3289
    #10 0x7fd3c94d14ce in acl_access_allowed (/usr/lib64/dirsrv/plugins/libacl-plugin.so+0x224ce)
    #11 0x7fd3c950498b in acl_access_allowed_main ldap/servers/plugins/acl/aclplugin.c:371
    #12 0x7fd3d80dfb0a in plugin_call_acl_plugin (/usr/lib64/dirsrv/libslapd.so.0+0x1c3b0a)
    #13 0x7fd3d804908e in test_filter_access ldap/servers/slapd/filterentry.c:956
    #14 0x7fd3d804908e in slapi_vattr_filter_test_ext_internal ldap/servers/slapd/filterentry.c:817
    #15 0x7fd3d804aeaa in slapi_vattr_filter_test_ext (/usr/lib64/dirsrv/libslapd.so.0+0x12eeaa)
    #16 0x7fd3c5b1bcb9 in ldbm_back_next_search_entry_ext ldap/servers/slapd/back-ldbm/ldbm_search.c:1713
    #17 0x7fd3d80aa4ff in iterate ldap/servers/slapd/opshared.c:1307
    #18 0x7fd3d80aa4ff in send_results_ext ldap/servers/slapd/opshared.c:1660
    #19 0x7fd3d80afc64 in op_shared_search (/usr/lib64/dirsrv/libslapd.so.0+0x193c64)
    #20 0x56159e8f726c in do_search ldap/servers/slapd/search.c:376
    #21 0x56159e8c7092 in connection_dispatch_operation ldap/servers/slapd/connection.c:662
    #22 0x56159e8c7092 in connection_threadmain ldap/servers/slapd/connection.c:1767
    #23 0x7fd3d58e4567  (/lib64/libnspr4.so+0x2b567)

previously allocated by thread T20 here:
    #0 0x7fd3d857bb78 in __interceptor_malloc (/lib64/libasan.so.5+0xefb78)
    #1 0x7fd3d7ff7277 in slapi_ch_malloc (/usr/lib64/dirsrv/libslapd.so.0+0xdb277)
    #2 0x7fd3c94ed499 in DS_LASIpGetter ldap/servers/plugins/acl/acllas.c:269
    #3 0x7fd3c9254128 in ACL_GetAttribute (/usr/lib64/dirsrv/libns-dshttpd-1.4.2.4.so+0x46128)
    #4 0x7fd3c92513b8 in LASIpEval lib/libaccess/lasip.cpp:496
    #5 0x7fd3c92556d5 in ACLEvalAce(NSErr_s*, ACLEvalHandle*, ACLExprHandle*, unsigned long*, PListStruct_s**, PListStruct_s*) lib/libaccess/oneeval.cpp:215
    #6 0x7fd3c9257400 in ACL_INTEvalTestRights lib/libaccess/oneeval.cpp:752
    #7 0x7fd3c925852d in ACL_EvalTestRights (/usr/lib64/dirsrv/libns-dshttpd-1.4.2.4.so+0x4a52d)
    #8 0x7fd3c94c7e81 in acl__TestRights ldap/servers/plugins/acl/acl.c:3289
    #9 0x7fd3c94d14ce in acl_access_allowed (/usr/lib64/dirsrv/plugins/libacl-plugin.so+0x224ce)
    #10 0x7fd3c950498b in acl_access_allowed_main ldap/servers/plugins/acl/aclplugin.c:371
    #11 0x7fd3d80dfb0a in plugin_call_acl_plugin (/usr/lib64/dirsrv/libslapd.so.0+0x1c3b0a)
    #12 0x7fd3d804908e in test_filter_access ldap/servers/slapd/filterentry.c:956
    #13 0x7fd3d804908e in slapi_vattr_filter_test_ext_internal ldap/servers/slapd/filterentry.c:817
    #14 0x7fd3d804aeaa in slapi_vattr_filter_test_ext (/usr/lib64/dirsrv/libslapd.so.0+0x12eeaa)
    #15 0x7fd3c5b1bcb9 in ldbm_back_next_search_entry_ext ldap/servers/slapd/back-ldbm/ldbm_search.c:1713
    #16 0x7fd3d80aa4ff in iterate ldap/servers/slapd/opshared.c:1307
    #17 0x7fd3d80aa4ff in send_results_ext ldap/servers/slapd/opshared.c:1660
    #18 0x7fd3d80b0191 in op_shared_search (/usr/lib64/dirsrv/libslapd.so.0+0x194191)
    #19 0x56159e8f726c in do_search ldap/servers/slapd/search.c:376
    #20 0x56159e8c7092 in connection_dispatch_operation ldap/servers/slapd/connection.c:662
    #21 0x56159e8c7092 in connection_threadmain ldap/servers/slapd/connection.c:1767
    #22 0x7fd3d58e4567  (/lib64/libnspr4.so+0x2b567)
Comment 2 mreynolds 2020-01-21 14:19:47 UTC 
This was introduced in https://bugzilla.redhat.com/show_bug.cgi?id=1790975

Closing this as a duplicate...

*** This bug has been marked as a duplicate of bug 1790975 ***