Bug 1518287

Summary: heap-use-after-free in csn_as
Product: Red Hat Enterprise Linux 7 Reporter: Viktor Ashirov <vashirov>
Component: 389-ds-baseAssignee: Ludwig <lkrispen>
Status: CLOSED ERRATA QA Contact: Viktor Ashirov <vashirov>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.5CC: lkrispen, lmiksik, mreynolds, nkinder, rmeggins
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: 389-ds-base-1.3.7.5-12 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-04-10 14:22:34 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Viktor Ashirov 2017-11-28 14:56:24 UTC
Description of problem:
=================================================================              
==25932== ERROR: AddressSanitizer: heap-use-after-free on address 0x600400d68df0 at pc 0x7f8ecdf7f062 bp 0x7f8e8f8e8100 sp 0x7f8e8f8e80f0
READ of size 8 at 0x600400d68df0 thread T45                                    
    #0 0x7f8ecdf7f061 in csn_as_string /usr/src/debug/389-ds-base-1.3.7.5/ldap/servers/slapd/csn.c:208
    #1 0x7f8ec1413f3c in csnpldata_free /usr/src/debug/389-ds-base-1.3.7.5/ldap/servers/plugins/replication/csnpl.c:428
    #2 0x7f8ec14145c9 in csnplInsert /usr/src/debug/389-ds-base-1.3.7.5/ldap/servers/plugins/replication/csnpl.c:142
    #3 0x7f8ec146dadc in ruv_add_csn_inprogress /usr/src/debug/389-ds-base-1.3.7.5/ldap/servers/plugins/replication/repl5_ruv.c:1540
    #4 0x7f8ec1441295 in replica_get_for_backend /usr/src/debug/389-ds-base-1.3.7.5/ldap/servers/plugins/replication/repl5_plugins.c:1398
    #5 0x7f8ec1443bb7 in multimaster_preop_modify /usr/src/debug/389-ds-base-1.3.7.5/ldap/servers/plugins/replication/repl5_plugins.c:343
    #6 0x7f8ece0381e3 in slapi_plugin_op_finished /usr/src/debug/389-ds-base-1.3.7.5/ldap/servers/slapd/plugin.c:2028 (discriminator 1)
    #7 0x7f8ece038658 in plugin_call_plugins /usr/src/debug/389-ds-base-1.3.7.5/ldap/servers/slapd/plugin.c:1972
    #8 0x7f8ece003718 in slapi_matchingrule_can_use_compare_fn /usr/src/debug/389-ds-base-1.3.7.5/ldap/servers/slapd/modify.c:993
    #9 0x7f8ece006776 in do_modify /usr/src/debug/389-ds-base-1.3.7.5/ldap/servers/slapd/modify.c:383
    #10 0x55a7ab7d3e1c in ?? /usr/src/debug/389-ds-base-1.3.7.5/ldap/servers/slapd/connection.c:624
    #11 0x7f8ecc11bc8a in PR_Select /usr/src/debug/nspr-4.17.0/pr/src/pthreads/../../../nspr/pr/src/pthreads/ptthread.c:216
    #12 0x7f8ece610867 in _ZN6__asan10AsanThread11ThreadStartEv _asan_rtl_     
    #13 0x7f8ecbabbdd4 in start_thread /usr/src/debug/glibc-2.17-c758a686/nptl/pthread_create.c:308
    #14 0x7f8ecb1699bc in __clone /usr/src/debug////////glibc-2.17-c758a686/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:113
0x600400d68df0 is located 0 bytes inside of 16-byte region [0x600400d68df0,0x600400d68e00)
freed by thread T45 here:              
    #0 0x7f8ece60cdd9 in __interceptor_free _asan_rtl_                         
    #1 0x7f8ecdf7b6c8 in slapi_ch_free /usr/src/debug/389-ds-base-1.3.7.5/ldap/servers/slapd/ch_malloc.c:270
    #2 0x7f8ec14155cf in csnplFreeCSNPL_CTX /usr/src/debug/389-ds-base-1.3.7.5/ldap/servers/plugins/replication/csnpl.c:442
    #3 0x7f8ecc1073f5 in PR_SetThreadPrivate /usr/src/debug/nspr-4.17.0/pr/src/threads/../../../nspr/pr/src/threads/prtpd.c:184
previously allocated by thread T45 here:                                       
    #0 0x7f8ece60cff5 in calloc _asan_rtl_                                     
    #1 0x7f8ecdf7b288 in slapi_ch_calloc /usr/src/debug/389-ds-base-1.3.7.5/ldap/servers/slapd/ch_malloc.c:180
    #2 0x7f8ecdf7ec93 in csn_dup /usr/src/debug/389-ds-base-1.3.7.5/ldap/servers/slapd/csn.c:118
    #3 0x7f8ec144014b in set_thread_primary_csn /usr/src/debug/389-ds-base-1.3.7.5/ldap/servers/plugins/replication/repl5_init.c:156
    #4 0x7f8ec146dc12 in ruv_add_csn_inprogress /usr/src/debug/389-ds-base-1.3.7.5/ldap/servers/plugins/replication/repl5_ruv.c:1532
    #5 0x7f8ec1441295 in replica_get_for_backend /usr/src/debug/389-ds-base-1.3.7.5/ldap/servers/plugins/replication/repl5_plugins.c:1398
    #6 0x7f8ec14438ef in multimaster_preop_delete /usr/src/debug/389-ds-base-1.3.7.5/ldap/servers/plugins/replication/repl5_plugins.c:269
    #7 0x7f8ece0381e3 in slapi_plugin_op_finished /usr/src/debug/389-ds-base-1.3.7.5/ldap/servers/slapd/plugin.c:2028 (discriminator 1)
    #8 0x7f8ece038658 in plugin_call_plugins /usr/src/debug/389-ds-base-1.3.7.5/ldap/servers/slapd/plugin.c:1972
    #9 0x7f8ecdf833f5 in op_shared_delete /usr/src/debug/389-ds-base-1.3.7.5/ldap/servers/slapd/delete.c:318
    #10 0x7f8ecdf83a1a in do_delete /usr/src/debug/389-ds-base-1.3.7.5/ldap/servers/slapd/delete.c:97
    #11 0x55a7ab7d3e38 in ?? /usr/src/debug/389-ds-base-1.3.7.5/ldap/servers/slapd/connection.c:614
    #12 0x7f8ecc11bc8a in PR_Select /usr/src/debug/nspr-4.17.0/pr/src/pthreads/../../../nspr/pr/src/pthreads/ptthread.c:216
Thread T45 created by T0 here:                             
    #0 0x7f8ece601a0a in __interceptor_pthread_create _asan_rtl_                                                       
    #1 0x7f8ecc11b95b in PR_Select /usr/src/debug/nspr-4.17.0/pr/src/pthreads/../../../nspr/pr/src/pthreads/ptthread.c:457
    #2 0x0                                                 
Shadow bytes around the buggy address:                     
  0x0c01001a5160: fa fa 00 00 fa fa 00 06 fa fa fa fa fa fa 00 01                                                      
  0x0c01001a5170: fa fa 07 fa fa fa fa fa fa fa 00 00 fa fa 03 fa                                                      
  0x0c01001a5180: fa fa 00 00 fa fa fa fa fa fa fa fa fa fa 00 00                                                      
  0x0c01001a5190: fa fa fa fa fa fa fa fa fa fa 00 00 fa fa 00 00                                                      
  0x0c01001a51a0: fa fa fa fa fa fa 00 07 fa fa fa fa fa fa fa fa                                                      
=>0x0c01001a51b0: fa fa 00 00 fa fa 00 00 fa fa 00 04 fa fa[fd]fd                                                      
  0x0c01001a51c0: fa fa fd fd fa fa fa fa fa fa fa fa fa fa fa fa                                                      
  0x0c01001a51d0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa                                                      
  0x0c01001a51e0: fa fa fa fa fa fa 00 00 fa fa fa fa fa fa fa fa                                                      
  0x0c01001a51f0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa                                                      
  0x0c01001a5200: fa fa fa fa fa fa fa fa fa fa 00 03 fa fa fa fa                                                      
Shadow byte legend (one shadow byte represents 8 application bytes):                                                   
  Addressable:           00                                
  Partially addressable: 01 02 03 04 05 06 07              
  Heap left redzone:     fa                                
  Heap righ redzone:     fb                                
  Freed Heap region:     fd                                
  Stack left redzone:    f1                                
  Stack mid redzone:     f2                                
  Stack right redzone:   f3                                
  Stack partial redzone: f4                                
  Stack after return:    f5                                
  Stack use after scope: f8                                
  Global redzone:        f9                                
  Global init order:     f6                                
  Poisoned by user:      f7                                
  ASan internal:         fe                                
==25932== ABORTING     

Version-Release number of selected component (if applicable):
389-ds-base-1.3.7.5-10.el7.x86_64

Comment 2 Ludwig 2017-11-29 09:45:28 UTC
Is it possible to reproduce this with repl logging enabled ?

which TET test is it ?

Comment 3 Viktor Ashirov 2017-11-29 10:13:31 UTC
This is autoMembers stress test suite (stress06).

I'll rerun tests with repl logging enabled.

Comment 4 Viktor Ashirov 2017-12-04 15:04:17 UTC
Errors before the crash is detected:
[04/Dec/2017:09:58:12.333604094 -0500] - ERR - NSMMReplicationPlugin - changelog program - _cl5WriteOperationTxn - retry (49) the transaction (csn=5a25627d001e
04cf0000) failed (rc=-30993 (BDB0068 DB_LOCK_DEADLOCK: Locker killed to resolve a deadlock))
[04/Dec/2017:09:58:12.373604505 -0500] - ERR - NSMMReplicationPlugin - changelog program - _cl5WriteOperationTxn - Failed to write entry with csn (5a25627d001e
04cf0000); db error - -30993 BDB0068 DB_LOCK_DEADLOCK: Locker killed to resolve a deadlock
[04/Dec/2017:09:58:12.415736767 -0500] - ERR - NSMMReplicationPlugin - write_changelog_and_ruv - Can't add a change for cn=Managers,cn=replsubGroups,dc=replAut
oMembers,dc=com (uniqid: 6161860a-d90311e7-ab37c49a-d2096d13, optype: 8) to changelog csn 5a25627d001e04cf0000

Comment 7 Viktor Ashirov 2018-01-30 12:54:38 UTC
Build tested: 
389-ds-base-1.3.7.5-14.el7.x86_64 (rebuilt with ASAN)

This crash didn't occur in acceptance tests, marking as VERIFIED, SanityOnly.

Comment 10 errata-xmlrpc 2018-04-10 14:22:34 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:0811