Bug 755725 - 389 programs linked against openldap crash during shutdown
Summary: 389 programs linked against openldap crash during shutdown
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: 389-ds-base
Version: 15
Hardware: x86_64
OS: Linux
unspecified
medium
Target Milestone: ---
Assignee: Rich Megginson
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Keywords:
Depends On:
Blocks: 389_1.2.10 769107
TreeView+ depends on / blocked
 
Reported: 2011-11-21 21:18 UTC by Jr Aquino
Modified: 2012-03-31 03:26 UTC (History)
4 users (show)

(edit)
Clone Of:
(edit)
Last Closed: 2012-02-02 17:27:20 UTC


Attachments (Terms of Use)
0001-Bug-755725-389-programs-linked-against-openldap-cras.patch (6.97 KB, patch)
2011-12-15 15:13 UTC, Rich Megginson
nhosoi: review+
Details | Diff

Description Jr Aquino 2011-11-21 21:18:43 UTC
Description of problem:
When restarting the dirsrv process, it receives a segfault

Version-Release number of selected component (if applicable):
389-ds-base-1.2.10-0.5.a5.fc15.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Install FreeIPA
2. Restart DIRSRV
3. check dmesg for segfaults
  
Actual results:
restart generates segfault

Expected results:
Restart without crashing

Additional info:


(gdb) thread apply all bt all

Thread 1 (Thread 0x7f4a76a387e0 (LWP 918)):
No symbol "all" in current context.
(gdb) thread apply all bt full

Thread 1 (Thread 0x7f4a76a387e0 (LWP 918)):
#0  __GI___libc_free (mem=0x185) at malloc.c:3709
        ar_ptr = <optimized out>
        p = <optimized out>
        hook = 0
#1  0x000000352be0e510 in ldap_int_destroy_global_options () at ../../../libraries/libldap/init.c:496
        gopts = 0x352c049360
#2  0x000000352660e896 in _dl_fini () at dl-fini.c:244
        array = 0x352c0475c8
        i = <optimized out>
        nmaps = 94
        nloaded = <optimized out>
        i = <optimized out>
        l = 0x7f4a76a459b0
        ns = 0
        maps = 0x7fff3ccac680
        maps_size = 752
        do_audit = 0
#3  0x0000003526a38991 in __run_exit_handlers (status=0, listp=0x3526d935a8, run_list_atexit=true) at exit.c:78
        atfct = <optimized out>
        onfct = <optimized out>
        cxafct = <optimized out>
        f = <optimized out>
#4  0x0000003526a38a15 in __GI_exit (status=<optimized out>) at exit.c:100
No locals.
#5  0x0000003526a213a4 in __libc_start_main (main=0x40c8d0 <main>, argc=7, ubp_av=0x7fff3ccacb08, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fff3ccacaf8) at libc-start.c:258
        result = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {0, -6666163846686787399, 4255964, 140734213311232, 0, 0, 6666311345183307961, -6694800918663219015}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x7fff3ccacb48, 0x1}, data = {prev = 0x0, cleanup = 0x0, 
              canceltype = 1019923272}}}
        not_first_call = <optimized out>
#6  0x000000000040f105 in _start ()
No symbol table info available.

Comment 1 Jr Aquino 2011-11-21 21:52:17 UTC
Adjustment to Reproduce issue.  Standard install does NOT appear to generate the segfault.

(Corrected Method)
Steps to Reproduce:
1. Install FreeIPA
2. Install FreeIPA Replica Server
3. Restart FreeIPA Master
4. check dmesg for the segfault

Comment 2 Rich Megginson 2011-11-21 22:24:25 UTC
The crash is occurring here:
static void
ldap_int_destroy_global_options(void)
...
#ifdef HAVE_CYRUS_SASL
	if ( gopts->ldo_def_sasl_authcid ) {
		LDAP_FREE( gopts->ldo_def_sasl_authcid );
		gopts->ldo_def_sasl_authcid = NULL;
	}
#endif

Either gopts->ldo_def_sasl_authcid is corrupted, or someone else freed it and didn't set it to NULL.  I suspect that in order to trigger this bug you'd have to attempt a SASL/GSSAPI bind in replication or DNA.

Comment 3 Rich Megginson 2011-12-12 22:55:39 UTC
now happening in dsgw CGI programs as well:
Core was generated by `/usr/lib64/dirsrv/dsgw-cgi-bin/doauth'.
Program terminated with signal 11, Segmentation fault.
#0 0x0000003eaca80c14 in ?? ()
Missing separate debuginfos, use: debuginfo-install glibc-2.14.90-19.x86_64
(gdb) up
#1 0x0000003ebda0eb90 in ldap_int_destroy_global_options () at ../../../libraries/libldap/init.c:496
496 LDAP_FREE( gopts->ldo_def_sasl_authcid );
(gdb) p *gopts
$1 = {ldo_valid = 0, ldo_debug = 32768, ldo_peer = 0x0, ldo_cldapdn = 0x0, ldo_is_udp = 0, ldo_tm_api = {tv_sec = 0, tv_usec = 0},
ldo_tm_net = {tv_sec = 0, tv_usec = 0}, ldo_version = 0, ldo_deref = 0, ldo_timelimit = 0, ldo_sizelimit = 0, ldo_tls_ctx = 0x0,
ldo_tls_connect_cb = 0, ldo_tls_connect_arg = 0x0, ldo_tls_info = {lt_certfile = 0x0, lt_keyfile = 0x0, lt_dhfile = 0x0,
lt_cacertfile = 0x0, lt_cacertdir = 0x0, lt_ciphersuite = 0x0, lt_crlfile = 0x0, lt_randfile = 0x0, lt_protocol_min = 0},
ldo_tls_mode = 0, ldo_tls_require_cert = 0, ldo_tls_impl = 0, ldo_defludp = 0x0, ldo_defport = 0, ldo_defbase = 0x0,
ldo_defbinddn = 0x0, ldo_def_sasl_mech = 0x0, ldo_def_sasl_realm = 0x0, ldo_def_sasl_authcid = 0x0, ldo_def_sasl_authzid = 0x0,
ldo_sasl_secprops = {min_ssf = 0, max_ssf = 0, maxbufsize = 0, security_flags = 0, property_names = 0x0, property_values = 0x0},
ldo_keepalive_idle = 0, ldo_keepalive_probes = 0, ldo_keepalive_interval = 0, ldo_refhoplimit = 0, ldo_sctrls = 0x0,
ldo_cctrls = 0x0, ldo_rebind_proc = 0, ldo_rebind_params = 0x0, ldo_nextref_proc = 0, ldo_nextref_params = 0x0,
ldo_urllist_proc = 0, ldo_urllist_params = 0x0, ldo_conn_cbs = 0x0, ldo_booleans = 0}
(gdb)

Comment 4 Rich Megginson 2011-12-14 23:53:23 UTC
I've been able to reproduce this using doauth from dsgw on F-16.  It looks as though the function ldap_int_destroy_global_options() is called twice during shutdown.  The first time it works fine.  The second time it fails.  In the debugger, I can see the value of ldo_def_sasl_authcid == 0x0, but
	if ( gopts->ldo_def_sasl_authcid ) {
		LDAP_FREE( gopts->ldo_def_sasl_authcid );
		gopts->ldo_def_sasl_authcid = NULL;
	}
this goes into LDAP_FREE anyway?????  And inside LDAP_FREE, the value is 0x185 (which strangely is 389 in decimal).  valgrind reports no errors except for the attempt to free 0x185 (which of course is not malloced).

Comment 5 Rich Megginson 2011-12-15 15:13:58 UTC
Created attachment 547304 [details]
0001-Bug-755725-389-programs-linked-against-openldap-cras.patch

Comment 6 Rich Megginson 2011-12-15 18:27:15 UTC
commit 91fa21f6af7871d73bb9d5e738ec4e31b5298463
Author: Rich Megginson <rmeggins@redhat.com>
Date:   Wed Dec 14 20:04:48 2011 -0700
    Reviewed by: nhosoi (Thanks!)
    Branch: master
    Fix Description: With recent versions of openldap, you cannot link with both
    ldap_r and ldap - when the shared object _fini is run, the _fini from the on
    will stomp on the _fini from the other, and the program will crash.  The fix
    is to link with ldap_r only in a threaded program, and ldap otherwise.
    Platforms tested: Fedora 16, RHEL6 x86_64
    Flag Day: no

Comment 7 Fedora Update System 2011-12-15 19:55:11 UTC
389-dsgw-1.1.8-1.fc15 has been submitted as an update for Fedora 15.
https://admin.fedoraproject.org/updates/389-dsgw-1.1.8-1.fc15

Comment 8 Fedora Update System 2011-12-15 19:56:36 UTC
389-dsgw-1.1.8-1.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/389-dsgw-1.1.8-1.fc16

Comment 9 Fedora Update System 2011-12-19 16:51:08 UTC
389-ds-base-1.2.10-0.6.a6.fc15 has been submitted as an update for Fedora 15.
https://admin.fedoraproject.org/updates/389-ds-base-1.2.10-0.6.a6.fc15

Comment 10 Fedora Update System 2011-12-19 16:52:31 UTC
389-ds-base-1.2.10-0.6.a6.el5 has been submitted as an update for Fedora EPEL 5.
https://admin.fedoraproject.org/updates/389-ds-base-1.2.10-0.6.a6.el5

Comment 11 Fedora Update System 2011-12-19 16:53:25 UTC
389-ds-base-1.2.10-0.6.a6.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/389-ds-base-1.2.10-0.6.a6.fc16

Comment 12 Fedora Update System 2011-12-21 14:40:22 UTC
slapi-nis-0.28-1.fc16,freeipa-2.1.4-3.fc16,389-ds-base-1.2.10-0.6.a6.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/slapi-nis-0.28-1.fc16,freeipa-2.1.4-3.fc16,389-ds-base-1.2.10-0.6.a6.fc16

Comment 13 Fedora Update System 2011-12-22 14:28:47 UTC
389-ds-base-1.2.10-0.6.a6.fc15,slapi-nis-0.28-1.fc15,freeipa-2.1.4-2.fc15 has been submitted as an update for Fedora 15.
https://admin.fedoraproject.org/updates/389-ds-base-1.2.10-0.6.a6.fc15,slapi-nis-0.28-1.fc15,freeipa-2.1.4-2.fc15

Comment 14 Fedora Update System 2011-12-22 22:42:55 UTC
Package slapi-nis-0.28-1.fc16, freeipa-2.1.4-3.fc16, 389-ds-base-1.2.10-0.6.a6.fc16:
* should fix your issue,
* was pushed to the Fedora 16 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing slapi-nis-0.28-1.fc16 freeipa-2.1.4-3.fc16 389-ds-base-1.2.10-0.6.a6.fc16'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2011-17313/slapi-nis-0.28-1.fc16,freeipa-2.1.4-3.fc16,389-ds-base-1.2.10-0.6.a6.fc16
then log in and leave karma (feedback).

Comment 15 Rich Megginson 2012-01-10 20:36:59 UTC
Upstream ticket:
https://fedorahosted.org/389/ticket/252

Comment 16 Fedora Update System 2012-01-24 00:33:35 UTC
389-ds-base-1.2.10-0.8.a7.el5 has been submitted as an update for Fedora EPEL 5.
https://admin.fedoraproject.org/updates/389-ds-base-1.2.10-0.8.a7.el5

Comment 17 Fedora Update System 2012-01-24 21:26:30 UTC
389-ds-base-1.2.10-0.9.a8.el5 has been submitted as an update for Fedora EPEL 5.
https://admin.fedoraproject.org/updates/389-ds-base-1.2.10-0.9.a8.el5

Comment 18 Fedora Update System 2012-02-02 17:27:20 UTC
389-ds-base-1.2.10-0.6.a6.fc15, slapi-nis-0.28-1.fc15, freeipa-2.1.4-3.fc15 has been pushed to the Fedora 15 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 19 Fedora Update System 2012-02-02 17:32:12 UTC
slapi-nis-0.28-1.fc16, 389-ds-base-1.2.10-0.6.a6.fc16, freeipa-2.1.4-4.fc16 has been pushed to the Fedora 16 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 20 Fedora Update System 2012-02-04 00:46:17 UTC
389-dsgw-1.1.9-1.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/389-dsgw-1.1.9-1.fc16

Comment 21 Fedora Update System 2012-02-04 02:27:52 UTC
389-dsgw-1.1.9-1.fc15 has been submitted as an update for Fedora 15.
https://admin.fedoraproject.org/updates/389-dsgw-1.1.9-1.fc15

Comment 22 Fedora Update System 2012-02-04 03:38:02 UTC
389-ds-base-1.2.10-0.10.rc1.el5 has been submitted as an update for Fedora EPEL 5.
https://admin.fedoraproject.org/updates/389-ds-base-1.2.10-0.10.rc1.el5

Comment 23 Fedora Update System 2012-02-13 20:13:59 UTC
389-ds-base-1.2.10.0-1.el5 has been submitted as an update for Fedora EPEL 5.
https://admin.fedoraproject.org/updates/389-ds-base-1.2.10.0-1.el5

Comment 24 Fedora Update System 2012-02-23 15:50:51 UTC
389-ds-base-1.2.10.2-1.el5 has been submitted as an update for Fedora EPEL 5.
https://admin.fedoraproject.org/updates/389-ds-base-1.2.10.2-1.el5

Comment 25 Fedora Update System 2012-03-05 23:13:34 UTC
389-ds-base-1.2.10.3-1.el5 has been submitted as an update for Fedora EPEL 5.
https://admin.fedoraproject.org/updates/389-ds-base-1.2.10.3-1.el5

Comment 26 Fedora Update System 2012-03-27 14:04:43 UTC
389-ds-base-1.2.10.4-2.el5 has been submitted as an update for Fedora EPEL 5.
https://admin.fedoraproject.org/updates/389-ds-base-1.2.10.4-2.el5

Comment 27 Fedora Update System 2012-03-29 19:21:19 UTC
389-ds-base-1.2.10.4-3.el5 has been submitted as an update for Fedora EPEL 5.
https://admin.fedoraproject.org/updates/389-ds-base-1.2.10.4-3.el5

Comment 28 Fedora Update System 2012-03-31 03:26:01 UTC
389-dsgw-1.1.9-1.fc15 has been pushed to the Fedora 15 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 29 Fedora Update System 2012-03-31 03:26:51 UTC
389-dsgw-1.1.9-1.fc16 has been pushed to the Fedora 16 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.