Bug 206288
| Summary: | sendmail crashes when LDAP routing is configured. | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | [Fedora] Fedora | Reporter: | John Haxby <jch> | ||||||||
| Component: | sendmail | Assignee: | Thomas Woerner <twoerner> | ||||||||
| Status: | CLOSED CANTFIX | QA Contact: | David Lawrence <dkl> | ||||||||
| Severity: | urgent | Docs Contact: | |||||||||
| Priority: | medium | ||||||||||
| Version: | 5 | CC: | lbenvenu | ||||||||
| Target Milestone: | --- | ||||||||||
| Target Release: | --- | ||||||||||
| Hardware: | x86_64 | ||||||||||
| OS: | Linux | ||||||||||
| Whiteboard: | |||||||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||||||
| Doc Text: | Story Points: | --- | |||||||||
| Clone Of: | Environment: | ||||||||||
| Last Closed: | 2007-08-29 17:20:33 UTC | Type: | --- | ||||||||
| Regression: | --- | Mount Type: | --- | ||||||||
| Documentation: | --- | CRM: | |||||||||
| Verified Versions: | Category: | --- | |||||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||||
| Embargoed: | |||||||||||
| Attachments: |
|
||||||||||
There's some odd line-wrapping in that example above, but I think you can see what's happening. Trying to debug this, the "LDAP *ld" pointer in sm_ldap_start starts out OK and becomes corrupt. In a wild flash of inspiration I changed "enable_pie yes" to "enable_pie no" in the sendmail.spec file and rebuilt to get this: [...] cc `-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic' -I. -I../../sendmail -I../../include -I/usr/include/db4 -I/usr/kerberos/include -Wall -DXDEBUG=0 -DTCPWRAPPERS -DNETINET6 -DHES_GETMAILHOST -DUSE_VENDOR_CF_PATH=1 -D_FFR_WORKAROUND_BROKEN_NAMESERVERS -D_FFR_SMTP_SSL -D_FFR_TLS_1 -DSASL=2 -D_FFR_UNSAFE_SASL -DSM_CONF_LDAP_MEMFREE=1 -DNOT_SENDMAIL -Dsm_snprintf=snprintf -D_FFR_MILTER_ROOT_UNSAFE -D_REENTRANT -DXP_MT -c -o main.o main.c /bin/sh: -c: line 0: unexpected EOF while looking for matching `'' /bin/sh: -c: line 1: syntax error: unexpected end of file make: *** [main.o] Error 2 which isn't promising. Nothing ventured nothing gained, I hacked around with stuff that gets generated when PIE is enabled (I replaced the value of the two APPENDDEF macros with a nothing). Anyway, that's a minor diversion. With PIE disabled, sendmail works fine. I don't know enough about PIE to know what's going on here, but there's certainly something unpleasant about it. Perhaps it's that the LDAP library needs to be PIE and isn't? (I'm guessing, in case it wasn't obvious!) The original example is now like this: $ /usr/sbin/sendmail -bt -d21.12 ADDRESS TEST MODE (ruleset 3 NOT automatically invoked) Enter <ruleset> <address> > /parse jch [...] -----trying rule: < $+ > < $+ > < $* > -----rule matches: $: < $( ldapmra $2 $: $) > < $( ldapmh $2 $: $) > < $1 > < $2 > < $3 > ldap_init/ldap_bind failed to localhost in map ldapmra: Unknown error 325 ldap_init/ldap_bind failed to localhost in map ldapmh: Unknown error 325 rewritten as: < jch @ example . com < TMPF > > < jch @ example . com < TMPF > > < jch < @ example . com . > > < jch @ example . com > < > -----trying rule: < $* < TMPF > > < $* > < $+ > < $+ > < $* > -----rule matches: $: $&{opMode} $| TMPF < $&{addr_type} > $| $3 rewrite: RHS $&{opMode} => "t" rewrite: RHS $&{addr_type} => "e r" rewritten as: t $| TMPF < e r > $| jch < @ example . com . > [...] final returns: jch @ example . com jch... Transient parse error -- message queued for future delivery mailer esmtp, host example.com., user jch > The transient error is because I don't have an LDAP server running. If I do then I don't even get that and the LDAP server gets a query and responds to it. What I'm struggling to say is that you can debug this problem without having to set up an LDAP server. Hi all,
Here at Mendoza state goverment, Argentina we used to have a rock solid setup
for ldap routing based on vanilla lachman-laser sendmail implementation on a
cluster of incoming servers with FC4 and RHEL4 for years.
When we ever try upgrade to FC5 x86_64 we hit this very same bug.
Same on FC6 x86_64.
But get this: ldap map for aliases works where ldap routing isn't!
try this in sendmail.mc on FC6 or 5:
define(`confLDAP_DEFAULT_SPEC', `-h ldap.exam.com -b dc=exam,dc=com')dnl
define(`ALIAS_FILE',`ldap -k
(&(objectClass=sendmailMTAAliasObject)(sendmailMTAAliasGrouping=aliases)
(|(sendmailMTACluster=${sendmailMTACluster})(sendmailMTAHost=$j))
(sendmailMTAKey=%0)) -v sendmailMTAAliasValue')dnl
this works for me on FC6 x86_64, but not the ldap routing setup, same as you.
Is this useful for debug the problem?
Thanks in advance,
l.
Please note: This bug is related to 64 bit arch. The bug _only_ shows on FC5 x86_64, FC6 x86_64. On FC5 i386, FC6 i386, even on FC4 x86_64, ldap routing still works great: $ cat /etc/fedora-release Fedora Core release 6 (Zod) $ cat /tmp/sendmail.mc | grep -i ldap | grep -v ^dnl define(`confLDAP_DEFAULT_SPEC',`-h localhost -b dc=example,dc=com')dnl LDAPROUTE_DOMAIN(`example.com')dnl FEATURE(`ldap_routing')dnl $ m4 </tmp/sendmail.mc >/tmp/sendmail.cf $ /usr/lib/sendmail -bt -C/tmp/sendmail.cf Enter <ruleset> <address> > /parse someone [...] final returns: someone @ example . com someone... Transient parse error -- message queued for future delivery mailer esmtp, host example.com., user someone > In previous post we omitted this *detail*. Our older hardware was not EM64T. We tried OS upgrade because brand new 64 bit servers arrived. It's true, you might have missed the fact that this was x86_64 specific (but possible s390x as well?) I've changed the hardware accordingly. Created attachment 140751 [details]
Patch to disable PIE for x86_64
I don't know why the PIE compilation is causing sendmail to fail when using
LDAP routing, but it does -- it's obviously some bad interaction with the ldap
libraries.
The attached patch is for the specfile for sendmail-8.13.8-1.fc5 and it does
two things: it makes enable_pie=no work :-) and sets that for x86_64. Having
the patch available makes it much easier for us to apply a fix until the
shipping sendmail works.
Can you please add a backtrace and core file for the original version? I can try -- with the latest update though, sendmai just hung rather than crashing. Can you not reproduce the problem with the instructions I gave? Following my own instructions, I added these three lines to my sendmail.mc: define(`confLDAP_DEFAULT_SPEC',`-h localhost -b dc=example,dc=com -M simple')dnl LDAPROUTE_DOMAIN(`example.com')dnl FEATURE(`ldap_routing')dnl (and I'll attach the sendmail.mc shortly). Note that to get good debug information I also had to install the openldap-debuginfo RPM. (It's interesting, it seems that sendmail was compiled on an RHEL4 machine :-), and this cd is necessary to get a good backtrace: # cd /usr/src/debug/sendmail-8.13.8/obj.Linux.2.6.9-34.ELsmp.x86_64/libsm # gdb sendmail GNU gdb Red Hat Linux (6.3.0.0-1.134.fc5rh) Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu"...Using host libthread_db library "/lib64/libthread_db.so.1". (gdb) run -d21.12 -bt Starting program: /usr/sbin/sendmail -d21.12 -bt [Thread debugging using libthread_db enabled] [New Thread 46912524077408 (LWP 11109)] ADDRESS TEST MODE (ruleset 3 NOT automatically invoked) Enter <ruleset> <address> > /parse jch Cracked address = $g Parsing envelope recipient address canonify input: jch @ example . com -----trying rule: $@ ----- rule fails -----trying rule: $* -----rule matches: $: $1 < @ > rewritten as: jch @ example . com < @ > -----trying rule: $* < $* > $* < @ > ----- rule fails -----trying rule: @ $* < @ > ----- rule fails -----trying rule: $* [ IPv6 : $+ ] < @ > ----- rule fails -----trying rule: $* : : $* < @ > ----- rule fails -----trying rule: : include : $* < @ > ----- rule fails -----trying rule: $* : $* [ $* ] ----- rule fails -----trying rule: $* : $* < @ > ----- rule fails -----trying rule: $* < @ > -----rule matches: $: $1 rewritten as: jch @ example . com -----trying rule: $* ; ----- rule fails -----trying rule: $* < $+ : ; > $* ----- rule fails -----trying rule: $* < $* ; > ----- rule fails -----trying rule: $@ ----- rule fails -----trying rule: $* -----rule matches: $: < $1 > rewritten as: < jch @ example . com > -----trying rule: $+ < $* > ----- rule fails -----trying rule: < $* > $+ ----- rule fails -----trying rule: < > ----- rule fails -----trying rule: < $+ > -----rule matches: $: $1 rewritten as: jch @ example . com -----trying rule: @ $+ , $+ ----- rule fails -----trying rule: @ [ $* ] : $+ ----- rule fails -----trying rule: @ $+ : $+ ----- rule fails -----trying rule: $+ : $* ; @ $+ ----- rule fails -----trying rule: $+ : $* ; ----- rule fails -----trying rule: $+ @ $+ -----rule matches: $: $1 < @ $2 > rewritten as: jch < @ example . com > -----trying rule: $+ < $+ @ $+ > ----- rule fails -----trying rule: $+ < @ $+ > -----rule matches: $@ $> Canonify2 $1 < @ $2 > Canonify2 input: jch < @ example . com > -----trying rule: $* < @ localhost > $* ----- rule fails -----trying rule: $* < @ localhost . uk . scalix . com > $* ----- rule fails -----trying rule: $* < @ localhost . UUCP > $* ----- rule fails -----trying rule: $* < @ [ $+ ] > $* ----- rule fails -----trying rule: $* < @ @ $=w > $* ----- rule fails -----trying rule: $* < @ @ $+ > $* ----- rule fails -----trying rule: $* < @ $+ . UUCP > $* ----- rule fails -----trying rule: $* < @ $+ . . UUCP . > $* ----- rule fails -----trying rule: $* < @ $* $=P > $* ----- rule fails -----trying rule: $* < @ $* $~P > $* -----rule matches: $: $&{daemon_flags} $| $1 < @ $2 $3 > $4 rewrite: RHS $&{daemon_flags} => "(NULL)" rewritten as: $| jch < @ example . com > -----trying rule: $* CC $* $| $* < @ $+ . $+ > $* ----- rule fails -----trying rule: $* CC $* $| $* ----- rule fails -----trying rule: $* $| $* < @ $* > $* -----rule matches: $: $2 < @ $[ $3 $] > $4 rewritten as: jch < @ example . com . > -----trying rule: $* $| $* ----- rule fails -----trying rule: $* < @ $=w > $* ----- rule fails -----trying rule: $* < @ $=M > $* ----- rule fails -----trying rule: $* < @ $={VirtHost} > $* ----- rule fails -----trying rule: $* < @ $* . . > $* ----- rule fails Canonify2 returns: jch < @ example . com . > rewritten as: jch < @ example . com . > canonify returns: jch < @ example . com . > parse input: jch < @ example . com . > -----trying rule: $* -----rule matches: $: $> Parse0 $1 Parse0 input: jch < @ example . com . > -----trying rule: < @ > ----- rule fails -----trying rule: $* : $* ; < @ > ----- rule fails -----trying rule: @ < @ $* > ----- rule fails -----trying rule: < @ $+ > ----- rule fails -----trying rule: $+ < @ > ----- rule fails -----trying rule: $* -----rule matches: $: < > $1 rewritten as: < > jch < @ example . com . > -----trying rule: < > $* < @ [ $* ] : $+ > $* ----- rule fails -----trying rule: < > $* < @ [ $* ] , $+ > $* ----- rule fails -----trying rule: < > $* < @ [ $* ] $+ > $* ----- rule fails -----trying rule: < > $* < @ [ $+ ] > $* ----- rule fails -----trying rule: < > $* < $* : $* > $* ----- rule fails -----trying rule: < > $* -----rule matches: $1 rewritten as: jch < @ example . com . > -----trying rule: < > $* ----- rule fails -----trying rule: $* < @ . $* > $* ----- rule fails -----trying rule: $* < @ $* . . $* > $* ----- rule fails -----trying rule: $* < @ $* @ > $* ----- rule fails -----trying rule: $* @ $* < @ $* > $* ----- rule fails -----trying rule: $* , $~O $* ----- rule fails -----trying rule: $* < @ > $* ----- rule fails -----trying rule: < @ $=w . > : $* ----- rule fails -----trying rule: $- < @ $=w . > ----- rule fails -----trying rule: < @ $+ > ----- rule fails -----trying rule: $* $=O $* < @ $=w . > ----- rule fails -----trying rule: $- ----- rule fails -----trying rule: < @ *LOCAL* > ----- rule fails -----trying rule: $* $=O $* < @ *LOCAL* > ----- rule fails -----trying rule: $* < @ *LOCAL* > ----- rule fails Parse0 returns: jch < @ example . com . > rewritten as: jch < @ example . com . > -----trying rule: < @ > ----- rule fails -----trying rule: $* -----rule matches: $: $> ParseLocal $1 ParseLocal input: jch < @ example . com . > -----trying rule: $* < @ $+ . REDIRECT . > ----- rule fails -----trying rule: $* < @ $+ . REDIRECT . > < i > ----- rule fails -----trying rule: $* < @ $+ . REDIRECT . > < $- > ----- rule fails ParseLocal returns: jch < @ example . com . > rewritten as: jch < @ example . com . > -----trying rule: $* -----rule matches: $: $> Parse1 $1 Parse1 input: jch < @ example . com . > -----trying rule: $+ < @ $={LDAPRoute} . > -----rule matches: $: $> LDAPExpand < $1 < @ $2 . > > < $1 @ $2 > < > LDAPExpand input: < jch < @ example . com . > > < jch @ example . com > < > -----trying rule: < $+ > < $+ > < $* > -----rule matches: $: < $( ldapmra $2 $: $) > < $( ldapmh $2 $: $) > < $1 > < $2 > < $3 > Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 46912524077408 (LWP 11109)] ldap_set_option (ld=0x557a6740, option=2, invalue=0x5555557a828c) at ../../../libraries/libldap/options.c:358 358 assert( LDAP_VALID( ld ) ); (gdb) bt #0 ldap_set_option (ld=0x557a6740, option=2, invalue=0x5555557a828c) at ../../../libraries/libldap/options.c:358 #1 0x00005555555e8ec5 in sm_ldap_setopts (ld=0x557a6740, lmap=0x5555557a8270) at ldap.c:1300 #2 0x00005555555ea624 in sm_ldap_start (name=Variable "name" is not available. ) at ldap.c:222 #3 0x0000555555594041 in ldapmap_open (map=0x5555557a8128, mode=Variable "mode" is not available. ) at map.c:3360 #4 0x0000555555590f2d in openmap (map=0x5555557a8128) at map.c:539 #5 0x00005555555a6863 in rewrite (pvp=0x7fffb73407e0, ruleset=Variable "ruleset" is not available. ) at parseaddr.c:1752 #6 0x00005555555a6c91 in rewrite (pvp=0x7fffb7345870, ruleset=Variable "ruleset" is not available. ) at parseaddr.c:1711 #7 0x00005555555a6c91 in rewrite (pvp=0x5555557251e0, ruleset=Variable "ruleset" is not available. ) at parseaddr.c:1711 #8 0x00005555555a9035 in parseaddr (addr=0x7fffb734bb27 "jch", a=0x7fffb73491d0, flags=0, delim=0, delimptr=Variable "delimptr" is not available. ) at parseaddr.c:135 #9 0x0000555555567149 in testmodeline (line=0x7fffb734bb20 "/parse", e=0x555555711d40) at main.c:4318 #10 0x000055555556ad9f in main (argc=3, argv=Variable "argv" is not available. ) at main.c:2094 (gdb) p *ld Cannot access memory at address 0x557a6740 When I stepped through this with the debugger to find out what was broken, I found nothing wrong with the code: it's just that at some point in sm_ldap_start() the "ld" variable acquires (or seems to acquire) a garbage value. As I said, I don't know what sudden leap of faith made me think of turning PIE off, but that did the trick and "ld" keeps a good value and previously non-working code starts working again. Is there perhaps a problem combining PIE code (from sendmail) with non-PIE code (from openldap)? Created attachment 141066 [details]
sendmail.mc file used to reproduce the problem.
Just pop this in /etc/mail and run "make".
Created attachment 141067 [details]
core file from sendmail
Not sure how much good this is going to do you, but you asked for it :-) The
backtrace from this core is less than useful, probably because sendmail is
compiled as a position-independant executable:
#0 ldap_set_option (ld=0x557a61d0, option=2, invalue=0x5555557a7d1c)
at ../../../libraries/libldap/options.c:358
#1 0x00005555555e8ec5 in sleep () from /etc/mail/sendmail
#2 0x00005555555ea624 in sleep () from /etc/mail/sendmail
#3 0x0000555555594041 in ldapmap_open () from /etc/mail/sendmail
#4 0x0000555555590f2d in seq_map_parse () from /etc/mail/sendmail
#5 0x00005555555a6863 in dequote_init () from /etc/mail/sendmail
#6 0x00005555555a6c91 in dequote_init () from /etc/mail/sendmail
#7 0x00005555555a6c91 in dequote_init () from /etc/mail/sendmail
#8 0x00005555555a9035 in dequote_init () from /etc/mail/sendmail
#9 0x0000555555567149 in intsig () from /etc/mail/sendmail
#10 0x000055555556ad9f in main () from /etc/mail/sendmail
(I'm pretty sure sleep() doesn't call ldap_set_option()).
Ok, the only platforms where this bug we could see were: - FC5 x86_64 - FC6 x86_64 We have not try it on ppc, s390, other 64 bit hardware besides x86_64. The problem seems to me linked to calls to /usr/lib64/libldap-2.3.so.0, part of openldap 2.3.x on x86_64. BTW, to fire the bug on FC5/6 x86_64, you have not to write your /etc/mail/sendmail.mc, or to be root. Just add these three lines to a copy of a vanilla sendmail.mc from sendmail RPM before mailer statements $ diff /tmp/sendmail.mc /etc/mail/sendmail.mc 40,42d39 < define(`confLDAP_DEFAULT_SPEC',`-h localhost -b dc=example,dc=com')dnl < LDAPROUTE_DOMAIN(`example.com')dnl < FEATURE(`ldap_routing')dnl $ m4 </tmp/sendmail.mc >/tmp/sendmail.cf $ /usr/lib/sendmail -bt -C/tmp/sendmail.cf ADDRESS TEST MODE (ruleset 3 NOT automatically invoked) Enter <ruleset> <address> > /parse someone Cracked address = $g Parsing envelope recipient address canonify input: someone @ example . com Canonify2 input: someone < @ example . com > Canonify2 returns: someone < @ example . com . > canonify returns: someone < @ example . com . > parse input: someone < @ example . com . > Parse0 input: someone < @ example . com . > Parse0 returns: someone < @ example . com . > ParseLocal input: someone < @ example . com . > ParseLocal returns: someone < @ example . com . > Parse1 input: someone < @ example . com . > LDAPExpand input: < someone < @ example . com . > > < someone @ example . com > < > Segmentation fault Keep in mind, as noted before, the bug is NOT found on previous x86_64: - FC4 x86_64 - RHEL4 x86_64 FC4 and RHEL4 were based on openldap 2.2.x. and worked fine on x86_64. We actually have four servers working with this setup. See the test ona a production server FC4 x86_64: $ rpm -q --queryformat "%{NAME} %{VERSION} %{RELEASE} %{ARCH}\n" kernel sendmail openldap kernel 2.6.17 1.2142_FC4 x86_64 kernel 2.6.17 1.2141_FC4 x86_64 sendmail 8.13.7 2.fc4.2 x86_64 openldap 2.2.29 1.FC4 x86_64 $ ldd /usr/lib/sendmail.sendmail [...] libldap-2.2.so.7 => /usr/lib64/libldap-2.2.so.7 (0x00002aaaab8dc000) liblber-2.2.so.7 => /usr/lib64/liblber-2.2.so.7 (0x00002aaaaba11000) [...] $ cat /tmp/sendmail.mc | grep -i ldap | grep -v ^dnl define(`confLDAP_DEFAULT_SPEC',`-h localhost -b dc=example,dc=com')dnl LDAPROUTE_DOMAIN(`example.com')dnl FEATURE(`ldap_routing')dnl $ m4 </tmp/sendmail.mc >/tmp/sendmail.cf $ /usr/lib/sendmail -bt -C/tmp/sendmail.cf Enter <ruleset> <address> > /parse someone [...] final returns: someone @ example . com someone... Transient parse error -- message queued for future delivery mailer esmtp, host example.com., user someone > quit The "Transient parse error" is not important, because there in no real setup of ldap routing for example.com here. The important fact is that sendmail NOT segfaults. Please have a look at http://people.redhat.com/twoerner/BZ/206288/. There is a new sendmail package for FC-6. Please try the following change in sendmail.spec for FC-5: -APPENDDEF(\`confMAPDEF', \`-DLDAPMAP')dnl +APPENDDEF(\`confMAPDEF', \`-DLDAPMAP -DLDAP_DEPRECATED')dnl Works perfectly. Thank you. Is there any danger of this fix being pushed out for Fedora Core 5 and Fedora Core 6? Other people are finding this, for example: https://bugzilla.scalix.com/show_bug.cgi?id=14427 FC-5 is EOL. There is a sendmail-8.14.1 package in FC-6 and F-7 testing. Closing as "CANTFIX" |
Description of problem: Sendmail crashes as soon as it tries to use LDAP for routing. Version-Release number of selected component (if applicable): sendmail-8.13.7-2.fc5.2.x86_64 How reproducible: Every time. Steps to Reproduce: 1. Configure sendmail for LDAP routing. I added this to my sendmail.mc file: define(`confLDAP_DEFAULT_SPEC',`-h localhost -b dc=example,dc=com -M simple')dnl LDAPROUTE_DOMAIN(`example.com')dnl feature(`ldap_routing')dnl Note that you don't need an LDAP server since it's not going to get that far :-) 2. Run "/usr/lib/sendmail -bt -d21.12" and type "/parse jch" at the ">" prompt. Actual results: [...] ParseLocal returns: jch < @ example . com . > rewritten as: jch < @ example . com . > -----trying rule: $* -----rule matches: $: $> Parse1 $1 Parse1 input: jch < @ example . com . > -----trying rule: $+ < @ $={LDAPRoute} . > -----rule matches: $: $> LDAPExpand < $1 < @ $2 . > > < $1 @ $2 > < > LDAPExpand input: < jch < @ example . com . > > < jch @ example . com > < > -----trying rule: < $+ > < $+ > < $* > -----rule matches: $: < $( ldapmra $2 $: $) > < $( ldapmh $2 $: $) > < $1 > < $2 > < $3 > Segmentation fault Expected results: Something that doesn't end with a segmentation fault :-)