Created attachment 893821 [details] proposed fix Description of problem: The nss-pam-ldapd daemon nslcd can be configured to find ldap servers via SRV lookups on a domain. With this configuration, a crash seen when starting nslcd can be caused by a number of specific SRV records presented to ldap_domain2hostlist(). The records are a set of 5+ SRV records for _ldap._tcp, all with a 5-digit port number, i.e. 12345. [root@auto1 ~]# dig SRV _ldap._tcp.rodan.local | grep ldap ; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.23.rc1.el6_5.1 <<>> SRV _ldap._tcp.rodan.local ;_ldap._tcp.rodan.local. IN SRV _ldap._tcp.rodan.local. 3600 IN SRV 1 5 15001 activedirectory2.rodan.local. _ldap._tcp.rodan.local. 3600 IN SRV 1 5 15001 activedirectory3.rodan.local. _ldap._tcp.rodan.local. 3600 IN SRV 1 5 15001 activedirectory4.rodan.local. _ldap._tcp.rodan.local. 3600 IN SRV 0 5 15001 ads.rodan.local. _ldap._tcp.rodan.local. 3600 IN SRV 0 5 15001 ads2.rodan.local. _ldap._tcp.rodan.local. 3600 IN SRV 0 5 15001 ads3.rodan.local. _ldap._tcp.rodan.local. 3600 IN SRV 1 5 15001 activedirectory.rodan.local. On startup this crash happens most of the time, and can appear differently: [root@auto1 ~]# nslcd -d nslcd: DEBUG: query rodan.local for SRV records *** glibc detected *** nslcd: realloc(): invalid next size: 0x0000000001e29480 *** ======= Backtrace: ========= /lib64/libc.so.6(+0x76166)[0x7fda91c7a166] /lib64/libc.so.6(+0x7bc17)[0x7fda91c7fc17] /lib64/libc.so.6(realloc+0xe5)[0x7fda91c7fdd5] /lib64/liblber-2.4.so.2(ber_memrealloc_x+0x2a)[0x7fda919fd87a] /lib64/libldap_r-2.4.so.2(ldap_domain2hostlist+0x423)[0x7fda92400f33] nslcd[0x407fb1] nslcd[0x408c0e] nslcd[0x40a21d] nslcd[0x403a75] /lib64/libc.so.6(__libc_start_main+0xfd)[0x7fda91c22d1d] nslcd[0x402d39] ======= Memory map: ======== 00400000-0041d000 r-xp 00000000 fd:00 405486 /usr/sbin/nslcd 0061d000-0061e000 rw-p 0001d000 fd:00 405486 /usr/sbin/nslcd 01e28000-01e49000 rw-p 00000000 00:00 0 [heap] 7fda8ed94000-7fda8ee05000 r-xp 00000000 fd:00 261638 /lib64/libfreebl3.so ... [root@auto1 ~]# nslcd -d nslcd: DEBUG: query rodan.local for SRV records *** glibc detected *** nslcd: realloc(): invalid next size: 0x00000000023b3480 *** *** glibc detected *** nslcd: malloc(): memory corruption: 0x00000000023b34f0 *** ^C [root@auto1 ~]# nslcd -d nslcd: DEBUG: query rodan.local for SRV records *** glibc detected *** nslcd: realloc(): invalid next size: 0x00000000020bd480 *** *** glibc detected *** nslcd: malloc(): memory corruption: 0x00000000020bd4f0 *** ^C [root@auto1 ~]# nslcd -d nslcd: DEBUG: query rodan.local for SRV records nslcd: DEBUG: add_uris_from_dns(): found uri: ldap://activedirectory4.rodan.local:15001 nslcd: DEBUG: add_uri(ldap://activedirectory4.rodan.local:15001) nslcd: DEBUG: add_uris_from_dns(): found uri: ldap://activedirectory.rodan.local:15001 nslcd: DEBUG: add_uri(ldap://activedirectory.rodan.local:15001) nslcd: DEBUG: add_uris_from_dns(): found uri: ldap://activedirectory3.rodan.local:15001 nslcd: DEBUG: add_uri(ldap://activedirectory3.rodan.local:15001) nslcd: DEBUG: add_uris_from_dns(): found uri: ldap://activedirectory2.rodan.local:15001 nslcd: DEBUG: add_uri(ldap://activedirectory2.rodan.local:15001) nslcd: DEBUG: add_uris_from_dns(): found uri: ldap://ads.rodan.local:15001 nslcd: DEBUG: add_uri(ldap://ads.rodan.local:15001) nslcd: DEBUG: add_uris_from_dns(): found uri: ldap://ads2.rodan.local:15001 nslcd: DEBUG: add_uri(ldap://ads2.rodan.local:15001) nslcd: DEBUG: add_uris_from_dns(): found uri: ldap://ads3.rodan.local:15001 The crash is related to the maximum length of a port specified in the SRV record. Any 4 digit or less ports did not crash with the same entries, so I suspected that in ldap_domain2hostlist(), doing STRLENOF(":65355") with the 5 digit port ends up missing a byte. It also only starts to show up when there are 5 or more entries in the host list, as it appears that the sprintf at the end of the hostent_count loop needs to be called enough times for this to corrupt the hostlist array. With a larger number of entries the crash would probably have a greater chance of happening. I'm attaching a patch that changes STRLENOF to sizeof, to account for the proper length needed in the buffer for a max length port. With the patch I tested multiple restarts of nslcd against the records and I wasn't able to reproduce the crash. I think this bug should be evaluated as a possible CVE. In the event of DNS hijacking/spoofing, a malicious nameserver presenting these specific SRV records may be able to cause a DoS to ldap services that utilize ldap_domain2hostlist(). Version-Release number of selected component (if applicable): openldap-2.4.23-34.el6_5.1 How reproducible: 1. Create SRV records in DNS as specified above 2. Configure nslcd to do SRV lookups to locate an ldap server (in /etc/nslcd.conf, set 'uri:DNS:domain') 3. Start nslcd in foreground debug mode (nslcd -d)
Acknowledgements: This issue was discovered by Matt Rogers of Red Hat.
Hey, sorry about that. I've verified that this does affect 2.4.39-6 as well [root@auto1 66]# rpm -qa | grep openldap openldap-devel-2.4.39-6.el6.x86_64 openldap-clients-2.4.39-6.el6.x86_64 openldap-servers-2.4.39-6.el6.x86_64 openldap-2.4.39-6.el6.x86_64 openldap-debuginfo-2.4.39-6.el6.x86_64 [root@auto1 66]# nslcd -d nslcd: DEBUG: query rodan.local for SRV records *** glibc detected *** nslcd: realloc(): invalid next size: 0x0000000000de8150 *** ======= Backtrace: ========= /lib64/libc.so.6(+0x76166)[0x7f8796dbb166] /lib64/libc.so.6(+0x7bc17)[0x7f8796dc0c17] /lib64/libc.so.6(realloc+0xe5)[0x7f8796dc0dd5] /lib64/liblber-2.4.so.2(ber_memrealloc_x+0x2a)[0x7f8796b3e88a] /lib64/libldap_r-2.4.so.2(ldap_domain2hostlist+0x423)[0x7f87975434b3] nslcd[0x407fb1] nslcd[0x408c0e] nslcd[0x40a21d] nslcd[0x403a75] /lib64/libc.so.6(__libc_start_main+0xfd)[0x7f8796d63d1d] nslcd[0x402d39] ======= Memory map: ======== 00400000-0041d000 r-xp 00000000 fd:00 405486 /usr/sbin/nslcd 0061d000-0061e000 rw-p 0001d000 fd:00 405486 /usr/sbin/nslcd 00de7000-00e08000 rw-p 00000000 00:00 0 [heap] 7f8793ed5000-7f8793f46000 r-xp 00000000 fd:00 261638 /lib64/libfreebl3.so 7f8793f46000-7f8794145000 ---p 00071000 fd:00 261638 /lib64/libfreebl3.so ldap_domain2hostlist() still has the vulnerable code in it on this version so my patch should still apply.
Howard, Hi, This is a strange openldap issue and possibly affects upstream builds as well, was wondering if you have any cycles to look at this?
(In reply to Huzaifa S. Sidhpurwala from comment #7) > Howard, > > Hi, This is a strange openldap issue and possibly affects upstream builds as > well, was wondering if you have any cycles to look at this? Hi, have you traced this under valgrind? I reproduced this configuration using dnsmasq: violino:~/OD/hobj/tests> tail /etc/dnsmasq.more.conf # redhat bug 1095976 srv-host=_ldap._tcp.rodan.local,activedirectory2.rodan.local,15001,1 srv-host=_ldap._tcp.rodan.local,activedirectory3.rodan.local,15001,1 srv-host=_ldap._tcp.rodan.local,activedirectory4.rodan.local,15001,1 srv-host=_ldap._tcp.rodan.local,ads.rodan.local,15001,1 srv-host=_ldap._tcp.rodan.local,ads2.rodan.local,15001,1 srv-host=_ldap._tcp.rodan.local,ads2.rodan.local,15001,1 srv-host=_ldap._tcp.rodan.local,activedirectory.rodan.local,15001,1 but see no crash: violino:~/OD/hobj/tests> ../clients/tools/ldapsearch -x -H ldap:///dc%3drodan%2cdc%3dlocal -s base -b dc=rodan,dc=local -d7 ldap_url_parse_ext(ldap:///dc%3drodan%2cdc%3dlocal) => ldap_bv2dn(dc=rodan,dc=local,0) <= ldap_bv2dn(dc=rodan,dc=local)=0 ldap_create ldap_url_parse_ext(ldap://activedirectory2.rodan.local:15001) ldap_url_parse_ext(ldap://activedirectory3.rodan.local:15001) ldap_url_parse_ext(ldap://activedirectory4.rodan.local:15001) ldap_url_parse_ext(ldap://ads.rodan.local:15001) ldap_url_parse_ext(ldap://ads2.rodan.local:15001) ldap_url_parse_ext(ldap://ads2.rodan.local:15001) ldap_url_parse_ext(ldap://activedirectory.rodan.local:15001) ldap_sasl_bind ldap_send_initial_request ldap_new_connection 1 1 0 ldap_int_open_connection ldap_connect_to_host: TCP activedirectory.rodan.local:15001 ldap_connect_to_host: getaddrinfo failed: Name or service not known ldap_int_open_connection ldap_connect_to_host: TCP ads2.rodan.local:15001 ^C Nor anything relevant from valgrind: /usr/bin/valgrind ../clients/tools/ldapsearch -x -H ldap:///dc%3drodan%2cdc%3dlocal -s base -b dc=rodan,dc=local -d7 ==28092== Memcheck, a memory error detector ==28092== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al. ==28092== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info ==28092== Command: ../clients/tools/ldapsearch -x -H ldap:///dc%3drodan%2cdc%3dlocal -s base -b dc=rodan,dc=local -d7 ==28092== ==28092== Conditional jump or move depends on uninitialised value(s) ==28092== at 0x4018E16: index (strchr.S:55) ==28092== by 0x4007FC2: expand_dynamic_string_token (dl-load.c:431) ==28092== by 0x40088D4: _dl_map_object (dl-load.c:2539) ==28092== by 0x400183D: map_doit (rtld.c:632) ==28092== by 0x400F705: _dl_catch_error (dl-error.c:177) ==28092== by 0x40010A6: do_preload (rtld.c:821) ==28092== by 0x4003AE1: dl_main (rtld.c:1635) ==28092== by 0x401678D: _dl_sysdep_start (dl-sysdep.c:241) ==28092== by 0x4005227: _dl_start (rtld.c:337) ==28092== by 0x4001657: ??? (in /lib/x86_64-linux-gnu/ld-2.17.so) ==28092== by 0x8: ??? ==28092== by 0x7FF0008A2: ??? ==28092== ==28092== Conditional jump or move depends on uninitialised value(s) ==28092== at 0x4018E1B: index (strchr.S:58) ==28092== by 0x4007FC2: expand_dynamic_string_token (dl-load.c:431) ==28092== by 0x40088D4: _dl_map_object (dl-load.c:2539) ==28092== by 0x400183D: map_doit (rtld.c:632) ==28092== by 0x400F705: _dl_catch_error (dl-error.c:177) ==28092== by 0x40010A6: do_preload (rtld.c:821) ==28092== by 0x4003AE1: dl_main (rtld.c:1635) ==28092== by 0x401678D: _dl_sysdep_start (dl-sysdep.c:241) ==28092== by 0x4005227: _dl_start (rtld.c:337) ==28092== by 0x4001657: ??? (in /lib/x86_64-linux-gnu/ld-2.17.so) ==28092== by 0x8: ??? ==28092== by 0x7FF0008A2: ??? ==28092== ldap_url_parse_ext(ldap:///dc%3drodan%2cdc%3dlocal) => ldap_bv2dn(dc=rodan,dc=local,0) <= ldap_bv2dn(dc=rodan,dc=local)=0 ldap_create ldap_url_parse_ext(ldap://activedirectory.rodan.local:15001) ldap_url_parse_ext(ldap://activedirectory2.rodan.local:15001) ldap_url_parse_ext(ldap://activedirectory3.rodan.local:15001) ldap_url_parse_ext(ldap://activedirectory4.rodan.local:15001) ldap_url_parse_ext(ldap://ads.rodan.local:15001) ldap_url_parse_ext(ldap://ads2.rodan.local:15001) ldap_url_parse_ext(ldap://ads2.rodan.local:15001) ldap_sasl_bind ldap_send_initial_request ldap_new_connection 1 1 0 ldap_int_open_connection ldap_connect_to_host: TCP ads2.rodan.local:15001 ^C==28092== ==28092== HEAP SUMMARY: ==28092== in use at exit: 39,349 bytes in 40 blocks ==28092== total heap usage: 180 allocs, 140 frees, 54,213 bytes allocated ==28092== ==28092== LEAK SUMMARY: ==28092== definitely lost: 0 bytes in 0 blocks ==28092== indirectly lost: 0 bytes in 0 blocks ==28092== possibly lost: 0 bytes in 0 blocks ==28092== still reachable: 39,349 bytes in 40 blocks ==28092== suppressed: 0 bytes in 0 blocks ==28092== Rerun with --leak-check=full to see details of leaked memory ==28092== ==28092== For counts of detected and suppressed errors, rerun with: -v ==28092== Use --track-origins=yes to see where uninitialised values come from ==28092== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0) I see no issue in vanilla libldap, anyway.
Looking at upstream git, this looks to have been fixed by d13285fd which would have the same effect as my patch: commit d13285fdd81eeec010dd273090d45e55f4b400fe Author: Kurt Zeilenga <kurt> Date: Mon Jul 8 18:45:53 2002 +0000 Fix possible under allocation of buffer diff --git a/libraries/libldap/dnssrv.c b/libraries/libldap/dnssrv.c index cc193e9..0f7c863 100644 --- a/libraries/libldap/dnssrv.c +++ b/libraries/libldap/dnssrv.c @@ -261,7 +261,7 @@ int ldap_domain2hostlist( /* weight = (p[2] << 8) | p[3]; */ port = (p[4] << 8) | p[5]; - buflen = strlen(host) + sizeof(":65355"); + buflen = strlen(host) + sizeof(":65355 "); hostlist = (char *) LDAP_REALLOC(hostlist, cur + buflen); if (hostlist == NULL) { rc = LDAP_NO_MEMORY;
(In reply to Matt Rogers from comment #9) > Looking at upstream git, this looks to have been fixed by d13285fd which > would have the same effect as my patch: > > commit d13285fdd81eeec010dd273090d45e55f4b400fe > Author: Kurt Zeilenga <kurt> > Date: Mon Jul 8 18:45:53 2002 +0000 How is it that your builds of OpenLDAP 2.4 are missing a fix from 12 years ago? > Fix possible under allocation of buffer > > diff --git a/libraries/libldap/dnssrv.c b/libraries/libldap/dnssrv.c > index cc193e9..0f7c863 100644 > --- a/libraries/libldap/dnssrv.c > +++ b/libraries/libldap/dnssrv.c > @@ -261,7 +261,7 @@ int ldap_domain2hostlist( > /* weight = (p[2] << 8) | p[3]; */ > port = (p[4] << 8) | p[5]; > > - buflen = strlen(host) + sizeof(":65355"); > + buflen = strlen(host) + sizeof(":65355 "); > hostlist = (char *) LDAP_REALLOC(hostlist, cur + buflen); > if (hostlist == NULL) { > rc = LDAP_NO_MEMORY;
I compared our code with upstream and tail section of ldap_domain2hostlist() is quite different. 332 add_size:; 333 p += size; 334 } 335 } 336 qsort(hostent_head, hostent_count, sizeof(srv_record), srv_cmp); 337 338 for(i=0; i<hostent_count; i++){ 339 int buflen; 340 buflen = strlen(hostent_head[i].hostname) + STRLENOF(":65355" ); 341 hostlist = (char *) LDAP_REALLOC(hostlist, cur+buflen+1); 342 if (hostlist == NULL) { 343 rc = LDAP_NO_MEMORY; 344 goto out; 345 } 346 if(cur>0){ 347 hostlist[cur++]=' '; 348 } 349 cur += sprintf(&hostlist[cur], "%s:%hd", hostent_head[i].hostname, hostent_head[i].port); 350 } 351 352 if (hostlist == NULL) { 353 /* No LDAP servers found in DNS. */ 354 rc = LDAP_UNAVAILABLE; 355 goto out; 356 } We include a patch for https://bugzilla.redhat.com/show_bug.cgi?id=733078 that involves these changes, ... - buflen = strlen(host) + STRLENOF(":65355 "); - hostlist = (char *) LDAP_REALLOC(hostlist, cur + buflen + 1); ... + qsort(hostent_head, hostent_count, sizeof(srv_record), srv_cmp); + + for(i=0; i<hostent_count; i++){ + int buflen; + buflen = strlen(hostent_head[i].hostname) + STRLENOF(":65355" ); + hostlist = (char *) LDAP_REALLOC(hostlist, cur+buflen+1); ... The STRLENOF(":65355" ); in the added line should have been STRLENOF(":65355 ");
I don't have permission to read 733078. Can you summarize what that report was about?
Sure, sorry about that. It's a request to add support for SRV lookups to honor weight/priority. There are more details (with the patch that is included in our packages) here: http://www.openldap.org/its/index.cgi/Incoming?id=7027;page=8
Thanks. A new patch is now in git upstream for ITS#7027.
Upstream references for the fix: http://www.openldap.org/its/index.cgi/Software%20Enhancements?id=7027 http://www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=blobdiff;f=libraries/libldap/dnssrv.c;h=de849e30d5b01ae855853c79e88fb06d7aea1137;hp=6d1bfa8e3c2b05ca5ed0ebebc00c3a30086bca95;hb=31995b535e10c45e698b62d39db998c51f799327;hpb=5de85b922aaa5bfa6eb53db6000adf01ebdb0736
(In reply to Vincent Danen from comment #19) > Upstream references for the fix: > > http://www.openldap.org/its/index.cgi/Software%20Enhancements?id=7027 > http://www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=blobdiff; > f=libraries/libldap/dnssrv.c;h=de849e30d5b01ae855853c79e88fb06d7aea1137; > hp=6d1bfa8e3c2b05ca5ed0ebebc00c3a30086bca95; > hb=31995b535e10c45e698b62d39db998c51f799327; > hpb=5de85b922aaa5bfa6eb53db6000adf01ebdb0736 For completeness' sake - the above patch prevents a crash, but the code is still broken. The complete fix is in the subsequent commit. http://www.openldap.org/devel/gitweb.cgi?p=openldap.git;a=commitdiff;h=eef1ca007f60fdcb9b5368608e87dd0b2404bceb
(Also for the record, the original patch with the bug was only present in the Red Hat tree. It was never committed to the main OpenLDAP repo by itself; it was committed with fixes on 2014-07-22. No vanilla OpenLDAP releases were ever affected by this bug.)
FYI, I backported upstream commits eef1ca007f60fdcb9b5368608e87dd0b2404bceb and 31995b535e10c45e698b62d39db998c51f799327.
Errata has been released for https://bugzilla.redhat.com/show_bug.cgi?id=1164369 I think this can be closed.
This flaw was never present in upstream openldap (comment 22), it was introduced to RHEL in openldap-2.3.43-dns-priority.patch on 2011-08-29 (solving bug 733435). For rhel-6 this was resolved as non-security in bug 1164369: https://rhn.redhat.com/errata/RHBA-2015-1292.html In rhel-7 the same rebase to 2.4.40 also (silently) resolved it: https://rhn.redhat.com/errata/RHSA-2015-2131.html rhel-5 is still affected, but the vulnerable configuration (slapd using dnssrv backend) is somewhat unusual (dnssrv is clearly marked as "experimental").
CVSS scoring based on an attacker able to send crafted DNS responses to slapd in a vulnerable configuration: while memory corruption is possible, it is not entirely controlled (SRV responses must parse correctly) and appears to quickly corrupt malloc metadata, leading to a crash before any C/I impact.