Bug 1144132

Summary: calloc in dl-reloc.c computes size incorrectly
Product: Red Hat Enterprise Linux 6 Reporter: Ben Woodard <woodard>
Component: glibcAssignee: Carlos O'Donell <codonell>
Status: CLOSED ERRATA QA Contact: Arjun Shankar <ashankar>
Severity: medium Docs Contact:
Priority: medium    
Version: 6.5CC: ashankar, foraker1, fweimer, legendre1, mcermak, pfrankli, spoyarek, tgummels
Target Milestone: rc   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: glibc-2.12-1.158.el6 Doc Type: Bug Fix
Doc Text:
When the dynamic loader is profiling an application it maintains an internal cache of pre-computed data. This size of this cache of data was being incorrectly computed as being much larger than required. This large allocation could cause some profiled applications to fail with out of memory errors. The dynamic loader has been corrected to allocate only the required amount of storage for the internal cache. Applications should no longer fail with out of memory errors for this particular situation.
Story Points: ---
Clone Of:
: 1144133 (view as bug list) Environment:
Last Closed: 2015-07-22 06:14:43 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1138860    

Description Ben Woodard 2014-09-18 18:23:58 UTC
This bug was discovered by Matt LeGendre who works at LLNL. He pointed out the problem and corrected my patch to the problem. It seems to exist in quite a range of glibc versions. The problem was discovered in RHEL6's glibc and it still exists in RHEL7 and upstream.

In elf/dl-reloc.c the AUDIT code does a calloc of a library's  DT_PLTRELSZ*sizeof(struct reloc_result) (this is the one we're seeing making ~300MB of allocations).  It's treating the DT_PLTRELSZ as a count of PLTREL entries, but DT_PLTRELSZ is the size in bytes of the PLTREL entries.  So it's doing a much larger memory allocation than necessary.

I looked and the code is the same in RHEL6,7, and upstream glibc.
He suggested a patch like:

diff --git a/elf/dl-reloc.c b/elf/dl-reloc.c
index d2c6dac..0c85f08 100644
--- a/elf/dl-reloc.c
+++ b/elf/dl-reloc.c
@@ -280,7 +280,8 @@ _dl_relocate_object (struct link_map *l, struct r_scope_elem *scope[],
          }
 
        l->l_reloc_result = calloc (sizeof (l->l_reloc_result[0]),
-                                   l->l_info[DT_PLTRELSZ]->d_un.d_val);
+                                   l->l_info[DT_PLTRELSZ]->d_un.d_val /
+                                   (l->l_info[DT_PLTREL]->d_un.d_val == DT_RELA ? sizeof(ElfW(Rela)) : sizeof(ElfW(Rel))));
        if (l->l_reloc_result == NULL)
          {
            errstring = N_("\

Comment 8 errata-xmlrpc 2015-07-22 06:14:43 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-1286.html