RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1293976 - CVE-2015-5229 glibc: calloc() returns non-zero'ed memory [rhel-7.3.0]
Summary: CVE-2015-5229 glibc: calloc() returns non-zero'ed memory [rhel-7.3.0]
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: glibc
Version: 7.2
Hardware: All
OS: Linux
high
high
Target Milestone: rc
: ---
Assignee: Carlos O'Donell
QA Contact: Arjun Shankar
URL:
Whiteboard:
: 1294080 (view as bug list)
Depends On: 1246713 1294080
Blocks: CVE-2015-5229
TreeView+ depends on / blocked
 
Reported: 2015-12-23 21:05 UTC by Jeff Layton
Modified: 2019-10-10 10:47 UTC (History)
17 users (show)

Fixed In Version: glibc-2.17-107.el7
Doc Type: Bug Fix
Doc Text:
It was discovered that the calloc implementation in glibc could return memory areas which contain non-zero bytes. This could result in unexpected application behavior such as hangs or crashes.
Clone Of: 1246713
Environment:
Last Closed: 2016-11-03 08:28:55 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)
reproducer for the problem (1.02 KB, text/plain)
2015-12-23 21:07 UTC, Jeff Layton 		no flags Details
patch - malloc: only do the alloc_perturb if the perturb_byte isn't 0 (793 bytes, patch)
2015-12-23 21:08 UTC, Jeff Layton 		no flags Details | Diff
View All


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2016:2573 0 normal SHIPPED_LIVE Low: glibc security, bug fix, and enhancement update 2016-11-03 12:05:56 UTC

Description Jeff Layton 2015-12-23 21:05:04 UTC 
+++ This bug was initially created as a clone of Bug #1246713 +++

Description of problem:
At Facebook we had an app that started hanging and crashing weirdly when going from glibc-2.12-1.149.el6.x86_64 to glibc-2.12-1.163.el6.x86_64.  Turns out this patch

glibc-rh1066724.patch

Introduced the problem.

You added the following bit to _int_malloc()

+  /* There are no usable arenas.  Fall back to sysmalloc to get a chunk from
+     mmap.  */
+  if (__glibc_unlikely (av == NULL))
+    {
+      void *p = sYSMALLOc (nb, av);
+      if (p != NULL)
+       alloc_perturb (p, bytes);
+      return p;
+    }
+

But this isn't ok, alloc_perturb unconditionally memset's the front byte to 0xf, unlike upstream where it checks to see if perturb_byte is set.  This needs to be changed to

if (p != NULL && && __builtin_expect(perturb_byte, 0))
   alloc_perturb (p, bytes);
return p;

The patch I've attached fixes the problem for me.

This problem is exacerbated by the fact that any sort of lock contention on the arena's results in us falling back on mmap()'ing a new chunk.  This is because we check to see if the uncontended arena we check is corrupt, and if it is we loop through, and if we loop to the beginning we know we didn't find anything.  Except if our initial arena isn't actually corrupt we'll still return NULL, so we fall back on this mmap() thing more often, which really makes things unstable.

Please get this fixed as soon as possible, I'd even go so far as to call it a possible security issue.

--- Additional comment from Carlos O'Donell on 2015-07-24 23:11:01 EDT ---

(In reply to Josef Bacik from comment #0)
> Created attachment 1055966 [details]
> patch to fix the problem.
> 
> Description of problem:
> At Facebook we had an app that started hanging and crashing weirdly when
> going from glibc-2.12-1.149.el6.x86_64 to glibc-2.12-1.163.el6.x86_64. 

Please note that there is already a RHEL 6.7.z errata that fixes this, and it was released two days ago:
https://rhn.redhat.com/errata/RHBA-2015-1465.html

Please update to glibc-2.12-1.166.el6_7.1.

One question, when you write "glibc-2.12-1.163.el6.x86_64" do you actually mean "glibc-2.12-1.166.el6.x86_64?" (note .166 not .163)?

Lastly, the robust malloc support has been backed out for the release, but we plan to put it back in as soon as we are certain we've corrected the remaining issues. Would you be interested in testing an unsupported non-production build with the new feature?

--- Additional comment from Josef Bacik on 2015-07-25 11:44:01 EDT ---

We're on Centos, not RHEL, we just happened to end up with the .163 release (I'm not sure how) before 6.7 was released.  Give me whatever package you want me to test, we don't care about unsupported, obviously we are capable of supporting ourselves ;).  I do need to have an src.rpm tho so I can build and test it on our systems and verify the issue I was seeing is actually fixed.

--- Additional comment from Carlos O'Donell on 2015-08-04 14:59:49 EDT ---

(In reply to Josef Bacik from comment #5)
> We're on Centos, not RHEL, we just happened to end up with the .163 release
> (I'm not sure how) before 6.7 was released.  Give me whatever package you
> want me to test, we don't care about unsupported, obviously we are capable
> of supporting ourselves ;).  I do need to have an src.rpm tho so I can build
> and test it on our systems and verify the issue I was seeing is actually
> fixed.

Sounds good. We'll get you something when we're ready. Thanks for agreeing to test :-)

--- Additional comment from Siddhesh Poyarekar on 2015-08-20 00:43:00 EDT ---

Removing the "already fixed in 6.7.z" from the title because it confused me the couple of times I read it.

--- Additional comment from Florian Weimer on 2015-08-24 05:58:22 EDT ---

This issue has been addressed in the following products:

  Red Hat Enterprise Linux 6

Via RHBA-2015:1465 https://rhn.redhat.com/errata/RHBA-2015-1465.html


The SRPM is available here: ftp://ftp.redhat.com/pub/redhat/linux/enterprise/6Server/en/os/SRPMS/glibc-2.12-1.166.el6_7.1.src.rpm
Comment 1 Jeff Layton 2015-12-23 21:06:00 UTC 
RHEL7 seems to have the same problem as Josef reported here. Attaching a reproducer and a patch that fixes this.
Comment 2 Jeff Layton 2015-12-23 21:07:15 UTC 
Created attachment 1109015 [details]
reproducer for the problem

Reproducer. Build this and watch the asserts pop.
Comment 3 Jeff Layton 2015-12-23 21:08:27 UTC 
Created attachment 1109016 [details]
patch - malloc: only do the alloc_perturb if the perturb_byte isn't 0

This patch fixes the problem for me.
Comment 4 Jeff Layton 2015-12-23 21:12:57 UTC 
Note that we have an official support contract as well and have opened a case for this:

    https://access.redhat.com/support/cases/#/case/01558997
Comment 6 Jeff Layton 2015-12-23 22:02:44 UTC 
Oh and to be clear, this is seen in glibc-2.17-106.el7_2.1.
Comment 7 Jeff Layton 2015-12-24 13:29:35 UTC 
...and appears to be a regression from RHEL7.1 (just by inspection, I haven't actually tested it on RHEL7.1 to confirm it though).
Comment 8 Paulo Andrade 2015-12-24 13:40:43 UTC 
*** Bug 1294080 has been marked as a duplicate of this bug. ***
Comment 11 Carlos O'Donell 2015-12-24 15:57:17 UTC 
This is indeed a regression between rhel-7.1 and rhel-7.2. We dropped one of the fixes that went into rhel-6.7/rhel-6.8 and we should not have, to be specific it was this fix:

commit e8349efd466cfedc0aa98be61d88ca8795c9e565
Author: Ondřej Bílka <neleai>
Date:   Mon Dec 9 17:25:19 2013 +0100

    Simplify perturb_byte logic.

Which is the correct way to resolve this problem by simplifying the perturbation logic to match upstream (moves the perturb == 0 check into one central place).
Comment 13 Carlos O'Donell 2015-12-24 16:21:29 UTC 
We consider this a serious issue and are looking into resolving this as quickly as possible. If you have any questions please don't hesitate to ask.

One possible workaround on affected systems may be to set the environment variable MALLOC_PERTURB to 255 to nullify the perturbation effects e.g. 0xff ^ 0xff == 0. This has performance implications though since that additional operation and write have a non-zero cost.
Comment 15 Jeff Layton 2015-12-24 16:34:07 UTC 
Thanks,

Yes, commit e8349efd466c seems like a much better fix since you always want to check the perturb_byte value before applying it.

We're also aware of the workaround and it does work. We decided instead to just build a glibc package for now with the patch above to use internally until RH releases an official fix.

Let us know if you'd like us to test out anything in the meantime.
Comment 16 Carlos O'Donell 2015-12-24 16:36:31 UTC 
(In reply to Jeff Layton from comment #15)
> Thanks,
> 
> Yes, commit e8349efd466c seems like a much better fix since you always want
> to check the perturb_byte value before applying it.
> 
> We're also aware of the workaround and it does work. We decided instead to
> just build a glibc package for now with the patch above to use internally
> until RH releases an official fix.
> 
> Let us know if you'd like us to test out anything in the meantime.

Thank you for the feedback Jeff.
Comment 17 Florian Weimer 2015-12-29 20:07:25 UTC 
(In reply to Jeff Layton from comment #2)
> Created attachment 1109015 [details]
> reproducer for the problem
> 
> Reproducer. Build this and watch the asserts pop.

Thanks, I expanded it and committed it upstream:

https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=1bd5483e104c8bde6e61dc5e3f8a848bc861872d
Comment 18 Jeff Layton 2015-12-29 20:13:21 UTC 
Actually the reproducer was provided by Idan Kedar <idank>...I cc'ed him here (I should have credited him when I uploaded it originally, my bad...)
Comment 20 Jeff Layton 2016-01-25 16:07:30 UTC 
We're at over a month since I first reported this. Any ETA on when the fixed packages will show up in the repo?
Comment 21 Carlos O'Donell 2016-01-25 18:05:51 UTC 
(In reply to Jeff Layton from comment #20)
> We're at over a month since I first reported this. Any ETA on when the fixed
> packages will show up in the repo?

Do you need immediate fix for this in rhel-7.2.z? Is this a production issue for you? Please talk to GSS if the workaround (set MALLOC_PERTRUB=255) is not working.

We plan to fix this for rhel-7.3.0 and rhel-7.2.z. The general availability of RHEL 7.3 is sometime in the future. The release of a rhel-7.2.z is also sometime in the future. I can't give you any specific dates unfortunately. 

All I can say is that we are releasing a rhel-7.2.z update as soon as we possibly can. I apologize for the delay.
Comment 27 errata-xmlrpc 2016-11-03 08:28:55 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2016-2573.html
Note You need to log in before you can comment on or make changes to this bug.