Bug 732928 (CVE-2011-3192)

Summary: CVE-2011-3192 httpd: multiple ranges DoS
Product: [Other] Security Response Reporter: Mark J. Cox <mjc>
Component: vulnerabilityAssignee: Red Hat Product Security <security-response-team>
Status: CLOSED ERRATA QA Contact:
Severity: high Docs Contact:
Priority: high    
Version: unspecifiedCC: alasdair.gow, bloch, brandono, caluml, casmith, cmurtaugh, fschwarz, gdr, gerrit.slomma, gianluca.varisco, jentrena, jhouska, jkaluza, jorton, kevin, lyarwood, me, mmello, moshiro, pcheung, pep, phil, pmathur, prc, raina, rdassen, redhat, rhel, ricardo.arguello, rmeggins, rmunilla, rvandolson, sgreen, sharp, shyam, simon.matter, soho, stanke, sven, tcallawa, tmz, tuju, upendra.gandhi, uwe.knop, vasmith, vdanen, ville, vincent1984, weli, wnefal+redhatbugzilla, xset1980, yozone
Target Milestone: ---Keywords: Security
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: httpd 2.2.20 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-08-08 10:59:40 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 733058, 733059, 733061, 733062, 733063, 733095, 733096, 733097, 733215, 733217, 733218, 733223, 735886, 736757, 736758, 736760, 745067    
Bug Blocks: 732930, 2191706    
Attachments:
Description Flags
(Incomplete) Patch used by Debian 6 stable, fixed none

Description Mark J. Cox 2011-08-24 07:52:25 UTC
An exploit was posted to full-disclosure labelled "Apache Killer".  This script creates a number of threads that use multiple Range headers to exhaust memory on the Apache server.

The ASF httpd development team are working on a fix for this issue:
http://www.gossamer-threads.com/lists/apache/dev/401638

Comment 1 Tomas Hoger 2011-08-24 14:01:27 UTC
(In reply to comment #0)
> An exploit was posted to full-disclosure labelled "Apache Killer".  This script
> creates a number of threads that use multiple Range headers to exhaust memory
> on the Apache server.

http://lists.grok.org.uk/pipermail/full-disclosure/2011-August/082299.html

Comment 5 Tomas Hoger 2011-08-24 16:39:18 UTC
Upstream has released an advisory, which documents possible configuration changes that can be used to mitigate this flaw before the final fix is available:

http://mail-archives.apache.org/mod_mbox/httpd-announce/201108.mbox/%3C20110824161640.122D387DD@minotaur.apache.org%3E

Note: the mitigation 1, option 1 (using SetEnvIf and conditional RequestHeader unset) can not be used with httpd packages in Red Hat Enterprise Linux 3 and 4 (based on httpd 2.0.x upstream versions) that do no allow env clauses with RequestHeader unset.

Comment 8 Tomas Hoger 2011-08-25 07:43:27 UTC
Created httpd tracking bugs for this issue

Affects: fedora-all [bug 733217]

Comment 11 Tomas Hoger 2011-08-25 09:00:13 UTC
Statement:

Before updated packages are deployed, users can deploy configuration changes to mitigate this flaw:
https://bugzilla.redhat.com/show_bug.cgi?id=CVE-2011-3192#c18

Comment 18 Tomas Hoger 2011-08-26 10:54:20 UTC
(In reply to comment #5)
> Upstream has released an advisory, which documents possible configuration
> changes that can be used to mitigate this flaw before the final fix is
> available:
> 
> http://mail-archives.apache.org/mod_mbox/httpd-announce/201108.mbox/%3C20110824161640.122D387DD@minotaur.apache.org%3E

Upstream has released updated advisory:

http://lists.grok.org.uk/pipermail/full-disclosure/2011-August/082427.html

Following paragraph of the updated advisory describes changes from the previous version:

  In addition to the 'Range' header - the 'Range-Request' header is equally
  affected. Furthermore various vendor updates, improved regexes (speed and
  accommodating a different and new attack pattern).

Note that previously published mitigations 1) and 3) were incomplete and failed to block attack using Request-Range header, which is treated as equivalent to Range header.  All previously deployed mitigations 1) and 3) need to be updated!

Upstream now also lists 1) Option 1 as being applicable to 2.2 versions only due to the bug in 2.0 versions.  That workaround can not be used with httpd packages in Red Hat Enterprise Linux 3 and 4, as was previously noted in comment #5.

Comment 19 Tomas Hoger 2011-08-26 20:30:22 UTC
(In reply to comment #18)
> Upstream has released updated advisory:
> 
> http://lists.grok.org.uk/pipermail/full-disclosure/2011-August/082427.html

Additional notes regarding mitigation 1, Option 2:

- Mitigation always unsets Request-Range header.  Additionally, a commented-out example is provided how Request-Range can be handled in a similar way as Range header and only reject request if too many ranges are specified:

    RewriteCond %{HTTP:range} !(bytes=[^,]+(,[^,]+){0,4}$|^$)
    # RewriteCond %{HTTP:request-range} !(bytes=[^,]+(?:,[^,]+){0,4}$|^$)

If the second RewriteCond is uncommented, the condition will be ANDed with previous, while is should be ORed.  That can be achieved by using [OR] flag for the first RewriteCond:

    RewriteCond %{HTTP:range} !(bytes=[^,]+(,[^,]+){0,4}$|^$) [OR]

- Rewrite rules specified in the server context are not automatically inherited by virtual hosts.  Therefore, this mitigation need to be duplicated in each virtual host, or each virtual host need to be explicitly configured to inherit server context rewrite rules.  See mod_rewrite documentation for details:

  http://httpd.apache.org/docs/2.2/mod/mod_rewrite.html#vhosts

Comment 20 Mark J. Cox 2011-08-29 15:24:16 UTC
Status update: Upstream are currently working on the security patch for this issue for Apache 2.2 and are expected to release 2.2.20 in due course.  Meanwhile we've been backporting and testing the preliminary patches so that we can rapidly prepare updates for the affected products once a final patch is agreed.

Comment 21 Ray Van Dolson 2011-08-29 15:36:39 UTC
Is an updated RHWAS Apache release planned as well, or just the base OS Apaches?

Comment 22 Tomas Hoger 2011-08-30 07:01:48 UTC
Patch in the upstream SVN 2.2.x branch:

http://svn.apache.org/viewvc/httpd/httpd/branches/2.2.x/modules/http/byterange_filter.c?r1=916627&r2=1162885

which consists of:

http://svn.apache.org/viewvc?view=revision&revision=1162874
http://svn.apache.org/viewvc?view=revision&revision=1162881
http://svn.apache.org/viewvc?view=revision&revision=1162885

Summary of the changes from the CHANGES file:

  *) SECURITY: CVE-2011-3192 (cve.mitre.org)
     core: Fix handling of byte-range requests to use less memory, to avoid
     denial of service. If the sum of all ranges in a request is larger than
     the original file, ignore the ranges and send the complete file.
     PR 51714. [Stefan Fritsch, Jim Jagielski, Ruediger Pluem, Eric Covener]

Upstream bug, which does not have much extra info:

https://issues.apache.org/bugzilla/show_bug.cgi?id=51714

Comment 24 Tomas Hoger 2011-08-30 13:04:32 UTC
(In reply to comment #21)
> Is an updated RHWAS Apache release planned as well, or just the base OS
> Apaches?

Red Hat Application Stack is in Phase 2 of its life cycle, during which only impact critical issues are addressed.  See errata support policy for details:

https://access.redhat.com/support/policy/updates/rhappstack/

For any support beyond defined scope, please contact Red Hat Support:

https://access.redhat.com/support/

Comment 25 Tomas Hoger 2011-08-30 14:34:25 UTC
(In reply to comment #18)
> Upstream has released updated advisory:
> 
> http://lists.grok.org.uk/pipermail/full-disclosure/2011-August/082427.html

Few more notes on how each workaround affects httpd behaviour:

1.1 SetEnvIf Range (?:,.*?){5,5} bad-range=1
    RequestHeader unset Range env=bad-range
    RequestHeader unset Request-Range

Requests with 5 or fewer ranges specified using Range header are processed as before - i.e. httpd responds with "206 Partial Content" and provides only specified file ranges when possible.  If more than 5 ranges are specified, Range header is ignored/unset, which causes httpd to respond with "200 OK" and return the whole requested file rather than only returning specified ranges.

Request-Range header is always unset, causing the whole file to be returned when client tries to use that header.

1.2 RewriteEngine on
    RewriteCond %{HTTP:range} !(bytes=[^,]+(,[^,]+){0,4}$|^$)
    RewriteRule .* - [F]
    RequestHeader unset Request-Range

Requests with 5 or fewer ranges specified using Range header are processed as before - i.e. httpd responds with "206 Partial Content" and provides only specified file ranges when possible.  If more than 5 ranges are specified, httpd responds with "403 Forbidden".

Request-Range header is always unset, causing the whole file to be returned when client tries to use that header (unless user of Range header makes httpd reject the request).

2   LimitRequestFieldSize 200

Requests with all headers with length up to 200 bytes are processed as before - i.e. httpd responds with "206 Partial Content" and provides only specified file ranges when possible.  If any header is longer than specified limit (it does not have to be Range or Request-Range, e.g. Cookie headers are known to often require more than 200 bytes), httpd rejects request with "400 Bad Request".

3   RequestHeader unset Range 
    RequestHeader unset Request-Range

Header Range and Request-Range are always ignored/unset, httpd always responds with "200 OK" and returns the whole requested file rather than only returning specified ranges.  With this workaround, clients are no longer able to continue / resume interrupted downloads (e.g. wget -c).

Comment 26 xset1980 2011-08-30 22:28:58 UTC
People, Debian has already fixed the problem in all its branches, which runs redhat not yet?. They can use their patches?.

Reference: http://www.debian.org/security/2011/dsa-2298

Comment 27 xset1980 2011-08-31 00:15:02 UTC
Patch of debian:

http://pastie.org/2457283

Comment 28 xset1980 2011-08-31 00:24:53 UTC
Created attachment 520712 [details]
(Incomplete) Patch used by Debian 6 stable, fixed

Comment 29 Tomas Hoger 2011-08-31 07:46:47 UTC
(In reply to comment #27)
> Patch of debian:
> 
> http://pastie.org/2457283

Debian used upstream patch which is mentioned in comment #22.  pastie in comment #27 and attachment in comment #28 both seem incomplete / truncated.  I marked that attachment obsolete.

Comment 30 Phil 2011-08-31 09:55:29 UTC
Yesterday Apache released a pre-test-version of 2.2.20 which contains the fix for CVE-2011-3192.

http://httpd.apache.org/dev/dist/

http://httpd.apache.org/dev/dist/CHANGES_2.2.20


Compiled fine w/ RHEL 6.1 and *seems* to be running without problems for me. Downloads tested using aria2 with 10 concurrent connections.

Anyways, there's a bug report at

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=639825

that 2.2.20 *might* break applications that use partial GETs.

Comment 31 Tomas Hoger 2011-08-31 12:19:51 UTC
Upstream has officially released httpd version 2.2.20:
  http://thread.gmane.org/gmane.comp.apache.devel/45465
  http://httpd.apache.org/security/vulnerabilities_22.html#2.2.20

Comment 32 Phil 2011-08-31 15:57:43 UTC
Apache 2.2.20 shows a different and possibly non-rfc-compliant behaviour as it answers with 200 OK where 206 Partial is expected:

> HEAD / HTTP/1.1
> Host: foo
> Range: bytes=0-

< HTTP/1.1 200 OK
< ...
< Server: Apache
< ...
< Accept-Ranges: bytes

(see http://tools.ietf.org/html/rfc2616#section-14.35.2)

If the client based his decision -- whether or not the server supports Range-Requests -- on the return-code instead of the Accept-Ranges-header the client might break.

Comment 33 errata-xmlrpc 2011-08-31 22:44:42 UTC
This issue has been addressed in following products:

  Red Hat Enterprise Linux 5
  Red Hat Enterprise Linux 6
  Red Hat Enterprise Linux 4

Via RHSA-2011:1245 https://rhn.redhat.com/errata/RHSA-2011-1245.html

Comment 47 Tomas Hoger 2011-09-14 07:19:32 UTC
(In reply to comment #18)
> Upstream has released updated advisory:
> 
> http://lists.grok.org.uk/pipermail/full-disclosure/2011-August/082427.html

Upstream advisory Update 3 / Final is now available:
  http://httpd.apache.org/security/CVE-2011-3192.txt

Following paragraph of the updated describes changes since the previous version:

  2.2.20 has a fix, 2.2.21 an improved one. Version 1.3 is not vulnerable.
  Further regex/rule improvements.  Explained DoS.  Added wiki link.
  Highlight fact that LimitRequestFieldSize workaround was insufficient.

If you've previously deployed mitigation 2 (using LimitRequestFieldSize) and do not yet have updated packages installed, should migrate to using different mitigation.

See comment #25 above for the explanation of what each mitigation does.

Comment 48 errata-xmlrpc 2011-09-14 19:02:04 UTC
This issue has been addressed in following products:

  Red Hat Enterprise Linux 5.6.Z - Server Only
  Red Hat Enterprise Linux 5.3 Long Life
  Red Hat Enterprise Linux 6.0.Z - Server Only

Via RHSA-2011:1294 https://rhn.redhat.com/errata/RHSA-2011-1294.html

Comment 49 errata-xmlrpc 2011-09-15 17:52:53 UTC
This issue has been addressed in following products:

  Red Hat Enterprise Linux 3 Extended Lifecycle Support

Via RHSA-2011:1300 https://rhn.redhat.com/errata/RHSA-2011-1300.html

Comment 52 errata-xmlrpc 2011-09-21 19:59:06 UTC
This issue has been addressed in following products:

  JBEWS 1.0 for RHEL 4
  JBEWS 1.0 for RHEL 5
  JBEWS 1.0 for RHEL 6

Via RHSA-2011:1329 https://rhn.redhat.com/errata/RHSA-2011-1329.html

Comment 53 errata-xmlrpc 2011-09-21 20:09:24 UTC
This issue has been addressed in following products:

  JBoss Enterprise Web Server 1.0.2

Via RHSA-2011:1330 https://rhn.redhat.com/errata/RHSA-2011-1330.html

Comment 55 Sven Hoexter 2011-10-04 12:45:41 UTC
Are there any plans to update the included patch to the version from the Apache 2.2.21 release?

Comment 56 Tomas Hoger 2011-10-04 17:05:45 UTC
The plan is to address all regression introduced by the original upstream fix for this issue in the next httpd update.  Relevant bugs are: bug #736592, bug #736593, and bug #736594.

There's currently no plan to add support for MaxRanges configuration directive.

Comment 57 pmathur 2011-10-05 16:26:37 UTC
Is httpd 2.2.3 running on RHEL 5.3 is also effected by it.

Comment 58 Tomas Hoger 2011-10-06 13:18:15 UTC
As you can read above, 2.0 and 2.2 versions were affected.  Update for RHEL 5.3 version of httpd was made available to customers with RHEL 5.3 Long Life support, see comment #48 above.

Comment 60 errata-xmlrpc 2011-10-13 19:43:55 UTC
This issue has been addressed in following products:

  Red Hat Web Application Stack for RHEL 5

Via RHSA-2011:1369 https://rhn.redhat.com/errata/RHSA-2011-1369.html

Comment 61 xset1980 2011-11-28 10:31:13 UTC
solved in all products

Comment 63 xset1980 2011-12-08 19:46:38 UTC
Oh great reverse engineered, i found this code yesterday!

Comment 64 xset1980 2011-12-08 19:51:28 UTC
@Ramon de C Valle

The variant called "rapache", no cause high cpu usage and memory exhaustion, only freeze the httpd server, tested in rhel6.1 and fedora 16 and both are vulnerable.

Comment 65 xset1980 2011-12-08 20:50:57 UTC
RHEL 6.1 are affected, and the mitigation methods no work, using centos6 or scientific linux 6.1, with the new rapache exploit

Comment 66 Tomas Hoger 2011-12-09 08:24:55 UTC
The mentioned "rapache" exploit does not trigger any bug related to CVE-2011-3192, but only exhausts all MaxClients connections.  More details in bug #761327, comment #11.