Bug 1112285 (CVE-2014-4616) - CVE-2014-4616 python: missing boundary check in JSON module
Summary: CVE-2014-4616 python: missing boundary check in JSON module
Keywords:
Status: CLOSED ERRATA
Alias: CVE-2014-4616
Product: Security Response
Classification: Other
Component: vulnerability
Version: unspecified
Hardware: All
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Red Hat Product Security
QA Contact:
URL:
Whiteboard: impact=moderate,public=20140519,repor...
Depends On: 1112293 1112294 1112903 1115517 1115518 1115519 1187779 1206574 1222534
Blocks: 1112298 1112724 1210268
TreeView+ depends on / blocked
 
Reported: 2014-06-23 13:47 UTC by Vasyl Kaigorodov
Modified: 2019-06-08 20:05 UTC (History)
45 users (show)

Fixed In Version: python 2.7.7, python 3.3.6, python 3.4.1
Doc Type: Bug Fix
Doc Text:
A flaw was found in the way the json module handled negative index argument passed to certain functions (such as raw_decode()). An attacker able to control index value passed to one of the affected functions could possibly use this flaw to disclose portions of the application memory.
Clone Of:
Environment:
Last Closed: 2016-11-03 21:09:24 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2015:1064 normal SHIPPED_LIVE Moderate: python27 security, bug fix, and enhancement update 2015-06-04 12:28:00 UTC
Red Hat Product Errata RHSA-2015:2101 normal SHIPPED_LIVE Moderate: python security, bug fix, and enhancement update 2015-11-19 11:04:15 UTC

Description Vasyl Kaigorodov 2014-06-23 13:47:14 UTC
It was reported [1] that Python built-in _json module have a flaw (insufficient bounds checking), which allows a local user to read current process' arbitrary memory.

Quoting the upstream bug report:
...
The sole prerequisites of this attack are that the attacker is able to
control or influence the two parameters of the default scanstring
function: the string to be decoded and the index.

The bug is caused by allowing the user to supply a negative index
value. The index value is then used directly as an index to an array
in the C code; internally the address of the array and its index are
added to each other in order to yield the address of the value that is
desired. However, by supplying a negative index value and adding this
to the address of the array, the processor's register value wraps
around and the calculated value will point to a position in memory
which isn't within the bounds of the supplied string, causing the
function to access other parts of the process memory.
...

Upstream bug report with additional technical details:
[1] http://bugs.python.org/issue21529

Upstream commits (taken from upstream tracker):

2.7: http://hg.python.org/cpython/rev/50c07ed1743d
3.1: http://hg.python.org/cpython/rev/a8facac493ef
3.2: http://hg.python.org/cpython/rev/8130b8c06606
3.3: http://hg.python.org/cpython/rev/4f15bd1ab28f
3.4: http://hg.python.org/cpython/rev/7b95540ced5c
3.5: http://hg.python.org/cpython/rev/3a414c709f1f

Comment 1 Vasyl Kaigorodov 2014-06-23 14:02:53 UTC
Created python tracking bugs for this issue:

Affects: fedora-all [bug 1112293]

Comment 2 Vasyl Kaigorodov 2014-06-23 14:02:57 UTC
Created python3 tracking bugs for this issue:

Affects: fedora-all [bug 1112294]

Comment 3 Tomas Hoger 2014-06-23 14:28:16 UTC
HackerOne report:
https://hackerone.com/reports/12297

Comment 4 Vincent Danen 2014-06-24 13:37:43 UTC
This was assigned CVE-2014-4616: http://openwall.com/lists/oss-security/2014/06/24/7

Comment 5 Tomas Hoger 2014-06-24 14:38:51 UTC
The original report suggests that this is a high severity issue because of "ubiquity of JSON parsing in Python applications".  However, practical impact seems rather limited.  To trigger the issue, negative value is passed as idx argument of raw_decode().  However, this argument is not documented in the Python standard library documentation, which currently says:

  raw_decode(s)

    Decode a JSON document from s (a str or unicode beginning with a JSON
    document) and return a 2-tuple of the Python representation and the index
    in s where the document ended.

    This can be used to decode a JSON document from a string that may have
    extraneous data at the end.

  https://docs.python.org/2/library/json.html#json.JSONDecoder.raw_decode
  https://docs.python.org/3/library/json.html#json.JSONDecoder.raw_decode

Therefore, the idx argument for the function is rarely used.  I've not been able to identify any application that could pass arbitrary negative value as idx.  The only use of the idx argument uses the following pattern from decode(), which is used to strip leading spaces from the input string:

  WHITESPACE = re.compile(r'\s*', FLAGS)
  _w=WHITESPACE.match
  raw_decode(s, idx=_w(s, 0).end())

This does not allow passing arbitrary negative index value to raw_decode().

It should be noted that the json module that is part of Python standard library is based on simplejson, currently using version 2.0.9.  The json module was added to the Python standard library in version 2.6, therefore python packages in Red Hat Enterprise Linux 5 (using python 2.4.3) and older are not affected by this issue.  The python packages in Red Hat Enterprise Linux 6 (using python 2.6.6) use json module based on simplejson 1.9.  The code affected by this issue was only introduced later, and added to Python standard library as part of this rebase from 1.9 to 2.0.9:

http://hg.python.org/cpython/rev/52199
http://bugs.python.org/issue4136

Therefore python packages in Red Hat Enterprise Linux 6 are not affected.  There are also python-simplejson packages shipped in Red Hat Enterprise Linux 5 and 6 (version 2.0.9), which are affected by this issue.

Recent upstream versions of simplejson (such as the version used in Red Hat Software Collections 1) also mitigate this issue by not allowing arbitrary negative index in raw_decode().  Starting with version 2.6.1 (based on upstream git repository tags), raw_decode() also performs stripping of leading spaces, which deals with negative index as a side effect:

https://github.com/simplejson/simplejson/commit/0fb0aea

However, it is possible to reproduce the issue by calling (undocumented) scan_once() function directly.  This problem was reported to simplejson upstream via:

https://github.com/simplejson/simplejson/issues/98

For the sake of completeness, API documentation of current upstream simplejson versions document idx argument for the raw_decode() function:

  raw_decode(s[, idx=0])

    Decode a JSON document from s (a str or unicode beginning with a JSON
    document) starting from the index idx and return a 2-tuple of the Python
    representation and the index in s where the document ended.

    This can be used to decode a JSON document from a string that may have
    extraneous data at the end, or to decode a string that has a series of
    JSON objects.

    JSONDecodeError will be raised if the given JSON document is not valid.

  http://simplejson.readthedocs.org/en/latest/#simplejson.JSONDecoder.raw_decode

Comment 7 Tomas Hoger 2014-06-24 19:21:08 UTC
(In reply to Tomas Hoger from comment #5)
> However, it is possible to reproduce the issue by calling (undocumented)
> scan_once() function directly.  This problem was reported to simplejson
> upstream via:
> 
> https://github.com/simplejson/simplejson/issues/98

simplejson fixed upstream in version 3.5.3 via the following commit:

https://github.com/simplejson/simplejson/commit/b7486b8

Comment 8 Luke Macken 2014-06-24 23:05:44 UTC
(In reply to Tomas Hoger from comment #7)
> (In reply to Tomas Hoger from comment #5)
> > However, it is possible to reproduce the issue by calling (undocumented)
> > scan_once() function directly.  This problem was reported to simplejson
> > upstream via:
> > 
> > https://github.com/simplejson/simplejson/issues/98
> 
> simplejson fixed upstream in version 3.5.3 via the following commit:
> 
> https://github.com/simplejson/simplejson/commit/b7486b8

I just built python-simplejson-3.5.3 and queued it up for testing. https://admin.fedoraproject.org/updates/python-simplejson-3.5.3-1.fc20

Comment 11 Vincent Danen 2014-06-27 14:17:47 UTC
This has been fixed in Fedora for python-simplejson:

https://lists.fedoraproject.org/pipermail/package-announce/2014-June/134761.html

Comment 12 Fedora Update System 2014-06-30 10:31:42 UTC
python-2.7.5-13.fc20 has been pushed to the Fedora 20 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 14 Tomas Hoger 2014-07-02 13:45:28 UTC
Statement:

This issue affects the versions of python as shipped with Red Hat Enterprise Linux 7, the versions of python-simplejson as shipped with Red Hat Enterprise Linux 5 and 6, and the versions of python33-python and python33-python-simplejson as shipped with Red Hat Software Collections. Red Hat Product Security has rated this issue as having Moderate security impact. Future updates may address this issue. For additional information, refer to the Issue Severity Classification: https://access.redhat.com/security/updates/classification/.

Comment 15 Tomas Hoger 2014-07-02 13:59:37 UTC
Created python26-simplejson tracking bugs for this issue:

Affects: epel-5 [bug 1115518]

Comment 16 Tomas Hoger 2014-07-02 13:59:42 UTC
Created python-simplejson tracking bugs for this issue:

Affects: fedora-all [bug 1115517]
Affects: epel-7 [bug 1115519]

Comment 17 Fedora Update System 2014-07-17 04:28:11 UTC
python3-3.3.2-9.fc19 has been pushed to the Fedora 19 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 18 Fedora Update System 2014-07-17 04:29:28 UTC
python-2.7.5-13.fc19 has been pushed to the Fedora 19 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 19 Fedora Update System 2014-11-09 15:45:14 UTC
python3-3.3.2-18.fc20 has been pushed to the Fedora 20 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 20 Fedora Update System 2014-11-10 06:36:22 UTC
python3-3.4.1-16.fc21 has been pushed to the Fedora 21 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 21 Fedora Update System 2014-11-13 18:14:22 UTC
python3-3.3.2-10.fc19 has been pushed to the Fedora 19 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 25 errata-xmlrpc 2015-06-04 08:29:54 UTC
This issue has been addressed in the following products:

  Red Hat Software Collections for Red Hat Enterprise Linux 7
  Red Hat Software Collections for Red Hat Enterprise Linux 6
  Red Hat Software Collections for Red Hat Enterprise Linux 6.5 EUS
  Red Hat Software Collections for Red Hat Enterprise Linux 6.6 EUS

Via RHSA-2015:1064 https://rhn.redhat.com/errata/RHSA-2015-1064.html

Comment 26 errata-xmlrpc 2015-11-19 12:42:43 UTC
This issue has been addressed in the following products:

  Red Hat Enterprise Linux 7

Via RHSA-2015:2101 https://rhn.redhat.com/errata/RHSA-2015-2101.html


Note You need to log in before you can comment on or make changes to this bug.