An integer overflow in jpc_dec_process_siz was found that can be triggered by crafted image file when given as input to imginfo
Created mingw-jasper tracking bugs for this issue:
Affects: fedora-all [bug 1396987]
Affects: epel-7 [bug 1396989]
Created jasper tracking bugs for this issue:
Affects: fedora-all [bug 1396986]
Affects: epel-5 [bug 1396988]
This issue was originally reported as an assertion failure. Upstream bug report is:
The fix for integer overflow problem in jpc_dec_process_siz() did not directly address the problem, but it did have a side effect of preventing the assertion failure with the original reproducer.
The original issue reporter was able to trigger the same assertion failure even after this fix using different reproducer. That was reported upstream in:
This got a separate CVE-2016-9390 (see bug 1396965).
The integer overflow check added in d91198a was also incomplete - it ensured that the product of dec->numhtiles * dec->numvtiles can fit into size_t type, but later assigned the value to dec->numtiles having a type of int. Assignment form size_t to int can truncate the value, allowing integer overflow to happen. This problem was reported upstream in:
and fixed in:
As previously noted, this integer overflow fix was not a proper fix for the original reported issue. However, it could be an issue of its own, as similar integer overflows prior to memory allocation often lead to heap-based buffer overflows later. That's not the case here, as the allocated dec->tiles array is not accessed beyond the (mis-computed) dec->numtiles.
As the problem of integer overflow in jpc_dec_process_siz() does not seem to have any security impact, it's not currently planned to be addressed in jasper packages in Red Hat Enterprise Linux 6 and 7.
Original reporter's advisory:
Relevant info from the advisory:
imginfo: /tmp/portage/media-libs/jasper-1.900.12/work/jasper-1.900.12/src/libjasper/base/jas_seq.c:90: jas_matrix<= yend' failed.
Re-considering inclusion for easier future testing.
This issue has been addressed in the following products:
Red Hat Enterprise Linux 6
Red Hat Enterprise Linux 7
Via RHSA-2017:1208 https://access.redhat.com/errata/RHSA-2017:1208