Bug 1537758 (CVE-2018-5785)

Summary:	CVE-2018-5785 openjpeg: integer overflow in opj_j2k_setup_encoder function in openjp2/j2k.c
Product:	[Other] Security Response	Reporter:	Laura Pardo <lpardo>
Component:	vulnerability	Assignee:	Red Hat Product Security <security-response-team>
Status:	CLOSED WONTFIX	QA Contact:
Severity:	low	Docs Contact:
Priority:	low
Version:	unspecified	CC:	hobbes1069, jaromir.capik, manisandro, nforro, rdieter, rhel8-maint
Target Milestone:	---	Keywords:	Security
Target Release:	---
Hardware:	All
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2019-06-08 03:38:04 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1537759, 1537760, 1537761, 1537762, 1538556
Bug Blocks:	1537764

Description Laura Pardo 2018-01-23 19:56:35 UTC

A flaw was found in OpenJPEG 2.3.0, there is an integer overflow caused by an out-of-bounds left shift in the opj_j2k_setup_encoder function (openjp2/j2k.c). Remote attackers could leverage this vulnerability to cause a denial of service via a crafted bmp file.

Reference:
https://github.com/uclouvain/openjpeg/issues/1057

Comment 1 Laura Pardo 2018-01-23 19:57:03 UTC

Created mingw-openjpeg2 tracking bugs for this issue:

Affects: fedora-all [bug 1537761]


Created openjpeg tracking bugs for this issue:

Affects: epel-all [bug 1537762]
Affects: fedora-all [bug 1537759]


Created openjpeg2 tracking bugs for this issue:

Affects: fedora-all [bug 1537760]

Comment 2 Huzaifa S. Sidhpurwala 2018-01-25 09:56:47 UTC

Analysis:

Running openjpeg compiled with UBSAN, i observe the following:

[huzaifas@babylon bin]$ ./opj_compress -n 1 -i /tmp/openjpeg_2-3_opj_compress_integer-overflow_opj_j2k_setup_encoder.bmp -o /tmp/null.j2k

/NotBackedUp/oss/openjpeg/src/lib/openjp2/j2k.c:7304:48: runtime error: shift exponent 4294967295 is too large for 32-bit type 'int'
[INFO] tile number 1 / 1
/NotBackedUp/oss/openjpeg/src/lib/openjp2/tcd.c:2382:32: runtime error: signed integer overflow: 0 - -2147483648 cannot be represented in type 'int [33]'
/NotBackedUp/oss/openjpeg/src/lib/openjp2/mct.c:109:31: runtime error: signed integer overflow: -2147483648 * 2 cannot be represented in type 'int'
/NotBackedUp/oss/openjpeg/src/lib/openjp2/mct.c:109:36: runtime error: signed integer overflow: -2147483648 + -2147483648 cannot be represented in type 'int'
[INFO] Generated outfile /tmp/null.j2k
encode time: 1 ms 


Which represents multiple integer overflow when converting from BMP to J2K format. However it seems that in the end the operation ends gracefully. There are no related buffer overflows etc.

The maximum security impact seems to be limited to invalid conversion to J2K format.

Comment 4 errata-xmlrpc 2021-11-09 17:55:54 UTC

This issue has been addressed in the following products:

  Red Hat Enterprise Linux 8

Via RHSA-2021:4251 https://access.redhat.com/errata/RHSA-2021:4251