A flaw was found in OpenJPEG 2.3.0, there is an integer overflow vulnerability in the opj_t1_encode_cblks function in openjp2/t1.c file. Remote attackers could leverage this vulnerability to cause a denial of service via a crafted bmp file. References: https://github.com/uclouvain/openjpeg/issues/1053
Created openjpeg tracking bugs for this issue: Affects: epel-all [bug 1536554] Affects: fedora-all [bug 1536553]
Analysis: I was able to reproduce this by using openjpeg compiled with -fsanitize=undefined. The following error message is displayed when the binary aborts: [huzaifas@babylon bin]$ ./opj_compress -n 1 -i /tmp/openjpeg_2-3_opj_compress_integer-overflow_opj_t1_encode_cblks.bmp -o /tmp/null.j2k [INFO] tile number 1 / 1 /NotBackedUp/oss/openjpeg/src/lib/openjp2/t1.c:2173:55: runtime error: signed integer overflow: 322385710 * 64 cannot be represented in type 'int' [ERROR] opj_t2_encode_packet(): only 22 bytes remaining in output buffer. 359 needed. [ERROR] Cannot encode tile failed to encode image: opj_encode failed to encode image: opj_end_compress failed to encode image which points to: 2173 tiledp[tileIndex] *= (1 << T1_NMSEDEC_FRACBITS); tiledp is an array of ints and the above operation causes an overflow. However it seems that insufficient data is caught correctly by following code and an ERROR message is displayed. The binary does not crash but seems to abort elegantly. So its likely that the only result of the flaw is that the image file is not rendered/converted by openjpeg.
This issue has been addressed in the following products: Red Hat Enterprise Linux 8 Via RHSA-2021:4251 https://access.redhat.com/errata/RHSA-2021:4251