Memory allocation failure in jas_malloc triggered by crafted file was found. CVE assignment: http://seclists.org/oss-sec/2016/q4/214
Original reporter's advisory: https://blogs.gentoo.org/ago/2016/10/18/jasper-memory-allocation-failure-in-jas_malloc-jas_malloc-c/ Relevant information form the advisory: Another round of fuzzing on an updated version (1.900.5) revealed a memory allocation failure. The complete ASan output: # imginfo -f $FILE THE BMP FORMAT IS NOT FULLY SUPPORTED! THAT IS, THE JASPER SOFTWARE CANNOT DECODE ALL TYPES OF BMP DATA. IF YOU HAVE ANY PROBLEMS, PLEASE TRY CONVERTING YOUR IMAGE DATA TO THE PNM FORMAT, AND USING THIS FORMAT INSTEAD. ==18943==ERROR: AddressSanitizer failed to allocate 0x1000002000 (68719484928) bytes of LargeMmapAllocator (error code: 12) ==18943==Process memory map follows: 0x000000400000-0x000000520000 /usr/bin/imginfo 0x00000071f000-0x000000720000 /usr/bin/imginfo 0x000000720000-0x000000724000 /usr/bin/imginfo 0x000000724000-0x0000013a6000 0x00007fff7000-0x00008fff7000 0x00008fff7000-0x02008fff7000 0x02008fff7000-0x10007fff8000 0x600000000000-0x602000000000 0x602000000000-0x602000010000 0x602000010000-0x603000000000 0x603000000000-0x603000010000 0x603000010000-0x604000000000 0x604000000000-0x604000010000 0x604000010000-0x606000000000 0x606000000000-0x606000010000 0x606000010000-0x60b000000000 0x60b000000000-0x60b000010000 0x60b000010000-0x619000000000 0x619000000000-0x619000020000 0x619000020000-0x625000000000 0x625000000000-0x625000020000 0x625000020000-0x640000000000 0x640000000000-0x640000003000 0x7f4f00738000-0x7f4f03593000 0x7f4f03593000-0x7f4f035fc000 /usr/lib64/libjpeg.so.62.2.0 0x7f4f035fc000-0x7f4f037fb000 /usr/lib64/libjpeg.so.62.2.0 0x7f4f037fb000-0x7f4f037fc000 /usr/lib64/libjpeg.so.62.2.0 0x7f4f037fc000-0x7f4f037fd000 /usr/lib64/libjpeg.so.62.2.0 0x7f4f037fd000-0x7f4f03990000 /lib64/libc-2.22.so 0x7f4f03990000-0x7f4f03b90000 /lib64/libc-2.22.so 0x7f4f03b90000-0x7f4f03b94000 /lib64/libc-2.22.so 0x7f4f03b94000-0x7f4f03b96000 /lib64/libc-2.22.so 0x7f4f03b96000-0x7f4f03b9a000 0x7f4f03b9a000-0x7f4f03bb0000 /usr/lib64/gcc/x86_64-pc-linux-gnu/4.9.3/libgcc_s.so.1 0x7f4f03bb0000-0x7f4f03daf000 /usr/lib64/gcc/x86_64-pc-linux-gnu/4.9.3/libgcc_s.so.1 0x7f4f03daf000-0x7f4f03db0000 /usr/lib64/gcc/x86_64-pc-linux-gnu/4.9.3/libgcc_s.so.1 0x7f4f03db0000-0x7f4f03db1000 /usr/lib64/gcc/x86_64-pc-linux-gnu/4.9.3/libgcc_s.so.1 0x7f4f03db1000-0x7f4f03db3000 /lib64/libdl-2.22.so 0x7f4f03db3000-0x7f4f03fb3000 /lib64/libdl-2.22.so 0x7f4f03fb3000-0x7f4f03fb4000 /lib64/libdl-2.22.so 0x7f4f03fb4000-0x7f4f03fb5000 /lib64/libdl-2.22.so 0x7f4f03fb5000-0x7f4f03fbb000 /lib64/librt-2.22.so 0x7f4f03fbb000-0x7f4f041bb000 /lib64/librt-2.22.so 0x7f4f041bb000-0x7f4f041bc000 /lib64/librt-2.22.so 0x7f4f041bc000-0x7f4f041bd000 /lib64/librt-2.22.so 0x7f4f041bd000-0x7f4f041d4000 /lib64/libpthread-2.22.so 0x7f4f041d4000-0x7f4f043d3000 /lib64/libpthread-2.22.so 0x7f4f043d3000-0x7f4f043d4000 /lib64/libpthread-2.22.so 0x7f4f043d4000-0x7f4f043d5000 /lib64/libpthread-2.22.so 0x7f4f043d5000-0x7f4f043d9000 0x7f4f043d9000-0x7f4f044d6000 /lib64/libm-2.22.so 0x7f4f044d6000-0x7f4f046d5000 /lib64/libm-2.22.so 0x7f4f046d5000-0x7f4f046d6000 /lib64/libm-2.22.so 0x7f4f046d6000-0x7f4f046d7000 /lib64/libm-2.22.so 0x7f4f046d7000-0x7f4f04891000 /usr/lib64/libjasper.so.1.0.0 0x7f4f04891000-0x7f4f04a90000 /usr/lib64/libjasper.so.1.0.0 0x7f4f04a90000-0x7f4f04a94000 /usr/lib64/libjasper.so.1.0.0 0x7f4f04a94000-0x7f4f04aa3000 /usr/lib64/libjasper.so.1.0.0 0x7f4f04aa3000-0x7f4f04aac000 0x7f4f04aac000-0x7f4f04ace000 /lib64/ld-2.22.so 0x7f4f04c67000-0x7f4f04cc2000 0x7f4f04cc2000-0x7f4f04ccd000 0x7f4f04ccd000-0x7f4f04cce000 /lib64/ld-2.22.so 0x7f4f04cce000-0x7f4f04ccf000 /lib64/ld-2.22.so 0x7f4f04ccf000-0x7f4f04cd0000 0x7ffeaeaca000-0x7ffeaeaeb000 [stack] 0x7ffeaeb8a000-0x7ffeaeb8c000 [vvar] 0x7ffeaeb8c000-0x7ffeaeb8e000 [vdso] 0xffffffffff600000-0xffffffffff601000 [vsyscall] ==18943==End of process memory map. ==18943==AddressSanitizer CHECK failed: /var/tmp/portage/sys-devel/llvm-3.8.1-r2/work/llvm-3.8.1.src/projects/compiler-rt/lib/sanitizer_common/sanitizer_common.cc:183 "((0 && "unable to mmap")) != (0)" (0x0, 0x0) #0 0x4c9ccd in AsanCheckFailed /var/tmp/portage/sys-devel/llvm-3.8.1-r2/work/llvm-3.8.1.src/projects/compiler-rt/lib/asan/asan_rtl.cc:67 #1 0x4d0803 in __sanitizer::CheckFailed(char const*, int, char const*, unsigned long long, unsigned long long) /var/tmp/portage/sys-devel/llvm-3.8.1-r2/work/llvm-3.8.1.src/projects/compiler-rt/lib/sanitizer_common/sanitizer_common.cc:159 #2 0x4d09f1 in __sanitizer::ReportMmapFailureAndDie(unsigned long, char const*, char const*, int, bool) /var/tmp/portage/sys-devel/llvm-3.8.1-r2/work/llvm-3.8.1.src/projects/compiler-rt/lib/sanitizer_common/sanitizer_common.cc:183 #3 0x4d9a2a in __sanitizer::MmapOrDie(unsigned long, char const*, bool) /var/tmp/portage/sys-devel/llvm-3.8.1-r2/work/llvm-3.8.1.src/projects/compiler-rt/lib/sanitizer_common/sanitizer_posix.cc:122 #4 0x421dbf in __sanitizer::LargeMmapAllocator::Allocate(__sanitizer::AllocatorStats*, unsigned long, unsigned long) /var/tmp/portage/sys-devel/llvm-3.8.1-r2/work/llvm-3.8.1.src/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_allocator.h:1033 #5 0x421dbf in __sanitizer::CombinedAllocator<__sanitizer::SizeClassAllocator64<105553116266496ul, 4398046511104ul, 0ul, __sanitizer::SizeClassMap, __asan::AsanMapUnmapCallback>, __sanitizer::SizeClassAllocatorLocalCache<__sanitizer::SizeClassAllocator64<105553116266496ul, 4398046511104ul, 0ul, __sanitizer::SizeClassMap, __asan::AsanMapUnmapCallback> >, __sanitizer::LargeMmapAllocator >::Allocate(__sanitizer::SizeClassAllocatorLocalCache<__sanitizer::SizeClassAllocator64<105553116266496ul, 4398046511104ul, 0ul, __sanitizer::SizeClassMap, __asan::AsanMapUnmapCallback> >*, unsigned long, unsigned long, bool, bool) /var/tmp/portage/sys-devel/llvm-3.8.1-r2/work/llvm-3.8.1.src/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_allocator.h:1302 #6 0x421dbf in __asan::Allocator::Allocate(unsigned long, unsigned long, __sanitizer::BufferedStackTrace*, __asan::AllocType, bool) /var/tmp/portage/sys-devel/llvm-3.8.1-r2/work/llvm-3.8.1.src/projects/compiler-rt/lib/asan/asan_allocator.cc:368 #7 0x421dbf in __asan::asan_malloc(unsigned long, __sanitizer::BufferedStackTrace*) /var/tmp/portage/sys-devel/llvm-3.8.1-r2/work/llvm-3.8.1.src/projects/compiler-rt/lib/asan/asan_allocator.cc:718 #8 0x4c0391 in malloc /var/tmp/portage/sys-devel/llvm-3.8.1-r2/work/llvm-3.8.1.src/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:53 #9 0x7f4f0474e170 in jas_malloc /tmp/portage/media-libs/jasper-1.900.5/work/jasper-1.900.5/src/libjasper/base/jas_malloc.c:117:9 #10 0x7f4f0474e170 in jas_alloc2 /tmp/portage/media-libs/jasper-1.900.5/work/jasper-1.900.5/src/libjasper/base/jas_malloc.c:141 #11 0x7f4f04764b4f in bmp_getinfo /tmp/portage/media-libs/jasper-1.900.5/work/jasper-1.900.5/src/libjasper/bmp/bmp_dec.c:297:25 #12 0x7f4f04764b4f in bmp_decode /tmp/portage/media-libs/jasper-1.900.5/work/jasper-1.900.5/src/libjasper/bmp/bmp_dec.c:132 #13 0x7f4f0470ef39 in jas_image_decode /tmp/portage/media-libs/jasper-1.900.5/work/jasper-1.900.5/src/libjasper/base/jas_image.c:380:16 #14 0x4f1686 in main /tmp/portage/media-libs/jasper-1.900.5/work/jasper-1.900.5/src/appl/imginfo.c:188:16 #15 0x7f4f0381d61f in __libc_start_main /var/tmp/portage/sys-libs/glibc-2.22-r4/work/glibc-2.22/csu/libc-start.c:289 #16 0x418e68 in _init (/usr/bin/imginfo+0x418e68) Affected version: 1.900.5 Fixed version: 1.900.11 Commit fix: https://github.com/mdadams/jasper/commit/65536647d380571d1a9a6c91fa03775fb5bbd256
Upstream bug report: https://github.com/mdadams/jasper/issues/35 There is a mix of two issues here. The test case provided by the reporter causes jasper to attempt to allocate large amount of memory. That attempt fails on most systems, and the failure is handled gracefully. However, when jasper is built with ASAN, ASAN terminates the process instead of allowing jasper to handle the allocation error. This is not a problem for non-ASAN builds, and not something that should be called security. Another concern is that jasper does not impose any upper limit on the amount of memory it allocates. Smaller allocations than those triggered by the provided reproducer may still succeed and cause jasper to consume an excessive amount of memory. Jasper upstream indicated they do not plan to introduce any arbitrary limit that would likely be unsuitable for certain use cases. The fix introduced in version 1.900.11 via the commit linked above is addition of an experimental memory allocator that tracks total memory allocations done by jasper, and new API allowing applications to set the limit. Command line utilities were updated to use the limit if jasper is compiled with this allocator enabled. Upstream clearly marks this feature as experimental and notes it may be removed in future versions. There's currently no plan to backport this fix to released Red Hat Enterprise Linux versions.