Bug 1388880 (CVE-2016-8886)

Summary: CVE-2016-8886 jasper: no upper limit on memory allocations in jas_malloc()
Product: [Other] Security Response Reporter: Adam Mariš <amaris>
Component: vulnerabilityAssignee: Red Hat Product Security <security-response-team>
Status: CLOSED WONTFIX QA Contact:
Severity: low Docs Contact:
Priority: low    
Version: unspecifiedCC: abhgupta, bmcclain, cfergeau, dblechte, dmcphers, eedri, erik-fedora, gklein, jialiu, jokerman, jridky, kseifried, lmeyer, lsurette, mgoldboi, michal.skrivanek, mike, mmccomas, rbalakri, rdieter, rh-spice-bugs, rjones, sherold, srevivo, tiwillia, ykaul, ylavi
Target Milestone: ---Keywords: Security
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: jasper 1.900.11 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-03-29 19:38:05 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1388873, 1388874, 1388875, 1388876    
Bug Blocks: 1314477    

Description Adam Mariš 2016-10-26 10:57:35 UTC
Memory allocation failure in jas_malloc triggered by crafted file was found.

CVE assignment:

http://seclists.org/oss-sec/2016/q4/214

Comment 1 Tomas Hoger 2017-03-29 19:17:09 UTC
Original reporter's advisory:

https://blogs.gentoo.org/ago/2016/10/18/jasper-memory-allocation-failure-in-jas_malloc-jas_malloc-c/

Relevant information form the advisory:

Another round of fuzzing on an updated version (1.900.5) revealed a memory allocation failure.

The complete ASan output:

# imginfo -f $FILE
THE BMP FORMAT IS NOT FULLY SUPPORTED!                                                                                                                                                                                                                                         
THAT IS, THE JASPER SOFTWARE CANNOT DECODE ALL TYPES OF BMP DATA.                                                                                                                                                                                                              
IF YOU HAVE ANY PROBLEMS, PLEASE TRY CONVERTING YOUR IMAGE DATA                                                                                                                                                                                                                
TO THE PNM FORMAT, AND USING THIS FORMAT INSTEAD.                                                                                                                                                                                                                              
==18943==ERROR: AddressSanitizer failed to allocate 0x1000002000 (68719484928) bytes of LargeMmapAllocator (error code: 12)                                                                                                                                                    
==18943==Process memory map follows:                                                                                                                                                                                                                                           
        0x000000400000-0x000000520000   /usr/bin/imginfo                                                                                                                                                                                                                       
        0x00000071f000-0x000000720000   /usr/bin/imginfo                                                                                                                                                                                                                       
        0x000000720000-0x000000724000   /usr/bin/imginfo                                                                                                                                                                                                                       
        0x000000724000-0x0000013a6000                                                                                                                                                                                                                                          
        0x00007fff7000-0x00008fff7000                                                                                                                                                                                                                                          
        0x00008fff7000-0x02008fff7000                                                                                                                                                                                                                                          
        0x02008fff7000-0x10007fff8000                                                                                                                                                                                                                                          
        0x600000000000-0x602000000000                                                                                                                                                                                                                                          
        0x602000000000-0x602000010000                                                                                                                                                                                                                                          
        0x602000010000-0x603000000000                                                                                                                                                                                                                                          
        0x603000000000-0x603000010000                                                                                                                                                                                                                                          
        0x603000010000-0x604000000000                                                                                                                                                                                                                                          
        0x604000000000-0x604000010000                                                                                                                                                                                                                                          
        0x604000010000-0x606000000000                                                                                                                                                                                                                                          
        0x606000000000-0x606000010000                                                                                                                                                                                                                                          
        0x606000010000-0x60b000000000                                                                                                                                                                                                                                          
        0x60b000000000-0x60b000010000                                                                                                                                                                                                                                          
        0x60b000010000-0x619000000000                                                                                                                                                                                                                                          
        0x619000000000-0x619000020000                                                                                                                                                                                                                                          
        0x619000020000-0x625000000000                                                                                                                                                                                                                                          
        0x625000000000-0x625000020000                                                                                                                                                                                                                                          
        0x625000020000-0x640000000000                                                                                                                                                                                                                                          
        0x640000000000-0x640000003000                                                                                                                                                                                                                                          
        0x7f4f00738000-0x7f4f03593000                                                                                                                                                                                                                                          
        0x7f4f03593000-0x7f4f035fc000   /usr/lib64/libjpeg.so.62.2.0                                                                                                                                                                                                           
        0x7f4f035fc000-0x7f4f037fb000   /usr/lib64/libjpeg.so.62.2.0                                                                                                                                                                                                           
        0x7f4f037fb000-0x7f4f037fc000   /usr/lib64/libjpeg.so.62.2.0                                                                                                                                                                                                           
        0x7f4f037fc000-0x7f4f037fd000   /usr/lib64/libjpeg.so.62.2.0                                                                                                                                                                                                           
        0x7f4f037fd000-0x7f4f03990000   /lib64/libc-2.22.so                                                                                                                                                                                                                    
        0x7f4f03990000-0x7f4f03b90000   /lib64/libc-2.22.so                                                                                                                                                                                                                    
        0x7f4f03b90000-0x7f4f03b94000   /lib64/libc-2.22.so                                                                                                                                                                                                                    
        0x7f4f03b94000-0x7f4f03b96000   /lib64/libc-2.22.so                                                                                                                                                                                                                    
        0x7f4f03b96000-0x7f4f03b9a000                                                                                                                                                                                                                                          
        0x7f4f03b9a000-0x7f4f03bb0000   /usr/lib64/gcc/x86_64-pc-linux-gnu/4.9.3/libgcc_s.so.1                                                                                                                                                                                 
        0x7f4f03bb0000-0x7f4f03daf000   /usr/lib64/gcc/x86_64-pc-linux-gnu/4.9.3/libgcc_s.so.1                                                                                                                                                                                 
        0x7f4f03daf000-0x7f4f03db0000   /usr/lib64/gcc/x86_64-pc-linux-gnu/4.9.3/libgcc_s.so.1                                                                                                                                                                                 
        0x7f4f03db0000-0x7f4f03db1000   /usr/lib64/gcc/x86_64-pc-linux-gnu/4.9.3/libgcc_s.so.1                                                                                                                                                                                 
        0x7f4f03db1000-0x7f4f03db3000   /lib64/libdl-2.22.so
        0x7f4f03db3000-0x7f4f03fb3000   /lib64/libdl-2.22.so
        0x7f4f03fb3000-0x7f4f03fb4000   /lib64/libdl-2.22.so
        0x7f4f03fb4000-0x7f4f03fb5000   /lib64/libdl-2.22.so
        0x7f4f03fb5000-0x7f4f03fbb000   /lib64/librt-2.22.so
        0x7f4f03fbb000-0x7f4f041bb000   /lib64/librt-2.22.so
        0x7f4f041bb000-0x7f4f041bc000   /lib64/librt-2.22.so
        0x7f4f041bc000-0x7f4f041bd000   /lib64/librt-2.22.so
        0x7f4f041bd000-0x7f4f041d4000   /lib64/libpthread-2.22.so
        0x7f4f041d4000-0x7f4f043d3000   /lib64/libpthread-2.22.so
        0x7f4f043d3000-0x7f4f043d4000   /lib64/libpthread-2.22.so
        0x7f4f043d4000-0x7f4f043d5000   /lib64/libpthread-2.22.so
        0x7f4f043d5000-0x7f4f043d9000
        0x7f4f043d9000-0x7f4f044d6000   /lib64/libm-2.22.so
        0x7f4f044d6000-0x7f4f046d5000   /lib64/libm-2.22.so
        0x7f4f046d5000-0x7f4f046d6000   /lib64/libm-2.22.so
        0x7f4f046d6000-0x7f4f046d7000   /lib64/libm-2.22.so
        0x7f4f046d7000-0x7f4f04891000   /usr/lib64/libjasper.so.1.0.0
        0x7f4f04891000-0x7f4f04a90000   /usr/lib64/libjasper.so.1.0.0
        0x7f4f04a90000-0x7f4f04a94000   /usr/lib64/libjasper.so.1.0.0
        0x7f4f04a94000-0x7f4f04aa3000   /usr/lib64/libjasper.so.1.0.0
        0x7f4f04aa3000-0x7f4f04aac000
        0x7f4f04aac000-0x7f4f04ace000   /lib64/ld-2.22.so
        0x7f4f04c67000-0x7f4f04cc2000
        0x7f4f04cc2000-0x7f4f04ccd000
        0x7f4f04ccd000-0x7f4f04cce000   /lib64/ld-2.22.so
        0x7f4f04cce000-0x7f4f04ccf000   /lib64/ld-2.22.so
        0x7f4f04ccf000-0x7f4f04cd0000
        0x7ffeaeaca000-0x7ffeaeaeb000   [stack]
        0x7ffeaeb8a000-0x7ffeaeb8c000   [vvar]
        0x7ffeaeb8c000-0x7ffeaeb8e000   [vdso]
        0xffffffffff600000-0xffffffffff601000   [vsyscall]
==18943==End of process memory map.
==18943==AddressSanitizer CHECK failed: /var/tmp/portage/sys-devel/llvm-3.8.1-r2/work/llvm-3.8.1.src/projects/compiler-rt/lib/sanitizer_common/sanitizer_common.cc:183 "((0 && "unable to mmap")) != (0)" (0x0, 0x0)
    #0 0x4c9ccd in AsanCheckFailed /var/tmp/portage/sys-devel/llvm-3.8.1-r2/work/llvm-3.8.1.src/projects/compiler-rt/lib/asan/asan_rtl.cc:67
    #1 0x4d0803 in __sanitizer::CheckFailed(char const*, int, char const*, unsigned long long, unsigned long long) /var/tmp/portage/sys-devel/llvm-3.8.1-r2/work/llvm-3.8.1.src/projects/compiler-rt/lib/sanitizer_common/sanitizer_common.cc:159
    #2 0x4d09f1 in __sanitizer::ReportMmapFailureAndDie(unsigned long, char const*, char const*, int, bool) /var/tmp/portage/sys-devel/llvm-3.8.1-r2/work/llvm-3.8.1.src/projects/compiler-rt/lib/sanitizer_common/sanitizer_common.cc:183
    #3 0x4d9a2a in __sanitizer::MmapOrDie(unsigned long, char const*, bool) /var/tmp/portage/sys-devel/llvm-3.8.1-r2/work/llvm-3.8.1.src/projects/compiler-rt/lib/sanitizer_common/sanitizer_posix.cc:122
    #4 0x421dbf in __sanitizer::LargeMmapAllocator::Allocate(__sanitizer::AllocatorStats*, unsigned long, unsigned long) /var/tmp/portage/sys-devel/llvm-3.8.1-r2/work/llvm-3.8.1.src/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_allocator.h:1033
    #5 0x421dbf in __sanitizer::CombinedAllocator<__sanitizer::SizeClassAllocator64<105553116266496ul, 4398046511104ul, 0ul, __sanitizer::SizeClassMap, __asan::AsanMapUnmapCallback>, __sanitizer::SizeClassAllocatorLocalCache<__sanitizer::SizeClassAllocator64<105553116266496ul, 4398046511104ul, 0ul, __sanitizer::SizeClassMap, __asan::AsanMapUnmapCallback> >, __sanitizer::LargeMmapAllocator >::Allocate(__sanitizer::SizeClassAllocatorLocalCache<__sanitizer::SizeClassAllocator64<105553116266496ul, 4398046511104ul, 0ul, __sanitizer::SizeClassMap, __asan::AsanMapUnmapCallback> >*, unsigned long, unsigned long, bool, bool) /var/tmp/portage/sys-devel/llvm-3.8.1-r2/work/llvm-3.8.1.src/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_allocator.h:1302
    #6 0x421dbf in __asan::Allocator::Allocate(unsigned long, unsigned long, __sanitizer::BufferedStackTrace*, __asan::AllocType, bool) /var/tmp/portage/sys-devel/llvm-3.8.1-r2/work/llvm-3.8.1.src/projects/compiler-rt/lib/asan/asan_allocator.cc:368
    #7 0x421dbf in __asan::asan_malloc(unsigned long, __sanitizer::BufferedStackTrace*) /var/tmp/portage/sys-devel/llvm-3.8.1-r2/work/llvm-3.8.1.src/projects/compiler-rt/lib/asan/asan_allocator.cc:718
    #8 0x4c0391 in malloc /var/tmp/portage/sys-devel/llvm-3.8.1-r2/work/llvm-3.8.1.src/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:53
    #9 0x7f4f0474e170 in jas_malloc /tmp/portage/media-libs/jasper-1.900.5/work/jasper-1.900.5/src/libjasper/base/jas_malloc.c:117:9
    #10 0x7f4f0474e170 in jas_alloc2 /tmp/portage/media-libs/jasper-1.900.5/work/jasper-1.900.5/src/libjasper/base/jas_malloc.c:141
    #11 0x7f4f04764b4f in bmp_getinfo /tmp/portage/media-libs/jasper-1.900.5/work/jasper-1.900.5/src/libjasper/bmp/bmp_dec.c:297:25
    #12 0x7f4f04764b4f in bmp_decode /tmp/portage/media-libs/jasper-1.900.5/work/jasper-1.900.5/src/libjasper/bmp/bmp_dec.c:132
    #13 0x7f4f0470ef39 in jas_image_decode /tmp/portage/media-libs/jasper-1.900.5/work/jasper-1.900.5/src/libjasper/base/jas_image.c:380:16
    #14 0x4f1686 in main /tmp/portage/media-libs/jasper-1.900.5/work/jasper-1.900.5/src/appl/imginfo.c:188:16
    #15 0x7f4f0381d61f in __libc_start_main /var/tmp/portage/sys-libs/glibc-2.22-r4/work/glibc-2.22/csu/libc-start.c:289
    #16 0x418e68 in _init (/usr/bin/imginfo+0x418e68)

Affected version: 1.900.5

Fixed version: 1.900.11

Commit fix:
https://github.com/mdadams/jasper/commit/65536647d380571d1a9a6c91fa03775fb5bbd256

Comment 2 Tomas Hoger 2017-03-29 19:38:05 UTC
Upstream bug report:

https://github.com/mdadams/jasper/issues/35


There is a mix of two issues here.  The test case provided by the reporter causes jasper to attempt to allocate large amount of memory.  That attempt fails on most systems, and the failure is handled gracefully.  However, when jasper is built with ASAN, ASAN terminates the process instead of allowing jasper to handle the allocation error.  This is not a problem for non-ASAN builds, and not something that should be called security.

Another concern is that jasper does not impose any upper limit on the amount of memory it allocates.  Smaller allocations than those triggered by the provided reproducer may still succeed and cause jasper to consume an excessive amount of memory.  Jasper upstream indicated they do not plan to introduce any arbitrary limit that would likely be unsuitable for certain use cases.  The fix introduced in version 1.900.11 via the commit linked above is addition of an experimental memory allocator that tracks total memory allocations done by jasper, and new API allowing applications to set the limit.  Command line utilities were updated to use the limit if jasper is compiled with this allocator enabled.

Upstream clearly marks this feature as experimental and notes it may be removed in future versions.  There's currently no plan to backport this fix to released Red Hat Enterprise Linux versions.