There was a unsigned -> signed integer cast issue in cmsAllocGamma in src/cmsgamma.c. The argument to this function - nEntries - may be read from the file and not validated before cmsAllocGamma is called. As nEntries in cmsAllocGamma is signed integer, its value may possibly be negative and can result in an insufficient memory allocation. This issue was fixed upstream in 1.17. Upstream CVS commit: http://lcms.cvs.sourceforge.net/viewvc/lcms/lcms/src/cmsgamma.c?view=diff&r1=1.16&r2=1.17 References: http://www.openwall.com/lists/oss-security/2008/11/28/3
Created attachment 325025 [details] Patch used in SuSE security updates This was extracted from SuSE liblcms-1.15-32.src.rpm. Original name of the patch was lcms-gamma-overflow.patch.
Affected lcms versions are currently in Red Hat Enterprise Linux 5 and Fedora EPEL 4 (both based on upstream 1.15).
CVE id CVE-2008-5317 was assigned to this issue: Integer signedness error in the cmsAllocGamma function in src/cmsgamma.c in Little cms color engine (aka lcms) before 1.17 allows attackers to have an unknown impact via a file containing a certain "number of entries" value, which is interpreted improperly, leading to an allocation of insufficient memory.
https://www.redhat.com/security/data/cve/CVE-2008-5317.html