Bug 1860364 (CVE-2020-14401)

Summary:	CVE-2020-14401 libvncserver: libvncserver/scale.c has a pixel_value integer overflow
Product:	[Other] Security Response	Reporter:	Marian Rehak <mrehak>
Component:	vulnerability	Assignee:	Red Hat Product Security <security-response-team>
Status:	CLOSED WONTFIX	QA Contact:
Severity:	low	Docs Contact:
Priority:	low
Version:	unspecified	CC:	jadahl, negativo17, oholy, rdieter
Target Milestone:	---	Keywords:	Security
Target Release:	---
Hardware:	All
OS:	Linux
Whiteboard:
Fixed In Version:	libvncserver 0.9.13	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2021-11-02 17:20:51 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1860365, 1860366, 1861159, 1861160
Bug Blocks:	1860332

Description Marian Rehak 2020-07-24 11:27:50 UTC

An issue was discovered in LibVNCServer before 0.9.13. libvncserver/scale.c has a pixel_value integer overflow.

Upstream commit:

https://github.com/LibVNC/libvncserver/commit/a6788d1da719ae006605b78d22f5a9f170b423af

Comment 1 Marian Rehak 2020-07-24 11:28:23 UTC

Created libvncserver tracking bugs for this issue:

Affects: epel-7 [bug 1860366]
Affects: fedora-all [bug 1860365]

Comment 2 Todd Cullum 2020-07-27 23:48:21 UTC

in libvncserver/scale.c, rfbScaledScreenUpdateRect() has `pixel_value += (srcptr2[z] << (8 * z));` which performs a shift operation on a 64-bit value using an 8-bit variable, which could cause an unsigned integer wrap. A few lines down, there is a write: `pixel_value = ((red & redMax) << redShift) | ((green & greenMax) << greenShift) | ((blue & blueMax) << blueShift);`. In the event of an unsigned wrap, the write could be incorrect and potentially cause a crash.