An issue was discovered in LibVNCServer through 0.9.11. rfbProcessClientNormalMessage() in rfbserver.c does not sanitize msg.cct.length, leading to access to uninitialized and potentially sensitive data or possibly unspecified other impact (e.g., an integer overflow) via specially crafted VNC packets.
Created libvncserver tracking bugs for this issue:
Affects: epel-7 [bug 1546859]
Affects: fedora-all [bug 1546860]
Created attachment 1400842 [details]
This prevents from the integer overflow.
However, it does not impose any length limit to prevent from allocating a lot of memory as I don't know what the sane limit should be.
Created attachment 1404775 [details]
Proposed fix to mitigate excessive memory use
The original patch fixes only the use of noninicialized memory. This second patch prevents from a DoS when a client request a valid but really big buffer size (2 GB) by limiting the acceptable size to 1 MB.
libvncserver-0.9.11-5.fc27 has been pushed to the Fedora 27 stable repository. If problems still persist, please make note of it in this bug report.
libvncserver-0.9.11-6.fc28 has been pushed to the Fedora 28 stable repository. If problems still persist, please make note of it in this bug report.
libvncserver-0.9.11-3.fc26 has been pushed to the Fedora 26 stable repository. If problems still persist, please make note of it in this bug report.
This issue has been addressed in the following products:
Red Hat Enterprise Linux 7
Via RHSA-2018:1055 https://access.redhat.com/errata/RHSA-2018:1055