It was found that current mechanism of queuing the future messages, i.e. messages having greater sequence numbers that are to be processed later, is prone to DoS attack by memory exhaustion, when attacker can fill up the queue with lots of large messages that are never going to be used. Only up to 10 messages in the future can be buffered and queue gets cleared when the connection is closed, thus attacker can exploit this only with opening many simultaneous connections.
Created openssl101e tracking bugs for this issue:
Affects: epel-5 [bug 1369507]
Created openssl tracking bugs for this issue:
Affects: fedora-all [bug 1369505]
Created mingw-openssl tracking bugs for this issue:
Affects: fedora-all [bug 1369506]
Another related flaw, with no CVE though:
Covered now by OpenSSL upstream security advisory and fixed in versions 1.0.1u and 1.0.2i.
DTLS buffered message DoS (CVE-2016-2179)
In a DTLS connection where handshake messages are delivered out-of-order those
messages that OpenSSL is not yet ready to process will be buffered for later
use. Under certain circumstances, a flaw in the logic means that those messages
do not get removed from the buffer even though the handshake has been completed.
An attacker could force up to approx. 15 messages to remain in the buffer when
they are no longer required. These messages will be cleared when the DTLS
connection is closed. The default maximum size for a message is 100k. Therefore
the attacker could force an additional 1500k to be consumed per connection. By
opening many simulataneous connections an attacker could cause a DoS attack
through memory exhaustion.
OpenSSL 1.0.2 DTLS users should upgrade to 1.0.2i
OpenSSL 1.0.1 DTLS users should upgrade to 1.0.1u
This issue was reported to OpenSSL on 22nd June 2016 by Quan Luo. The fix was
developed by Matt Caswell of the OpenSSL development team.
This issue has been addressed in the following products:
Red Hat Enterprise Linux 6
Red Hat Enterprise Linux 7
Via RHSA-2016:1940 https://rhn.redhat.com/errata/RHSA-2016-1940.html