A flaw was found in OpenSSL's DTLS protocol implementation when handling out-of-sequence handshake messages. Quoting Robin Seggelmann's description in the upstream bug report: http://rt.openssl.org/Ticket/Display.html?id=1838&user=guest&pass=guest Whenever a handshake message arrives with an unexpected sequence number, it is passed to the function dtls1_process_out_of_seq_message(). This function discards the data if the sequence number is lower than the expected value and buffers it, if is a future message. When discarding, the message fragment length remains 0 which indicates that nothing has to be buffered. Due to a misplaced if condition to check the length, sometimes fragments with no data but with the length of the dropped message are buffered. This causes a bus error when processing later. NULL pointer dereference resulting in DTLS server crash can happen in dtls1_retrieve_buffered_fragment() when doing memcpy from frag->fragment. Upstream patch: http://marc.info/?l=openssl-cvs&m=123871365032446&w=2 Issue is fixed in 1.0.0-beta2, not yet in the latest 0.9.8 version to date - 0.9.8k.
Upstream CVS commit link: http://cvs.openssl.org/chngview?cn=17958
This issue did not affect versions of openssl as shipped in Red Hat Enterprise Linux 3 and 4. This issue affects openssl version as shipped in Red Hat Enterprise Linux 5 and it will be addressed in the openssl packages update in Red Hat Enterprise Linux 5.4. There is no update planned before than, as both DTLS specification and OpenSSL's implementation is still in development and unlikely to be used in production environments. There is no component shipped in Red Hat Enterprise Linux 5 using OpenSSL's DTLS implementation, except for OpenSSL's testing command line client - openssl.
This issue has been addressed in following products: Red Hat Enterprise Linux 5 Via RHSA-2009:1335 https://rhn.redhat.com/errata/RHSA-2009-1335.html