Bug 2540205

Disclaimer: Community trackers are created by Red Hat Product Security team on a best effort basis. Package maintainers are required to ascertain if the flaw indeed affects their package, before starting the update process.

A vulnerability was found in the BusyBox TLS implementation (networking/tls_pstm_montgomery_reduce.c). A unit confusion error exists in the buffer allocation for the Montgomery reduction operation. The code calls xzalloc(2*pa+1) where pa is measured in pstm_digit units (4 or 8 bytes each), but the allocation treats this value as a byte count. This results in an allocation approximately 4x to 8x smaller than required.

When a TLS client sends a crafted ClientKeyExchange message with an all-zeros payload, the RSA decryption path triggers the Montgomery reduction, which writes digit-sized elements beyond the allocated buffer boundary. This constitutes a pre-authentication out-of-bounds heap write.

The confirmed impact is a pre-authentication denial of service (crash). While the heap buffer overflow is theoretically exploitable for remote code execution, this was not demonstrated. On Fedora, system-level mitigations including ASLR, PIE, full RELRO, and SELinux confinement make practical code execution extremely unlikely.