A posting to Full Disclosure mailing list indicated that multiple flaws (null pointer dereference, off-by-one and others resulting in DoS/crash) were in BN (multiprecision integer arithmetics) part of openssl between 0.9.8k-1.0.1e and included proof of concept code. It is not yet certain whether these are legitimate flaws or not. Oss-sec posting: http://seclists.org/fulldisclosure/2013/Dec/8
All of the above code segments use the BIGNUM struct/API in a wrong way. For example, memory is allocated for the data portion of the BN structure using the following: x->d = (BN_ULONG *) malloc(1); and later other BN_ functions are called to process it, however all these functions expect the data portion of the structure to be in multiple of BN_ULONG. malloc(1) is supposed to return a pointer to a memory area of size 1 byte. (This may not happen under glibc normally as it may return a pointer to a larger area of memory) So when a BN_ function is called, it results in a heap OOB read/write, similar to what the reporter mentions. Changing the code to use malloc as follows fixes the problem: x->d = (BN_ULONG *) malloc(1 * sizeof(BN_ULONG)); Note: As mentioned above, due to various reasons malloc(1) may not return a memory area of 1 byte, but more than that. On a typical x86_64 machine it returns ~ 24 bytes. Due to this it may not cause an OOB read/write in the above test case. But that is due to glibc's implementation of malloc()
I looked at all the issues in the fd email. The ones using malloc() are similar to the ones described in comment #2. The one using "BN_sqr(o, z, ctx);" looks like a very minor bug, i filed an upstream bug and a github pull request. This has no security angle to it though. http://rt.openssl.org/Ticket/Display.html?user=guest&pass=guest&id=3410 https://github.com/openssl/openssl/pull/140